Skip to content
Context Windowby Alex Janjic
Menu
contextwindow.us/posts/the-ai-operating-model-in-practice-part-2Article

The AI Operating Model in Practice - Give product teams ownership of the agents they ship - Part 2

A refund assistant can pass permission checks and still leave customers with wrong proposals or unresolved payments. Give the product team an owner who can stop intake, fix the workflow and see every affected case through to an answer.

Engineering management
Source links stay with the relevant section
A refund case follows one main path to a resolved tray while side paths lead to tool repair, refund review and access control. A yellow marker shows where new cases can be paused.

Give product teams ownership of the agents they ship

In this fictional example, a refund assistant keeps proposing full refunds for orders that only qualify for partial refunds. The shared refund tool works and permission checks pass. Operations handles only cases sent to its queue. Meanwhile, a customer waits and the assistant repeats the wrong proposal.

The refund product lead must own this workflow. Someone on that team needs the authority to stop new cases, correct the proposals and follow each affected customer case to an answer. A shared platform can help the team do that work. It cannot take over the customer outcome.

A permission check can stop an unapproved refund. It cannot show that the proposed amount was right, that a reviewer had time to check it or that a payment completed after a timeout. Those questions need separate checks and named owners.

Name the owner of the customer outcome

The refund product lead is the accountable service owner in this example. The lead decides which requests the assistant may handle and what counts as a correct proposal. The same lead decides when the workflow can take more traffic. If a case goes wrong, the lead owns the customer response even when a shared component caused the fault.

The platform team provides shared identity checks, access controls, tool adapters, traces and tools for checking agent results. A trace is a record of the steps a request took. Platform staff find and repair defects in those shared parts. They do not decide whether a refund amount meets product policy.

Refund specialists define the refund rules. They supply representative cases and mark proposals as correct or incorrect. Product quality assurance (QA) owns the set of cases used to check a release. QA adds confirmed failures to that set. Product staff use the cases to set acceptance criteria, meaning the checks a release must pass, and decide whether to allow more traffic.

The security policy lead owns access rules and decides exceptions to those rules. Other specialists decide their own policies when the workflow touches their area. The product team cannot expand a tool's access on its own to clear a case. A policy exception does not transfer responsibility for an unresolved customer refund to security.

A tool owner and a service owner have different jobs

Suppose a shared order-reading tool returns an old balance. The platform tool owner checks the adapter and the data it receives. That owner works with the downstream service owner and fixes or disables the shared tool. The refund product lead stops or narrows the affected route and owns the queue of cases that may have used the old balance. After the repair, the product team checks those cases again.

Now suppose the tool returns the right order, but the assistant keeps proposing the wrong amount. The product team changes the workflow or its acceptance criteria with the refund specialist. QA adds those failures to the release cases. Platform staff can show where the proposal went wrong without taking over the product decision.

Where a refund failure goes

Product incident ownerTriage cases
Product service ownerCustomer outcome and restart
Shared-tool ownerRepair shared tool
Domain expert and QAReview refund cases
Security policy ownerDecide access exceptions
  1. Product incident ownerEscalate affected casesProduct service owner
  2. Product service ownerRequest tool investigationShared-tool owner
  3. Shared-tool ownerReturn fix and evidenceProduct service owner
  4. Product service ownerReview wrong proposalsDomain expert and QA
  5. Product service ownerRequest policy decisionSecurity policy owner
Read this diagram as text

The product incident owner triages affected cases and escalates them to the product service owner. The service owner remains responsible for each customer and decides when intake can restart. For a shared-tool fault, the service owner asks the shared-tool owner to investigate. The tool owner returns a fix and evidence to the service owner. For wrong refund proposals, the service owner asks the domain expert and QA to review the decision rules and cases. For an access exception, the service owner asks the security policy owner for a separate policy decision.

One service owner keeps responsibility for the customer while tool, refund and policy questions go to the people who can decide them.

Keep each decision with one owner

The incident owner runs the response and can stop affected traffic at once. The platform tool owner decides how to repair a shared adapter. Security decides a policy exception. The product service owner decides whether the refund workflow can resume and remains responsible for the customer case throughout. Consulting other people does not divide that decision.

[Part 1, Enforce agent permissions before a tool runs](https://contextwindow.us/posts/the-ai-operating-model-in-practice-part-1) shows a scripted local refund example. Its permission tests include an allowed request and rejected requests with missing or mismatched approval, expired approval, cross-tenant access and direct service calls. Those tests show what the sample's permission boundary did. They do not show whether a refund proposal was correct or whether a downstream payment completed.

Keep a one-page ownership record

Use one record for each agent workflow rather than one for each tool. First write down the roles. Then name a person and deputy for each role in the team's working record. The person who can stop intake must be reachable. Reviewers must have time to do the work before new cases arrive.

The next table is a reusable blank record. Copy it for one workflow and replace the directions in the second column. Record actual review hours and time set aside for cases and release decisions.

Blank one-page agent ownership record. Fill every field for one workflow before it takes real cases.
MeasureFieldFill in for your workflow
PurposePurposeWrite the customer task and the limit of the service.
Allowed actionsAllowed actionsList reads, proposals and approved writes separately.
Data scopeData scopeState which customers and records the assistant may use.
Accountable service ownerAccountable service ownerName the role that owns outcomes and restart decisions.
Quality criteriaQuality criteriaWrite the checks a release must pass.
Evaluation-data ownerEvaluation-data ownerName who keeps labeled release cases current.
Tool-policy ownerTool-policy ownerName who decides permitted tool scopes and exceptions.
Incident ownerIncident ownerName who triages failures and assigns case handlers.
Review coverageReview coverageName qualified reviewers and the hours they are available.
Budget ownerBudget ownerName who decides spending limits and what happens at the limit.
Stop authorityStop authorityName who can pause intake now and who can restart it.
Escalation routeEscalation routeShow how cases reach the service, tool and policy owners.
Reserved operating timeReserved operating timeSet calendar time for case checks and release decisions.

Filled record: fictional refund assistant

Every role, hour and amount in this record is illustrative. The record shows who would make each decision. It does not describe an operating service.

Illustrative ownership and authority for one fictional refund workflow. USD amounts are a monthly model and tool spending example, not measured cost.
MeasureFieldIllustrative decision
PurposePurposeHelp support staff decide customer-requested refunds. Never claim completion without a matching receipt.
Allowed actionsAllowed actionsRead the signed-in tenant's order; propose an amount and reason; request an approved refund through the business service.
Data scopeData scopeThe current tenant's order, payment and support case only. No other tenant or unrelated customer history.
Accountable service ownerAccountable service ownerRefund product lead; owns the workflow, rollout decision and affected customer cases.
Quality criteriaQuality criteriaNo wrong amount or reason in the release cases; no unauthorized write in mandatory permission cases; every unknown outcome stays pending.
Evaluation-data ownerEvaluation-data ownerProduct QA lead; maintains labeled cases with a refund specialist and adds each confirmed failure to the release set.
Tool-policy ownerTool-policy ownerSecurity policy lead; owns permitted scopes and exceptions. The platform tool owner implements shared controls.
Incident ownerIncident ownerRefund product on-call lead; triages alerts, assigns a case handler and coordinates the response.
Review coverageReview coverageSupport operations lead schedules qualified refund reviewers on workdays, 09:00-17:00. Outside coverage new proposals wait.
Budget ownerBudget ownerRefund product lead; illustrative model and tool cap of USD 500 per month, alert at USD 400, stop new intake at the cap.
Stop authorityStop authorityRefund product lead decides pause and restart. Product on-call lead can stop traffic at once. Security can block access that breaks policy.
Escalation routeEscalation routeCase handler to incident owner to product service owner; shared-tool defects to platform tool owner; policy exceptions to security.
Reserved operating timeReserved operating timeService owner: 30 minutes each workday for open cases and one hour weekly for release decisions. QA: 90 minutes weekly for failures.

Use coverage to decide how many cases to admit

The support operations lead supplies the reviewer schedule. The product service owner uses it to decide whether to admit new cases. If coverage ends or the queue no longer fits the time available, the product lead stops new intake. Existing cases still need handlers and a response.

[Make human review capacity an AI rollout gate](https://contextwindow.us/posts/make-human-review-capacity-an-ai-rollout-gate) explains how to check a review schedule against incoming cases and customer deadlines. A calendar entry alone is not proof that the queue can be cleared on time.

How to use the ownership record for a stop or restart decision

For a wrong proposal, record the affected cases, the route that sent them and the person who will answer each customer. Have the refund specialist identify the correct amount. QA adds the failure to the release cases. The service owner checks the corrected proposals and confirms reviewer time before restarting intake. For a timeout after an approved request, keep the case pending until the downstream business record or a matching receipt resolves the original attempt. Record the case handler and the customer's response deadline. The product service owner decides when this route can resume. The incident owner coordinates both responses; the shared-tool owner repairs any shared fault. An access change goes to the security policy lead for a separate decision.

When the assistant keeps proposing the wrong refund

In the fictional example, the on-call incident owner groups repeated wrong proposals and gives every affected case a handler. That owner can stop the affected route immediately. The product service owner tells support what to say to customers and pauses expansion while the team finds the cause. The refund specialist checks the policy and decides the correct amount for each case.

Product QA adds confirmed wrong proposals and close calls to the release cases. Backend engineers check the order data, the assistant's proposal and the business service's decision as separate steps. If the shared tool supplied an old balance, the platform tool owner investigates and repairs it. If the balance was right, the product team fixes its workflow and acceptance criteria.

Security joins only if the proposed repair changes data access, approval rules or exception policy. The security lead decides that question. The product service owner still decides whether the corrected workflow passes its checks and can resume. Reviewers need scheduled time to revisit the affected cases. A passing permission test does not make repeated wrong proposals acceptable.

When the tool fails after an approved refund

Consider another fictional case. The assistant requests an approved refund. The downstream payment service may have saved it, but the reply times out. The incident owner marks the case pending, stops automatic retries that could send another refund and gives a case handler the task of checking the outcome. Support tells the customer the refund is being checked, not that it failed or succeeded.

The platform tool owner investigates the adapter and asks the downstream service owner for its business record or a matching receipt. A local permission check only shows that the request was allowed. It does not show that the payment service finished the refund. If the original outcome cannot be verified, the case stays pending. The team does not send a new refund to clear the queue.

The product service owner owns the unresolved case, the customer's response deadline and the decision to keep this route stopped. The incident owner coordinates the case handler. Platform staff repair the tool and the downstream service owner checks its records. Security joins if the repair requires different access. The product service owner resumes intake only when the cause is fixed, the case is reconciled and review coverage is available.

A small team still needs all the decisions

One person on a small team may act as both product service owner and on-call incident owner. A backend engineer may maintain a tool adapter and its traces. QA may work with a support specialist to keep refund cases current. This can work if everyone knows which decision they own, has a deputy for absence and has time for the job.

As more teams ship agents, a platform team can provide shared identity, access controls, records of agent steps and tools for checking results. Each product team still owns its customer outcomes, acceptance criteria, rollout and budget. Refund specialists can review cases across teams without taking over their incident queues. Security can keep one route for access decisions. These roles do not require a new department or a fixed hiring target.

The team can learn these skills while doing the work. Backend engineers can pair on permission checks, receipt lookup and traces. QA can turn real failures into release cases with refund specialists. Product staff can read those cases and decide when intake should stop. The refund specialists can explain why an amount that looks reasonable is wrong. Each activity needs time on the calendar. A role name will not clear a queue.

Use the next planning meeting

Pick one agent workflow and fill the blank record together. Assign the service owner and a deputy by role. Reserve actual time for review and case reconciliation, which means checking an uncertain outcome against the business record.

Walk through one wrong proposal and one timeout without a receipt. Ask who can stop intake, who investigates the tool and who stays with the customer until the case is resolved. If a decision has no owner or the review time does not fit, keep that route closed until the team fixes the gap.

Context Window dispatch

Practical AI engineering you can actually use

Working code, experiments, and production lessons. Published when there is something worth sending, never padded with AI news.

Confirmation is required. Read the privacy policy. Unsubscribe at any time.

The email edition is preparing to launch. No campaigns are being sent yet.