Check whether review can finish
An AI system proposes a refund. A person must approve it before money moves. The model passes its quality checks, but that result says nothing about whether a reviewer can finish before the refund deadline.
Add a capacity gate to the launch checklist. This is a rule that allows work to enter only when qualified reviewers have time to complete the required checks. Keep the quality check too.
The engineering manager needs a launch schedule that the review team can support. The operations lead needs protected review time. The product owner needs a traffic limit tied to the decision deadline.
The worked example below holds the model-quality result and workload constant. A quality-only gate allows all three staffing schedules. The capacity gate rejects two schedules and permits the adequately staffed control.
Separate the review routes
Start with the approval rule for each action. Record who may review it, what they must check and when the decision is due. Keep the action blocked until every required approval is complete.
Mandatory review happens before the action. Retrospective sampling means checking selected cases after an action. Reducing that sample cannot remove a required approval before an action.
An escalation is a case that needs a reviewer with extra authority or skills. Count each case once, but count every review step it needs. A case that needs two reviews consumes time in both reviewer groups.
For this illustrative refund workflow, intake assigns each case to one of two routes. A standard case needs one standard reviewer. An escalated case goes directly to a specialist whose review includes the initial checks.
These routes do not overlap. The example has no separate retrospective sample. If sampling uses the same staff in your workflow, reserve its time before calculating launch capacity.
Keep every mandatory route behind review
- Proposed workProposed arrivalAdmission check
- Admission checkStandard slot availableStandard review
- Admission checkSpecialist slot availableSpecialist review
- Admission checkNo feasible slotNo execution
- Standard reviewReview finishesCompleted review
- Specialist reviewReview finishesCompleted review
- Completed reviewApproved and not expiredApproved action
- Completed reviewRejected or expiredNo execution
- Standard reviewRemaining workDeadline monitor
- Specialist reviewRemaining workDeadline monitor
- Deadline monitorReduce or stop intakeAdmission check
- Deadline monitorDeadline expires before completionNo execution
Read this diagram as text
Intake identifies the route and deadline. The admission check reserves time with standard reviewers or qualified specialists. Work without a feasible slot stays unexecuted. Both review routes lead to a completed decision. Only approved work that has not expired can execute. Rejected or expired work stays unexecuted. A deadline monitor checks both queues and updates admission limits. It also keeps expired work from executing.
Fill in the illustrative worksheet
All values below are illustrative. They describe one batch of refund proposals, not results from a company rollout. The model-quality check has the same passing result in every staffing case.
The planning window runs from 09:00 to 09:30. Eleven new cases arrive together at 09:00: eight standard cases and three escalated cases. Their required decisions are due by 09:30.
Three older cases are already waiting at 09:00. Two need standard review and one needs specialist review. All three are due by 09:10 and each still needs its full review.
Each assigned reviewer has 30 scheduled minutes. Five minutes belong to other duties from 09:25 to 09:30. The reviewer therefore has one usable review window from 09:00 to 09:25.
Reviews take five uninterrupted minutes for a standard case and ten for an escalated case. These sample times include reading, deciding and recording the result. A reviewer handles one case at a time.
The arrival batch is fixed and there are no further arrivals in this window. Routing happens at intake. A standard reviewer cannot cover specialist work and specialists do not cover standard work in this schedule.
| Measure | Route | New cases | Older cases | Minutes per case | Total review minutes |
|---|---|---|---|---|---|
| Standard | Standard review | 8 | 2 | 5 | (8 + 2) × 5 = 50 |
| Specialist | Specialist review | 3 | 1 | 10 | (3 + 1) × 10 = 40 |
Compare the same workload under three schedules
Normal coverage assigns two standard reviewers and one specialist. Standard capacity is 2 × 25 = 50 minutes. Specialist capacity is 1 × 25 = 25 minutes against 40 minutes of required work.
Both standard reviewers take an older case first. Those cases finish at 09:05. Each reviewer then completes four new cases by 09:25. Every standard case meets its deadline.
The specialist completes the older case at 09:10 and one new case at 09:20. Another ten-minute review cannot fit before other duties start at 09:25. Two new specialist cases would miss their deadlines.
The staffing-loss case removes one standard reviewer before 09:00. The remaining reviewer finishes both older cases by 09:10 and three new cases by 09:25. Five new standard cases remain, alongside the two specialist cases.
The control assigns two standard reviewers and two specialists from the existing staff. It protects their review time without moving another review obligation out of the worksheet. No new hires are assumed.
In the control, the first specialist handles the older case and one new case by 09:20. The second handles the other two new cases by 09:20. Standard work still finishes by 09:25.
| Measure | Illustrative schedule | Standard / specialist minutes | Quality-only gate | Capacity gate | Work that would remain at 09:30 |
|---|---|---|---|---|---|
| Normal coverage | 2 standard reviewers, 1 specialist | 50 / 25 | Allow: quality passes | Reject: specialist shortfall | 2 specialist cases; 20 minutes |
| Staffing loss | 1 standard reviewer, 1 specialist | 25 / 25 | Allow: quality passes | Reject: both groups short | 5 standard cases and 2 specialist cases; 45 minutes |
| Adequately staffed control | 2 standard reviewers, 2 specialists | 50 / 50 | Allow: quality passes | Allow: all decisions fit | 0 cases; 0 minutes |
Check timing as well as total minutes
The normal schedule has 75 usable staff minutes against 90 minutes of review demand. That total already shows a shortfall. It does not explain which cases will miss their deadlines.
An individual schedule explains why 20 minutes remain rather than 15. The specialist has five unused minutes, but the next review needs ten uninterrupted minutes. Spare standard-review time would not remove this skill limit.
The control has 100 usable minutes and needs 90. Its explicit assignment shows that the older cases finish by 09:10 and new cases finish before 09:30. Total minutes alone would not prove either result.
Average utilization means the share of available staff time used by work. A low average can still hide a late arrival, a short staffing gap or work that only one person can review.
Keep unfinished work in the next interval. In the normal schedule, that would mean carrying 20 specialist minutes into 09:30 with the original deadlines already missed. Moving the row must not reset those deadlines.
Worksheet fields and the launch decision rule
Use one row per case and review step. Record the case identifier, route, arrival time, original deadline, remaining review minutes and eligible reviewers. For work already waiting, use its remaining work rather than counting a full review again. Keep a separate calendar for each reviewer. Subtract breaks, meetings, other duties and protected sampling time from scheduled minutes. Record where these deductions fall on the clock. Never count the same person twice across reviewer groups. For each interval, closing unfinished minutes equal opening unfinished minutes plus arriving review minutes minus work actually performed. Keep remaining work attached to its case. Also count cases without completed decisions, since minutes alone can hide unfinished reviews. Assign the earliest deadlines first within each eligible group. Fit each review into a real availability window. Do not pause a review across a break unless the handling-time assumption explicitly allows that. Include handoff time when another person takes over. When a case needs standard review followed by specialist review, create two linked steps. The second step cannot start before the first finishes. Reserve both steps before allowing the action to depend on this route. Allow launch only when quality passes and every required review has an eligible assignment that ends by its deadline. Include existing work and all proposed arrivals. Reject a schedule with a missed deadline, overlapping staff assignments or an unresolved review requirement. A simple deadline-first schedule is a practical check, not a proof that every rejected schedule is impossible. With shared skills or linked steps, try another valid assignment before declaring a staffing shortfall. In this example, the rejected schedules lack enough qualified minutes regardless of assignment. Keep the stated arrival pattern and handling times beside the decision. Recalculate when either changes. A schedule that fits these inputs does not establish that every possible burst will fit.
Turn the worksheet into an admission limit
Admission control means limiting how much work may enter a workflow. With normal coverage, this example can accept at most eight new standard cases and one new specialist case at 09:00. Older work keeps priority.
That is a limit of nine new cases only when the route counts are known. It is not permission to accept any nine cases. A batch with two new specialist cases already exceeds the declared specialist schedule.
A smaller starting batch could contain four standard cases and one specialist case. Including older work, that needs 30 standard minutes and 20 specialist minutes. It leaves time unused without removing any required review.
If routing is unknown before admission, reserve for the possible specialist path or hold intake until routing is known. Do not turn an average escalation percentage into an unconditional traffic allowance.
Test changes in review mix as separate schedules. For example, move one of the eight new standard cases into specialist review. Specialist demand becomes 50 minutes and even the two-specialist control cannot finish five ten-minute cases before 09:25.
Reject that changed batch or adjust its arrival schedule. Do not shorten a mandatory review to preserve the original traffic target.
Stop new work before approvals become late
Give the engineering rollout owner authority to lower admission or stop the affected route. The operations review lead maintains handling times, reviewer qualifications and coverage. Put actual names against both roles in the launch checklist.
Before each admission, check the live queue against the remaining staffing calendar. Reserve review time for work already accepted. Do not allocate that time again to a new arrival.
Recalculate when coverage drops, review policy changes or a model update changes the route mix. A slower review also changes the schedule. The owner should stop new admissions when required work no longer fits.
Track queue age, meaning time since a case arrived, alongside remaining review time. A case near its deadline needs more attention than its position in a queue suggests.
For a specialist case due at 09:30, the example's availability window requires a start by 09:15. Waiting until 09:20 would leave ten clock minutes but only five staffed minutes.
Use an earlier warning to give the owner time to respond. For this example, a specialist case still waiting at 09:10 can trigger a schedule check. Choose the response margin for your actual operating delay.
When review cannot finish, keep the action blocked. Decline new intake or defer it only under a service rule that allows the delay. Preserve the original deadline for work already accepted.
A manual queue is not free capacity. If another team takes deferred work, include its qualifications, deadlines and existing workload in the same calculation. Otherwise the transfer only moves the shortage.
Run the launch check on Monday
Bring the engineering rollout owner and operations review lead to a tabletop exercise. This is a discussion using a worksheet rather than live traffic. Write down the review routes, staff calendars and decision deadlines first.
Walk through the normal schedule, remove one reviewer and then run the adequately staffed control. Keep the quality result and case list unchanged. Check each completion time rather than voting on whether staffing feels sufficient.
The test fails if the gate admits either shortfall schedule. It also fails if it cannot distinguish the adequately staffed control using the declared inputs. The control must pass because its required reviews fit.
Record preparation time and participant time separately. The arithmetic does not establish a cost saving or a production improvement.
Add the worksheet, route limits and stop authority to the launch checklist. For the normal schedule here, the full 11-case batch stays blocked. Only a smaller batch with a feasible review schedule may enter.