Multi-Agent Orchestration: Design Handoffs and Recovery Before Adding Roles

Splitting an agent into a planner, specialist, and reviewer does not automatically improve reliability. If all three repeatedly paraphrase the same summary, an error can acquire more confidence with each handoff. Multiple agents are useful when the task can draw on relatively independent evidence streams and the coordinator can identify disagreements.

For a drop in event enrichment match rate, input ingestion, dimension-table updates, and job health can be investigated in parallel. The final causal assessment still needs consistent definitions; it cannot be settled by a vote.

Define deliverables when splitting the task

The ingestion investigation examines publisher input volumes, missing fields, and version changes. The dimension investigation checks lookup hit rates, update delays, and key coverage. The runtime investigation checks errors, backpressure, and deployment records. All assignments should share an incident window and match-rate definition, with explicit tools, deadlines, query budgets, required evidence, and stopping conditions.

Anthropic’s engineering article describes parallel exploration in a multi-agent research system while also discussing coordination and resource costs. The relevant idea here is task isolation; its internal benchmark gains are not a performance forecast for another business. See the multi-agent research system .

A compact task contract could be:

task_id: association-incident/input-check
window: [2026-11-06T09:00:00Z, 2026-11-06T10:00:00Z]
question: Are missing join keys concentrated in one input version?
allowed_tools: [query_quality_metrics, read_schema_history]
budget: {queries: 6, deadline_seconds: 90}
deliverable: evidence_bundle_v1
stop_when: hypothesis confirmed, rejected, or budget exhausted

Use an ordinary concurrent workflow when fixed assignments are sufficient. Add model-driven planning when new investigative branches must be generated from intermediate evidence. This avoids spending calls on coordination that the task does not need.

Hand off facts, hypotheses, and open questions

“I think it is a schema change” is not a sufficient handoff. Include at least:

{
  "task_id": "association-incident/input-check",
  "status": "completed",
  "facts": [{
    "claim": "In the demo data, v3 has a higher missing-key rate than v2",
    "evidence_id": "query-demo-18",
    "metric_version": "missing_key_v1",
    "population": "authorized sample"
  }],
  "hypotheses": ["A field-mapping change may affect joins"],
  "unknowns": ["Actual failed-join samples have not been checked"],
  "artifacts": ["evidence://incident/input/18"]
}

Evidence references point to access-controlled queries and result snapshots. The summary is a guide to those artifacts. The coordinator must inspect key evidence and validate time ranges, sample coverage, and metric definitions. Two agents agreeing can still reflect one shared, incorrect query; that is not two independent pieces of evidence.

When results conflict, check definitions first. One agent may group by event time and another by ingestion time. Late data alone can produce opposite trends. Preserve both results and request a targeted check instead of asking a third model to choose the more convincing story.

Orchestration state is operational data

Persist task states such as PENDING → RUNNING → SUCCEEDED / FAILED / TIMED_OUT, together with attempt number, input hash, artifact location, and lease expiry. A retry keeps the logical task ID and increments the attempt. The coordinator accepts only valid results for the current input version.

Worker restarts and duplicate messages can cause repeated execution. Read-only investigations are often safe to rerun, but ticket creation and configuration changes need separate idempotent operation IDs. A framework checkpoint does not give an external API exactly-once behavior. LangGraph explicitly discusses retries, idempotency keys, and unknown outcomes caused by missing acknowledgements. See fault tolerance .

Complete a task:
  Verify task_id, input_hash, and attempt are still valid
  Write the evidence bundle to an immutable artifact location
  Conditionally move RUNNING -> SUCCEEDED and save artifact hash
  Return the same result for duplicate completion notifications

Recover a task:
  Find unfinished tasks whose leases have expired
  Check for external actions with unknown outcomes
  Create a new attempt if safe; otherwise reconcile the outcome

Do not use one large shared conversation as the only state store. Structured state supports recovery and validation; retain the conversation for debugging. Keep sensitive samples in authorized storage and hand off controlled references instead of copying samples into every agent’s context.

Bound concurrency and the strength of conclusions

A shared query service can become the bottleneck. Apply global concurrency limits, per-source quotas, and a total task budget. Run inexpensive bucket-level checks first, then investigate abnormal buckets. A slow branch can return partial evidence at its deadline, but the final report must identify uncovered areas. A timeout does not mean no anomaly was found.

To decide whether multiple agents are worthwhile, compare the single-agent and parallel designs on the same task set: successful completion, coverage of critical evidence, incorrect assertions, P95 latency, and cost per successful task. Faster production of a longer answer is not enough if its evidence becomes harder to trace.

The core of multi-agent engineering is a clear boundary for each branch, a verifiable handoff, and recoverable failure. With those in place, better model capability can translate into more dependable results.

References

• Anthropic: How we built our multi-agent research system , parallel exploration and coordination.
• Anthropic: Building effective agents , fixed workflows versus dynamic orchestration.
• LangGraph: Fault tolerance , retries, timeouts, and idempotency boundaries. This article does not depend on a particular API version.