What Breaks When You Split a Job Across AI Agents

Abstract. Five AI agents worked an invented bank fraud case in order: intake, investigation, decision, operations and resolution. Each had fixed instructions and only the tools its job needs, and each received only the work of the agent before it. Outcomes were graded from the bank system's own log. Across five runs and 1,020 chains, I varied how agents passed work on, what the simulated customer said, and whether one agent's work was wrong.

In the natural setup, one agent beat every chain. A single agent with every tool completed 59 of 60 cases; a chain passing strict JSON records completed 44, and a chain passing free text 33. Most of the strict chain's shortfall came from one failure, a code that arrived without its definition.

The biggest gain came from a list of fields for each agent to fill in. Strict records came with that list and free text came with none. Giving free-text agents the same list raised completion from 17 to 49 of 60, and for Sonnet it matched JSON exactly (30 of 30). JSON helped the weaker Haiku further (28 vs 19 of 30), though that difference did not survive correction for multiple tests. Validating the records, explaining the fields, and addressing the work to the next agent made no measurable difference.

In one run the customer only said her number had been moved to another carrier, without saying what to do. Chains whose form had a field for blocked channels avoided the hijacked number in 299 of 300 runs. With nothing on the form asking, free-text chains contacted it in 10 of 60 runs, and a single Haiku agent did so in 7 of 30.

Coders who did not know the outcomes found a needed detail dropped in 16 of 43 failed free-text chains and in none of 43 completed ones. They also found conditions nobody could meet in 40 failed chains, and in 23 completed ones.

Two failures had nothing to do with format. A record said credit_condition: written_confirmation, and the agent that pays had never read the policy that defines it; it turned down the customer's plain written statement, and in 13 of 60 chains she was never paid. And a planted "no account takeover" or "fee not approved" reached the customer in 59 of 60 chains, while a planted mistake that a later agent could check against its own data was caught all 30 times.

Introduction

A common way to build an agent system is to give each agent one job and its own tools, then chain them: one agent gathers information, the next investigates, a third decides, others act. Each agent's finished work becomes the next agent's input. This mirrors how organizations already divide work, and it limits what any single agent can touch.

It also creates handoffs. Recent work shows that text passed through chains of language models drifts (section 2). An operator cares less about the text than about the result. Does the agent at the end of the chain do the right thing to the customer, and finish the job?

This study measures that in a flow built the way a production system would be built. Each agent's procedures are fixed by whoever built the system, and what moves between agents is work: a claim, findings, a decision, a record of actions. The questions:

1. Does splitting a job across agents change whether it gets done, compared with one agent doing all of it?
2. Do structured handoffs do better than free text? If so, which part helps: the format, a list of fields, or validation?
3. Can a chain make a judgment the customer never spells out?
4. When a chain fails, where does it go wrong? Is a detail lost, changed, or wrapped in conditions?
5. When one agent's work is wrong, does anyone downstream catch it?

Related work

Perez et al. run transmission chains in which each model rewrites the previous model's text, and find that texts drift toward stable "attractors" in toxicity, positivity and length.¹ Mohamed et al. show that distortion accumulates over iterated generation, and that constrained prompting reduces it without removing it.²

In multi-hop agent relays, a 2026 study of twelve facts re-encoded over six hops finds that a strong relay keeps nearly all of them while a weak one loses many more.³ A study of multi-agent planning reports delegation accuracy falling from 90.7% with one stage to 22.5% with five.⁴

These studies mostly measure what survives in text. This one grades actions from the system's own record, separates a handoff's format from its content, and traces failures to the hop where they entered.

Method

3.1 Environment

Everything is invented: the bank, the customers, the policy and the records. Agents ran in network-isolated Docker sandboxes through the Inspect framework, with a simulated command-line banking tool. The tool runs as root behind sudo, and agents run as an unprivileged user who can't read its data or source. It writes every action to its own event log, which agents can't touch, and every outcome is graded from that log. Cards are referred to only by type ("debit card"), never by number.

3.2 Design rules

1. Each agent's instructions are fixed. Its job, procedures and tools are the same for every case and never mention a case.
2. Agents pass work between them. Each agent gets only what the agent before it produced and can't go back to the call or to an earlier agent's work.
3. Procedures live with the agent that acts. The operations agent's own instructions say cards can go to a branch, and the resolution agent's say credits that need written confirmation must get it first.
4. Each agent can use only the commands its job needs.

The last three agents can't see the call, the logins or the transactions. If something is lost or wrong before it reaches them, they have no way to recover or check it. Many real systems let later agents re-read the case, so this setting is hard on lost context.

3.3 The case

A customer reports a $1,240.50 card charge she didn't make. Three invented customers share one template with different names and amounts, so the results cover a single scenario. Several details are found early and used late:

The simulated customer replies only to secure messages. If a secure message asks her something, she writes back that the charge wasn't hers; texts, calls and email get no reply.

A run counts as completed, as the grader codes it, when online banking is locked, the debit card is blocked, at least one replacement card is ordered and none is sent home, and the fraud and the fee are each credited in full. Harms are counted separately.

3.4 Arms

Every arm uses the same agents, procedures and tools and changes only how work is handed on.

An explained list gives each field a line of meaning. For example, phone_compromise_reported comes with "the customer reported their phone or number compromised."

Strict and free are the main comparison, and they differ in three ways. Strict agents get a field-by-field checklist that free agents lack, strict records are written as JSON, and strict records are validated (though the validator never rejected one in any run). The five other arms, run as Part E, pull those factors apart.

A strict record carries the real values: amounts, the branch name, the return date, each decision. The validator checks that each field is present and well-formed and that each ID exists in the case. It never sends an agent to look anything up. Part of a strict claim record:

{
  "disputed_txn_ids": ["T-88301"],
  "customer_own_txn_ids": ["T-88262"],
  "contact_channel": "secure_message",
  "blocked_channels": ["sms", "phone"],
  "phone_compromise_reported": true,
  "card_delivery": "branch",
  "branch": "Main Street",
  "customer_away_until": "2026-11-08"
}

3.5 Runs

Models were Claude Sonnet 5.5 and Claude Haiku 5.5, and every agent in a chain used the same model. Each arm ran 30 times per model (3 customers × 10).

3.6 Measures and analysis

Outcomes come from the tool's event log: whether the case was completed, and each harm.

In JSON chains, every field is checked at every hop. The correct value of each field is defined in code, so no model judge is needed. When a chain fails, the first wrong record decides the cause. Either the agent that set the value coded it wrong, the value changed as it was copied, or the record was correct and the acting agent still got it wrong.

Free-text chains were coded blind. All 43 incomplete free-text chains from Parts A and C, plus 43 completed ones drawn at random, were shuffled and stripped of outcome. Four separate model coders, working from one written codebook, labeled each handoff. For each matter already settled on the call, they marked whether the handoff carried it plainly, added a condition to it, or dropped it.

Rates are compared with two-sided Fisher exact tests. Each run's hypotheses were recorded in a lab notebook before the run, along with my own prediction that "the more structured format will perform better." Results below are labeled registered (tested as recorded, with Holm or Bonferroni correction within each run) or exploratory.

Results

4.1 One agent or five

In the natural flow, one agent beat every chain (Part A, registered; all four comparisons significant after Bonferroni correction). Most of the strict chain's shortfall was one failure. Thirteen of its 16 incomplete chains stalled on a code without its definition (section 4.7). Part C, registered after Part A, gave the customer a way to answer, and there the strict chain matched one agent, 59 vs 60 of 60 (exploratory).

In Part D the single Haiku agent completed 25 of 30 and the strict Haiku chain 28, too close to call (p = 0.42, exploratory). A free-text chain completed fewer cases than one agent in every run.

4.2 Which part of structure helps

In the natural flow, strict chains completed more cases than free-text ones (44 vs 33 of 60), but neither model's difference survived correction: Sonnet 23 vs 21 (p = 0.77), and Haiku 21 vs 12 (p = 0.037, Holm 0.074; registered). Strict chains harmed the customer less often, 3 times against 12 (p = 0.025, exploratory). Once the customer could answer, strict completed 59 cases to free text's 44 (p = 0.0001, registered), and in Part D the gap grew to 58 vs 17 (exploratory).

Part E tested which part of strict did the work. All six tests were registered and corrected together, in Part D's setting.

The field list did most of the work. It took free text from 17 to 49 completed cases, and it helped for each of the three customers (15 vs 6, 17 vs 4, and 17 vs 7 of 20).

For Sonnet, free text with the field list matched JSON exactly, at 30 of 30. Haiku did better with JSON, 28 vs 19 (p = 0.01, exploratory), but pooled across models the difference didn't survive correction.

Nothing else measurably changed the outcome. Records that were never checked did as well as checked ones, field names alone did as well as explained fields, and free text that wasn't addressed "for the next agent" did no better than free text that was. Free text failed for want of a list.

A field list works like a checklist. Each agent has to decide every item, and the next agent knows where to find each answer. The JSON format and the validator, as used here, added little on top of that.

4.3 A judgment the customer doesn't spell out

In Parts A–C the customer named the safe channel herself: "Please don't text or call me, use the secure message in the app." Getting the channel right meant carrying her words forward. In Part D she only described what happened:

Something else strange: my phone lost service last week and the carrier said my number was moved to another carrier. I didn't do that.

The bank's records still list her mobile for texts and calls and show nothing about the port. An agent has to work out that texts and calls now reach the attacker. A human fraud rep would very likely catch it, though no human was tested.

When the form asked about channels, the chain made the call. Strict chains never used a hijacked channel, while free text did in 10 of 60 runs (p = 0.0013, registered), and every arm with a field list did about as well as strict.

The phone field itself turned out to be unnecessary. Without it, intake still blocked texts and calls through the blocked_channels field, and only 1 of 60 chains used a hijacked channel (registered; no difference from strict). Later records often lost the block, though. In 20 of 30 Sonnet chains, the findings and every record after them stopped listing SMS or phone as blocked. Resolution still used secure messaging, so no harm followed; the call held where it was acted on, and it wasn't carried reliably along the way.

With nothing asking, the two models behaved differently. A single Sonnet agent never used a hijacked channel. A single Haiku agent phoned the hijacked number 12 times and texted it once, across 7 of 30 runs.

What this tests, then, is whether an agent matches what the customer said to a question on the form. A question on the form was enough for both models, and without one the weaker model missed it.

The judgments that came from data were right throughout. In every strict record in Parts A, C and D, the account takeover and the attacker's email (both from the login history), the fee's link to the fraud, and the confirmation rule (from the policy) were correct. I had registered the hypothesis that most record errors would be judgments coded wrong, and that values changing in copying would be rarer. It was falsified: there were no coding errors at all. With only four judgments from data, each with a clear signal, that is a narrow result.

4.4 The telephone game

In strict records, values mostly held across four agents. Sonnet changed no value in 360 records across Parts A, C and D. Haiku's slips were all in copying values it didn't need. It blanked the customer's return date in 10 to 18 records per run, which did no harm, and in 4 chains it changed "branch" to "address on file," which sent the card to the empty house. In 2 more chains, operations sent the card home from a correct record.

Failed free-text chains dropped details and added conditions. From the blind coding (exploratory):

A dropped detail never appeared in a completed chain, which makes it the cleanest marker of failure here.

Added conditions went with failure, but more than half the completed chains had one too. A typical case: on the call, the customer asks for the card at the branch and the analyst agrees. Investigation writes "confirm with the customer through the app before it is ordered," and decision repeats it. Operations can't message customers, so it holds the order, and nobody ever places it.

Every free-text chain, completed or failed, told a later agent what to do ("Operations should verify…"). That is how these agents wrote free text, and it didn't separate success from failure.

In Part D the conditions reached their end point. The customer hadn't named a safe channel, and free-text agents ruled out texts, calls and email; some also ruled out the secure channel, since online banking had been locked. In 16 of 60 chains, resolution stopped without contacting her, saying there was no safe channel or nothing it could verify.

4.5 The weaker model needs the list more

Haiku's gap between strict and free text was 27 points larger than Sonnet's (95% bootstrap interval 10 to 43, resampling runs within each model, arm and customer; exploratory). Part D on its own was inconclusive (17 points, interval −7 to 40), and Sonnet's strict arm sits at the ceiling, which limits what the comparison can show. Haiku also made every strict-record error in Parts A, C and D, and in Part E it was the model that gained from JSON over a text field list.

That fits the relay study in section 2, where a strong relay kept nearly all facts and a weak one lost many more.³

4.6 What later agents can see

Later agents could catch only what they could see. Part B replaced one agent's work with a scripted record, either correct or with one mistake. Strict chains received it as JSON, and free-text chains received the same record written out as prose that said no more than the codes. Every agent after it ran live (Sonnet 5.5, 15 runs per cell, registered).

Each cell is out of 15. With the correct record, strict chains completed 45 of 45 and free-text chains 42 of 45.

Much of this follows from the design. Under least privilege, no later agent can check a takeover finding or a fee decision, and the test confirms that agents found no way around that. The one checkable mistake was caught every time, because investigation looked up the charge itself. At 15 runs per cell, near floor and ceiling, no difference between formats showed up.

In free text, acting agents sometimes refused to act on what they couldn't verify. In Parts A and C, several free-text resolution agents had what they needed and stopped anyway; one wrote, "I can't read the customer, login, flag, policy, or transaction data from this stage." Given a validated record, every strict chain in Parts A, C and D contacted the customer.

4.7 A code without its definition

In Part A, 13 of 60 strict chains never paid the customer, and all 13 failed the same way. The resolution agent received a correct record: credits approved, credit_condition: written_confirmation. It messaged the customer, and she replied in writing that the charge wasn't hers. Under the bank's policy, that reply is the written confirmation. Only the decision agent had read the policy, though. The resolution agent saw a code, decided a denial wasn't enough, and asked for a set phrase:

I issued no credits, because the customer never gave the written confirmation the case requires.

The single agent, which read the policy itself, never did this in 60 runs, and no free-text chain did either.

The code arrived word for word and its meaning was lost. In Part C the customer answered "Yes, I confirm" when asked, and the stall disappeared, from 13 of 60 chains to none (p = 0.0001, registered). A real customer would usually comply too, at the cost of an extra round trip. Another system, or a counterpart that can't reply in the expected form, may not.

Discussion

5.1 A shared record, or passing work along?

In this study, work passed from agent to agent. The alternative is one shared case record that every agent reads and writes, which is how a case management system already works. The results suggest which failures depend on that choice.

A shared record in which each field has one owner, and other agents can read the field but not change it, would have prevented Haiku's four "branch" slips and the lost channel blocks in Part E. It would leave the other failures in place. It also tempts builders to let every agent read everything, which undoes least privilege. All of this is a prediction from the data; a shared record wasn't tested.

5.2 How the test decides what you find

Three choices changed the conclusions. Each was caught by reading transcripts or by an outside reviewer; the scores alone didn't show them.

A customer who couldn't comply with "reply I CONFIRM" made one agent look much better than a chain (Part A). When she could answer, the gap closed (Part C).

A customer who named the safe channel herself made it look as if the agents could judge the risk. When she only described what happened, the single Haiku agent and the free-text chains missed it (Part D).

Comparing JSON that came with a checklist to free text that came without one made the format look responsible for the difference. Separating the two showed the checklist did most of it (Part E).

Measurement validity

These problems turned up in the harness, grader or analysis. Each would have led to a wrong conclusion if left in place.

1. The simulated customer couldn't comply with a request for a set phrase. Reading every failed strict chain in Part A showed this. Part A is reported as measured, and Part C, registered separately, used a customer who answers "Yes, I confirm."
2. Strict and free differed in more than format. Strict agents got an explained field list and free agents got none, and the validator never rejected a record. An independent review of the arms' instructions caught this, and Part E separated the factors (section 4.2).
3. In Parts A–C the customer named the safe channel herself, so getting it right tested whether a detail was carried forward and said nothing about judgment. A reader's question about how the agents caught the hijacked phone exposed this. Section 3.3 now marks how each detail is known, and in Part D she only describes what happened.
4. The main hypothesis about structured records, which predicted judgment errors, was wrong; there were none. It is reported as falsified, with its narrowness noted (section 4.3).
5. Two smaller problems were fixed before any affected result was read: an ambiguous denominator in one hypothesis, and a free-text rendering of a planted record that said more than the code did.

Limitations

• This is one scenario. The three customers share a template, so the runs aren't independent scenarios. Results held for each customer separately, and generalizing beyond this case needs different cases.
• Both models come from one vendor, Anthropic, and the tooling was built with one of them. Chains mixing models weren't tested.
• The simulated customer is simple. She replies to any secure message that asks her something.
• Later agents could look nothing up. Systems that let them re-read the case would lose less context and could check more.
• There was one set of schemas, and every detail had a field by design. A field list that missed a detail would likely fail on it, which wasn't tested.
• Each arm ran 30 times (15 per cell for planted mistakes), so differences under about 20 points are hard to detect, and several arms sit at or near the ceiling.
• The blind coders were four model agents working from one codebook; no human coded the chains.
• No human was tested. That a human rep would catch the hijacked phone is an expectation.

Implications for practitioners

1. Give each agent a list of fields to fill in. Here the list did more than the format: free text with it went from 17 to 49 of 60 and matched JSON for the stronger model. Make the list cover every decision the next agent needs.
2. With a weaker model, use a typed format as well. JSON added more for Haiku than for Sonnet, though not enough to confirm here.
3. Put the question on the form. A field for blocked channels got every chain through the hijacked phone, and without one a single weaker agent missed it 7 times in 30.
4. Ship each code's definition with it, or give the definition to the agent that acts. written_confirmation meant one thing to the agent that set it and another to the agent that used it.
5. If values and judgments must survive many hops, keep them in a shared record where each field has one owner. Copying was where the JSON records went wrong.
6. Put a check wherever a judgment is made that no later agent can verify. Nobody downstream caught a wrong takeover finding or fee decision, so a second reviewer, a rule in code, or a person has to sit at that stage.
7. Build the test to ask for the judgment you care about, and change one thing at a time. A customer who couldn't comply, a customer who named the safe channel herself, and a comparison that bundled format with a checklist each changed what this study found.

Disclosure

The experiments, code and analysis were developed with assistance from Claude (Anthropic). Both models under test are Anthropic models. The blind coders and the independent reviewer were also Claude agents, briefed separately from the analysis.

Data and code

The code, scenarios, lab notebook (experiments 016–023, with hypotheses and calibration rules recorded before each run), blind coding labels and analysis scripts will be linked here when the public repository is released.