2 entries
#evaluation
Entries
What Breaks When You Split a Job Across AI Agents
Five AI agents worked a bank fraud case in sequence, each passing its work to the next. Giving each agent a list of fields to fill in mattered most: it raised completed cases from 17 to 49 of 60. With JSON records, a chain came close to one agent doing the whole job; with plain text, it fell well short. Chains lost details, added conditions nobody could meet, and passed along wrong conclusions no later agent could check.
Experiment, 27 minAccidental Harm From an LLM Agent: Acting Before Reading Its Tools
LLM agents patching servers caused damage not by breaking policy but by acting before they understood their tools. A small model rebooted servers by accident in nearly half of runs; both models restarted nodes still serving traffic in 88% of rolling-cluster runs. Simple interface changes stopped the reboots.
Experiment, 32 min