Skip to content
Road to Intelligence

Concept · Chapter 13: Agents

Multi-Agent Systems

Should knowUnderstand7 minDifficulty

Multi-agent systems coordinate separate model contexts or roles to accomplish a shared task.

The problem

A single context can become overloaded when subtasks are separable.

The solution

Delegate bounded work and integrate evidence through explicit coordination.

The consequence

Parallel work adds communication, integration cost and correlated failure risks.

You should understand first

  1. Text as Data
  2. Probability and Distributions
  3. Conditional Probability and Bayes' Theorem
  4. Probability of Sequences
  5. Language Modeling
  6. Vectors
  7. Dot Product
  8. Embeddings
  9. Attention
  10. Softmax
  11. Self-Attention
  12. Causal Masking
  13. Entropy
  14. Loss Functions
  15. Cross-Entropy Loss
  16. Autoregressive Next-Token Prediction
  17. Pretraining at Scale
  18. GPT-1 → GPT-2 → GPT-3
  19. In-Context Learning
  20. The Turing Test
  21. Symbolic AI
  22. Logic and Rules
  23. Expert Systems
  24. Knowledge Representation
  25. The Knowledge-Acquisition Bottleneck
  26. From Rules to Learning
  27. Supervised, Unsupervised and Self-Supervised Learning
  28. Expected Value and Variance
  29. Reinforcement Learning
  30. MDPs, Policies and Value
  31. Decoding: Greedy, Temperature, Top-k, Top-p
  32. One-Hot Encoding
  33. Tokenization
  34. Chat Templates
  35. System Prompts and Instructions
  36. Structured Outputs and Constrained Decoding
  37. Tool Calling
  38. LLM Agents
  39. Agent Runtime
  40. Search
  41. Planning
  42. Agent Planning and ReAct
  43. Multi-Agent Systems

Separate contexts, shared goal

A worker can inspect callers while another proposes regression cases. They may run the same model with different contexts and tools. AutoGen provides an early framework for programming conversations of this kind.

The integration is real work

Define ownership before letting workers edit. Give each a clear input, output and permission scope. A coordinator must reconcile conflicting conclusions and run checks on the combined change.

Two agents that share a mistaken premise are not two independent witnesses. More messages can spread the error. Delegation is useful when separation reduces context overload or enables parallel progress; a three-line bug may take longer to explain to workers than to fix.

Fair comparison: keep the total compute and tool budget comparable when measuring one agent against several. Otherwise an apparent architectural advantage may come from simply trying more candidates.

What to remember

  • Multi-agent systems coordinate separate model contexts or roles to accomplish a shared task.
  • Delegate bounded work and integrate evidence through explicit coordination.
  • Parallel work adds communication, integration cost and correlated failure risks.

Key papers