Multi-Agent Debate

In plain words

When you proofread your own report, you miss your own mistakes. When three friends read it, each one catches something different. Multi-agent debate does the same with AI: several copies of the model answer the same question separately, then read and criticise each other for a few rounds, and settle on one answer. You get fewer wrong answers, but you pay more tokens and wait longer.

How it flows

A tiny example

answers = [think(question) for _ in range(3)]
for _ in range(2):
    new = []
    for i, a in enumerate(answers):
        peers = answers[:i] + answers[i+1:]
        new.append(critique_and_update(a, peers))
    answers = new
print(most_common(answers))

Notice that round 1 is answered alone, and only after that does each agent see the peers, so the first opinions stay independent.

What you will learn

• What multi-agent debate is: many copies of a model answer, then argue, then agree.
• Why arguing makes answers more correct, and when it just wastes money.
• Full mesh vs sparse topology, and why sparse is usually enough.
• The three ways debate quietly fails in production.

The problem, simply

Think about your final-year project. You wrote the whole report alone. You proofread it yourself three times. And still, in the review, the professor found a silly mistake on page 2 that you had read six times.

That is the thing about checking your own work. Your eyes skip over your own errors. You made the mistake because you believed something wrong, and you keep believing the same wrong thing while proofreading.

Now suppose instead you gave the report to Priya, Rahul and Sneha in your group. Each one reads it separately. Priya catches the wrong formula. Rahul catches the missing citation. Sneha says the conclusion does not match the data. Three people found three different mistakes, because they have different blind spots.

Same story with AI agents. One model checking its own answer is you proofreading your own report. Multi-agent debate is handing it to your group.

The idea

The debate protocol

The basic recipe has a name from a 2023 research paper: Society of Minds. Do not worry about the name, the recipe is simple.

You take N copies of the model. Each one answers the same question on its own, without seeing the others. Then over R rounds, each one reads what the others said, criticises it, and updates its own answer. After R rounds you take the answer they agree on.

The original experiments used N=3 agents and R=2 rounds, mostly because more copies cost more money. On harder problems, more agents and more rounds gave better answers.

One nice finding: mixing different models beats using three copies of the same model. Two different models together did better than either one debating itself. Makes sense, no? Three students from the same coaching class make the same mistakes. Three students from different backgrounds do not.

A worked example

Suppose Karthik asks the agent: "Is Rc4 a legal move in this chess position?"

Agent A says yes. Agent B says yes. Agent C says no, because the rook is pinned to the king.

Round 1 ends. Now A and B read C's reason. A rechecks, sees the pin, changes to "no". B also changes. Round 2, all three say no. Converged, and correct.

One model alone would have said yes and moved on happily.

IMP

Note: The magic is not that three agents are smarter. It is that a wrong answer usually cannot survive one good counter-argument.

Full mesh vs sparse topology

Topology just means who reads whom.

In full mesh, everyone reads everyone. With N=5 agents and R=3 rounds, that is 15 proposals, and each agent reads 4 peers each round. That is 60 read-and-critique operations. Every one of those costs tokens, which costs rupees.

In a star (also called hub-and-spoke), one agent is the hub and the other four only read the hub. Same 15 proposals, but only 12 critique operations instead of 60.

A 2024 study found that on many tasks, sparse topologies (star, ring) matched full-mesh accuracy at a fraction of the token cost. So do not reach for full mesh by default.

IMPRemember: More debate is not free. Every extra agent and every extra round multiplies your token bill and your response time.

When debate helps and when it hurts

It helps for factual answers (independent proposals cross-check each other, so fewer hallucinations), for rule-following (one agent forgets a rule, another remembers), and for open-ended reasoning (different framings narrow down to the right one).

It hurts when the user is waiting on screen, because N times R rounds happen one after another. It hurts at scale, because you pay N times R times the tokens. And it is silly for a simple lookup — checking today's IRCTC fare once is cheaper than five agents debating it.

Build it

"""Toy multi-agent debate. No API calls, no libraries. Just python3 file.py."""
from collections import Counter

TRUTH = "7"

# Each "agent" is a fake model: its first guess, and how many peers must
# argue for a different answer before it will change its mind.
# In real life these would be model calls with different prompts or vendors.
AGENTS = {
    "A": {"guess": "42", "needs": 1},
    "B": {"guess": "42", "needs": 1},
    "C": {"guess": "42", "needs": 1},
    "D": {"guess": "42", "needs": 2},   # the stubborn one
    "E": {"guess": "7",  "needs": 9},   # the one who is actually right
}

def propose(name, peers):
    """Give this agent's answer after reading its peers' answers."""
    me = AGENTS[name]
    if not peers:                       # round 1: answer alone, no peeking
        return me["guess"]
    votes = Counter(p for p in peers if p != me["guess"])
    if votes:
        other, count = votes.most_common(1)[0]
        if count >= me["needs"]:        # enough peers disagree, so switch
            return other
    return me["guess"]

def debate(topology, rounds=3):
    """topology: agent -> list of agents it is allowed to read."""
    answers = {n: propose(n, []) for n in AGENTS}
    critiques = 0
    for r in range(1, rounds + 1):
        fresh = {}
        for name, can_read in topology.items():
            peers = [answers[p] for p in can_read]
            critiques += len(peers)     # every peer read costs tokens
            fresh[name] = propose(name, peers)
        answers = fresh
        if len(set(answers.values())) == 1:      # all agree, stop early
            return answers[name], r, critiques
    winner = Counter(answers.values()).most_common(1)[0][0]
    return winner, rounds, critiques

names = list(AGENTS)
full_mesh = {n: [p for p in names if p != n] for n in names}   # everyone reads everyone
star = {n: ([] if n == "E" else ["E"]) for n in names}         # E is the hub

for label, topo in [("full mesh", full_mesh), ("star", star)]:
    answer, used, ops = debate(topo)
    tag = "correct" if answer == TRUTH else "wrong"
    print(f"{label:10s} answer={answer} ({tag})  rounds={used}  critique_ops={ops}")

Look at two numbers in the output. Both wirings end up on the same, correct answer, but critique_ops is many times larger for full mesh. That gap is your token bill. Also notice the star never fully converges, because stubborn agent D only ever hears one voice arguing against it and needs two.

Where you will see this

• Coding assistants like Claude Code and Cursor that generate a patch, then run a second pass to review it before showing you.
• Orchestrator-and-workers setups, where one main agent farms out sub-tasks and then synthesises the replies. That synthesis step is debate wearing a different shirt.
• Support bots for banks and telecoms, where one agent drafts the reply and another checks it against policy before it is sent.
• Automatic grading and evaluation systems, where several model judges score an answer and the majority wins.
• Medical and legal document review tools, where the cost of one confident wrong answer is very high.

Common mistakes

• Giving every agent the same prompt. Identical prompts produce nearly identical answers, so there is nothing to critique. You pay three times for one opinion.
• Letting them agree too early. If all agents latch onto the first wrong answer in round 1, the debate just confirms it. Force distinct proposals in round 1.
• Trusting a single hub. In a star topology, if the hub is confidently wrong it poisons everyone. Rotate the hub, or use two hubs.
• Using debate for everything. A price lookup or a date calculation does not need five agents. Use debate only where a wrong answer actually costs something.
• Forgetting the clock. Rounds run one after another. Five agents times three rounds can turn a 2-second reply into a 30-second wait.

If they ask in an interview

Q: What is multi-agent debate, and why does it beat a single model checking itself?

A: You run N copies of the model, each proposes an answer independently, then over R rounds they read and criticise each other and update. A single model self-checking has the same blind spots as when it answered, so it tends to confirm itself. Independent agents have different blind spots, so a wrong answer usually gets challenged.

Q: How would you reduce the cost of a debate system?

A: Change the topology. Full mesh means every agent reads every peer each round, which grows fast. A star or ring means each agent reads only one or two peers, so critique operations drop a lot while accuracy on many tasks stays about the same. You can also stop early the moment everyone agrees.

Q: When would you not use debate?

A: When latency matters, since rounds are serial, or at high volume, since you pay N times R tokens per question. And for simple factual lookups, where one call or one database query is both cheaper and more reliable than five agents arguing.

Try these

1. Add a rule that in round 1 every agent must give a different answer. Does the debate reach the correct answer more often, or just take longer?
2. Make each agent return an answer plus a confidence from 0 to 1, and pick the winner by total confidence instead of by count. See if it helps or if one loud wrong agent takes over.
3. Change the star topology so the wrong agent is the hub. Watch how fast the whole group goes wrong, then add a second hub and try again.
4. Grow to five agents and three rounds, and print critique operations for full mesh, star and ring side by side. Write down the cost difference.

Words, simply

Quick recap

• Multi-agent debate = N agents propose independently, criticise each other for R rounds, then converge; it fixes the blind spots a single self-checking model cannot see.
• Sparse wiring like a star gives most of the accuracy of full mesh at a small fraction of the token cost.
• Use it where a wrong answer is expensive, not for cheap lookups, and always guard against everyone agreeing on the first wrong answer.