Prototype, Pilot, or Production

In plain words

Before a food stall runs at the college fest, four friends taste the dish in the hostel, then a small counter serves fifty people while a senior watches, and only then does the college allow a full three-day stall. Agent builds work the same way. A prototype asks whether it works at all, a pilot asks whether it is safe with a small real audience, and production means your team owns it every day. Pick the stage from the risk, not from how neat the code looks.

How it flows

A tiny example

stage = choose_stage(needs_real_users, risk_small, team_ready)
missing = required_controls(stage) - controls_you_have()
if missing:
    print("not ready for", stage, missing)
else:
    run(stage)

Notice the decision uses only exposure, risk and readiness, and then checks that the matching controls actually exist.

What you will learn

• The three stages of any agent build: prototype, pilot, production.
• How to pick the stage from the question you are trying to answer, not from how finished the code looks.
• What controls each stage needs before you let real people near it.
• How to stop a prototype from quietly turning into a production system.

The problem, simply

Think about your college fest. Somebody has an idea for a new food stall. First, four friends cook one dish in the hostel kitchen and taste it themselves. If it is terrible, nobody is harmed. That is a test.

Next, they put up a small counter on day one of the fest, serve about fifty people, keep one senior standing there watching, and stop selling the moment something goes wrong. Real customers, real money, but small and watched.

Only after that does the college let them run a proper stall for three days with a bank account, a licence and someone responsible if a student falls sick. Same food, completely different level of responsibility.

Agents are exactly the same. See, the problem is that in software all three stages look identical from outside. It is the same Python file, the same screen, the same "it works". So a weekend hack quietly starts handling real customer refunds, and nobody notices until it refunds ₹4 lakh by mistake.

The idea

There are three stages, and each one exists to answer a different question.

Three stages, three questions

• Prototype asks: can this thing work at all?
• Pilot asks: does it work safely with a small group of real users, on real data?
• Production asks: can we own this every single day, at the reliability we promised?

Notice that none of these questions is about polish. A prototype can be beautifully written and still be throwaway. A pilot can touch real production data and still be a pilot, because the audience and the power it has are limited. Production starts the day your team accepts that if it breaks at 2 a.m., somebody wakes up.

What each stage needs

A prototype should be throwaway, kept away from real systems, narrow in what it does, and honest about the one question it is answering. Do not spend a week on clean architecture before you know the idea works.

A pilot needs six things: a named group of users, one human owner, a fixed time limit and limited authority, an audit trail plus a rollback button, thresholds for success and for danger, and clear exit rules for expand, fix, or stop. If you cannot write the stop rule, you are not ready to run a pilot.

Production needs a promise about uptime, someone on call, a security and privacy review, cost and capacity limits, rollback and recovery, continuous monitoring, and even a plan for switching it off one day.

IMP

Note: A pilot without a stop rule is not a pilot. It is a slow, unannounced launch.

A worked example

Suppose Priya builds an agent that answers student queries about hostel fee refunds.

Week one, she runs it on ten fake queries she typed herself. No real student sees it. That is a prototype. The only question is whether the agent reads the fee rules correctly.

Week three, it works. Now she gives it to thirty final-year students of one hostel block, for two weeks. The agent can only draft a refund reply; the warden approves before sending. Every draft is logged. If more than three drafts are wrong, they stop. That is a pilot.

Month three, the college wants it for all 4,000 students, sending replies on its own. Now someone must be responsible when it fails during fee week. That is production, and it starts the moment authority becomes automatic, not the moment the code is deployed.

Stage drift

Here is the trick most teams miss. Nobody ever decides "let us promote this prototype". It just drifts. Somebody shares the link. Somebody connects it to the real database. Somebody gives it write access. Users, data, and power arrive, but ownership and controls never do.

IMPRemember: a prototype becomes dangerous the moment it gains users, data, or authority without gaining an owner.

The fix is to make the stage a real thing in the system, not a line in a document. A yellow banner saying "DEMO ONLY" stops nobody. Config flags, access control, and logs do.

Build it

This little program takes a few build situations, decides the stage, and prints the controls each stage needs.

"""Decide the stage for a build, and list the controls it needs."""

# Controls that must exist before you may sit in each stage.
CONTROLS = {
    "prototype": ["throwaway code", "fake or sample data", "no real users"],
    "pilot": ["named audience", "human owner", "time limit",
              "audit log", "rollback", "stop rule"],
    "production": ["uptime promise", "on-call owner", "security review",
                   "cost limits", "monitoring", "retirement plan"],
}

def choose_stage(needs_real_people, risk_is_small, team_is_ready):
    """Pick the least risky stage that can still answer the question."""
    if not needs_real_people:
        return "prototype"
    if risk_is_small and team_is_ready:
        return "production"
    return "pilot"

def check(name, claimed_stage, have):
    """Compare the controls you actually have against the ones required."""
    missing = [c for c in CONTROLS[claimed_stage] if c not in have]
    status = "OK" if not missing else "NOT READY"
    print(f"{name}: claims {claimed_stage} -> {status}")
    for item in missing:
        print(f"    missing: {item}")

builds = [
    # name, needs real people, risk is small, team is ready, controls in place
    ("Priya fee-refund draft bot", False, False, False, ["throwaway code",
        "fake or sample data", "no real users"]),
    ("Hostel block trial", True, False, False, ["named audience",
        "human owner", "time limit", "audit log"]),
    ("Campus-wide auto refunds", True, True, True, ["monitoring",
        "cost limits"]),
]

for name, real, small, ready, have in builds:
    stage = choose_stage(real, small, ready)
    print(f"\n--- {name} ---")
    print(f"suggested stage: {stage}")
    check(name, stage, have)

Look at the third build. It claims production, but it is missing an on-call owner, a security review, an uptime promise and a retirement plan. That is exactly the gap that makes a launch scary. Notice also that the decision uses only three facts about the situation, never a word about how good the code looks.

Where you will see this

• Coding agents like Claude Code and Cursor start read-only or with a diff you approve, before they are allowed to edit and commit on their own.
• Customer-support bots at Swiggy or Flipkart first suggest replies to a human agent, and only later reply directly to customers.
• Payment and refund flows almost always go through a small pilot with one city or one merchant group before a national rollout.
• Company internal tools: an HR chatbot is tested with one team of twenty people before all offices get it.
• Hackathon and college projects: the demo works on stage, then someone connects it to the real college database and it becomes a live service overnight.

Common mistakes

• Choosing the stage by how polished the code looks. Polish is not evidence. A clean prototype that nobody real has used still proves nothing about safety.
• Running a pilot with no stop rule. If there is no threshold that makes you switch it off, you will keep it running through every problem and call it "learning".
• Depending on a warning banner. Users ignore banners. Only config, permissions and logs actually limit what a prototype can touch.
• Over-engineering the prototype. Spending two weeks on retries, caching and clean layers before you know the idea works is wasted effort.
• Treating deployment as production. Having a URL is not production. Production starts when a human accepts responsibility for it every day.

If they ask in an interview

Q: What is the difference between a prototype, a pilot and a production system?

A: They answer three different questions: can it work, does it work safely with a small real audience, and can we run it reliably forever. The difference is in exposure, consequence and ownership, not in code quality or looks.

Q: How would you decide if an agent is ready for production?

A: I would check evidence from a bounded pilot first: real users, measured accuracy, a full audit trail, and no unresolved safety issues. Then check operational readiness, meaning an owner, monitoring, rollback, cost limits and a security review. If either half is missing, it stays a pilot.

Q: What is stage drift and how do you prevent it?

A: Stage drift is when a prototype slowly gains real users, real data or real authority without gaining controls or an owner. I prevent it by enforcing the stage technically, with separate credentials, restricted permissions and telemetry, so the stage is visible from the system itself.

Try these

1. Take three projects you have built, and label each one prototype, pilot or production based on who uses it and who is responsible, not on whether it is deployed.
2. Write full pilot exit criteria for one of them: audience, owner, duration, success threshold, danger threshold, and the exact rule that stops it.
3. Extend the program above so a build that claims production but is missing more than two controls gets pushed back to pilot automatically.
4. Add one technical control to any project of yours that makes it impossible for the prototype version to reach real data, such as a separate read-only key.

Words, simply

Quick recap

• The stage is decided by the question you are answering and the damage a mistake can do, never by how finished the code looks.
• A pilot means real users inside hard limits: named audience, owner, time limit, logs, rollback and a stop rule.
• Enforce stages in config, permissions and logs, because a warning banner cannot stop stage drift.