All engineering notesENGINEERING NOTES

Fork the execution, not the trace

A failed agent run shouldn't be a terminal artifact. Fork any execution from an exact boundary, change one decision, and continue with the real inherited state — without replaying the expensive prefix or re-firing production side effects.

Actae Engineering12 min read
forkingcounterfactualdurable executionexperimentsagentsside effects

Here is the uncomfortable shape of most agent development today. An agent runs for eleven minutes, spends real money, makes four tool calls, and then makes one bad decision at step 7. You now have exactly two options: read the trace to figure out what happened, or throw the whole run away and start over from step 1 with a different prompt.

That second option is the expensive one, and it is the one people actually need. The run was great until step 7. You want to keep steps 1–6 and change what happens at 7. There is a word for that in version control — a fork — and it is the thing missing from execution infrastructure.

This post is about treating an execution as a forkable object: not a trace, not a log, not a workflow definition, but a run with identity, state, history, side effects, and a boundary you can fork from.

What a fork is

A fork creates a new execution whose history and state begin at a boundary you choose in an existing run. The parent is never modified. The child inherits the durable state that existed at that boundary — the real accumulated context, not a summary of it — plus the lineage that says exactly where it came from.

The important property is exactness. You ask for a boundary; the runtime resolves it and hands back an immutable receipt that records both what you requested and what was actually inherited: the resolved cursor, the source state version, a fingerprint of the state, and whether the boundary is restorable. If the exact checkpoint you asked for doesn’t exist, you get told, not quietly given something later. You should never have to guess whether a fork really started where you think it did.

A fork is not:

  • a copy of trace rows — it is a live execution with a real, independent future;
  • a process clone — the application still owns the loop and decides what runs next;
  • a merge tool — forks stay independent and the parent is untouched.

The opportunity: change the decision, not the prefix

Once a run is forkable, the unit of experimentation changes. It is no longer “run the whole pipeline again with a different prompt.” It is “fork at the decision and change only the decision.”

Concretely, with the Python SDK: a run reaches step 7, and you want to try a different model from that exact point.

# The run already got through steps 1..7. Fork their real state.
fork = await parent.fork(
    at_step=7,
    name="run-42/model-b",
    intervention={"model": "candidate"},     # opaque; your app decides what it means
    tool_policies={"email.send": "block", "*": "auto"},
)

async with fork:                            # continues at step 8
    ...                                       # steps 1..7 are NOT re-run

Two things are worth calling out. First, intervention is an opaque descriptor: the runtime stores it and exposes it on the fork, but it is your application that reads it and changes behavior. The substrate does not pretend to interpret your prompt or pick your model. Second, the fork is a real child execution with its own event stream — you can step it, crash it, resume it, or fork it again.

The same call works from an existing run without holding a live session object, which is how you fork a run that already finished or crashed:

session = await AgentSession.resume(
    actae, "run-42",
    fork_at_step=7, name="run-42/fallback-tool",
)

The opportunity: fork without re-firing production side effects

This is the part that makes forking usable on real workloads rather than only on toy pipelines. Suppose step 5 sent an email, and step 11 sends another. If you fork at step 7 and continue, the fork must not re-send the step-5 email just because that effect exists in the inherited history.

A fork can declare, per tool, how inherited work is treated: replay a previously recorded result instead of performing it again, refuse and surface a blocked boundary, or run live. That is the tool_policies map above. A fork with email.send set to block physically cannot re-send an email; an exact inherited call under auto returns the recorded result instead of re-executing.

The honest boundary

The runtime coordinates retries, fences stale workers, and replays recorded results. It cannot make an external side effect exactly-once by itself: that still requires the downstream to be idempotent or queryable. What changes is that the window collapses from “any duplicated step” to “the one external dependency that ignored your key.” That is a far smaller and far more testable claim.

The opportunity: compare realities, not text

Two forks from the same boundary is where the interesting work starts. You fork at step 7 with two different interventions, let both continue, and then compare them structurally — not a fuzzy text diff of the final answer, but the actual state each fork reached, which tool calls each made, and where they diverged.

diff  = await actae.diff_states("run-42", "run-42/model-b")
trail = await actae.decision_trail("run-42/model-b")
# diff:   added / removed / changed state paths, plus per-side divergence
# trail:  lineage chain + the immutable inherited boundary + the tool ledger

This answers a question a trace fundamentally cannot: what did the winning fork know that the losing one did not? And because both forks share a recorded boundary, the comparison is fair — you are comparing futures from the same past, not two unrelated runs.

The opportunity: close the loop with experiments

A fork is not useful if the result evaporates. Forks are first-class experiment members: record an outcome and a score on each, rank them, and promote the winner. Promotion is an append-only decision — it records that a fork won and closes the losing path, rather than silently overwriting history.

await actae.set_outcome("run-42/model-b", "promoted", score=0.92)
await fork.promote()

The full loop is therefore: build, run, fail, inspect, fork, experiment, compare, promote. Every step is durable, and the boundary that each fork inherited is provable after the fact.

The opportunity: fork an entire agent team

Multi-agent systems are where this gets genuinely hard with ordinary tooling. If a researcher, a writer, and a tool-runner exchange durable messages, “what if the researcher had concluded something else?” is not a question you can answer by re-running one agent. You have to fork the whole team from a consistent point.

Execution groups make every member a durable identity bound to a channel, with leased ownership, causal message edges, and replay-to-live delivery. A group fork freezes or reactivates members and applies per-tool policy, so you can create a counterfactual of the team — same history up to the boundary, different continuation, with a receipt that records how the divergence propagated.

The opportunity: fork your framework’s checkpoint, not your code

None of this requires adopting a new agent runtime. The adapters meet frameworks where they already persist state. A LangGraph thread can be forked at an intermediate checkpoint into a new thread that resumes from that point. A Claude Agent SDK session can fork its persisted transcript and continue with the inherited conversation. LangChain, OpenAI Agents, CrewAI, and GitHub Copilot have equivalent fork paths.

You keep the framework and the loop; the fork becomes the checkpoint boundary, and you get lineage, comparison, and an audit receipt for free.

Why this is not “time travel debugging”

Debugging tools let you inspect and, in some cases, replay a run. A fork lets you continue it. The difference is the difference between reading a git log and checking out a branch: one is observation, the other is a live state from which new work happens, with its own durable future.

It is also not a workflow engine. A workflow engine takes over your control flow and runs it through its own model. A forkable execution leaves prompts, models, tools, and planning exactly where they are — in your code and your framework — and makes the boundaries underneath them durable. That is precisely why counterfactual forking is possible at all: the application owns the “what next,” so you can change it without rewriting the system.

Patterns worth stealing

  • Refine the last step, not the whole run. Fork at step N, change step N+1, compare. The expensive prefix is paid once.
  • Counterfactual incident review. A production run failed. Fork at the decision that caused it, try the fix, and keep the evidence of what changed.
  • Fork an expensive production run into an experiment. With side-effect policy set to refuse irreversible calls, a fork can safely explore an alternative on top of real state.
  • Regression-test against real state. Fork a known production boundary and assert the new logic produces the expected state — a test that starts from reality, not a fixture.
  • Human-in-the-loop forks. Fork a path that pauses for a durable approval and resumes days later without keeping a process alive.

What it looks like end to end

The flagship workflow, in one code block: run once, fork the failure, change the decision, compare, promote.

# 1. A baseline run that ends in a bad outcome.
async with AgentSession(actae, "research-7f2a", state_fn=state) as s:
    ...                                       # steps 1..11

# 2. Fork at the decision, not at the start.
fork = await s.fork(
    at_step=7,
    name="research-7f2a/model-b",
    intervention={"model": "candidate"},
    tool_policies={"email.send": "block", "*": "auto"},
)

# 3. Continue from step 8 with the inherited state.
async with fork:
    ...                                       # steps 8..11, prefix not re-run

# 4. Compare realities, then promote the winner.
diff = await actae.diff_states("research-7f2a", "research-7f2a/model-b")
await actae.set_outcome("research-7f2a/model-b", "promoted", score=0.92)
await fork.promote()

Try it on a run you already have

Forking is available in the Python, Go, and TypeScript SDKs, over the same REST and WebSocket API, on the Cloud runtime or self-hosted. The boundary modes, the immutable receipt, the side-effect policies, and the experiment loop are all documented.

If you want the mechanics rather than the opportunities, start with forking & recovery, coordination for the tool-execution ledger, and the canonical API. To see it move first, run the 75-second walkthrough — crash, resume, fork, compare — or start an assisted evaluation with one real workflow of your own.

Disclosure, as always: I work on Actae, which is the runtime implementing everything described above. We are honest about the limits — fork continuation is application-owned, and exactly-once external effects still require a cooperative downstream — because the value of the primitive depends entirely on your ability to trust the boundary it gives you.