A flat log can tell you that five things happened. It often cannot tell you which operation caused the next one, which failure triggered a fallback, or whether three tool calls were children of one planning step or unrelated work.

That distinction matters for AI agents because the path is part of the behavior.

I maintain AgentInspect, an open-source TypeScript toolkit for inspecting agent executions locally. This article explains why I chose execution trees as the primary debugging model, using synthetic fixtures verified against agent-inspect@6.17.4.

The problem with reading an agent run as a timeline

Consider a support agent that performs these operations:

09:00:00.000 plan started
09:00:00.020 inventory request started
09:00:00.060 inventory request failed: 503
09:00:00.061 inventory request started
09:00:00.120 inventory request succeeded
09:00:00.150 answer completed

This is enough to reconstruct a simple story, but the reconstruction is happening in your head. Add nested agents, parallel tools, reused operation names, and interleaved application logs, and timestamps stop being a reliable picture of causality.

An execution tree makes the relationship explicit:

support-agent
├── plan
├── fetch-inventory (failed: 503)
├── fetch-inventory (success)
└── draft-answer

The tree does not replace raw event data. It is a projection of that data for the question developers usually ask first: What path did this run take?

Capture meaningful boundaries in TypeScript

AgentInspect provides wrappers for a run and for named steps. Here is a deliberately small example:

import { inspectRun, step } from "agent-inspect";

await inspectRun(
  "travel-planner",
  async () => {
    const plan = await step("plan", async () => ({
      destinations: ["SFO", "SEA"],
    }));

    const [flights, hotels] = await Promise.all([
      step.tool("search-flights", async () => [
        { id: "F-101", price: 220 },
      ]),
      step.tool("search-hotels", async () => [
        { id: "H-202", nightly: 180 },
      ]),
    ]);

    return step.llm("rank-options", async () => ({
      plan,
      flights,
      hotels,
    }));
  },
  { traceDir: "./.agent-inspect" },
);

This is manual instrumentation. It does not claim that a wrapper can automatically discover every framework-internal operation. The purpose is to record the boundaries you care about: the run, its planning step, the two sibling tool calls, and the final model-facing step.

Then inspect the run locally:

npx agent-inspect view travel-planner \
  --dir .agent-inspect \
  --summary

Four shapes that reveal different classes of bugs

1. Nested work exposes ownership

A three-level synthetic fixture renders like this:

Execution Tree:
✔ outer (120ms)
  ✔ middle (80ms)
    ✔ inner (50ms)

Those two spaces are not decoration. They tell us that inner belongs to middle, which belongs to outer. If inner fails, we know which higher-level operation owned it. With flat logs, matching IDs or surrounding timestamps would be required to infer the same structure.

Nesting is especially useful when one agent delegates to another, a tool performs several sub-operations, or a retrieval step owns both a query rewrite and a vector search.

2. Fallbacks expose recovery behavior

Now consider an error-recovery fixture:

Execution Tree:
✖ tool:primary-search (100ms)
    Error: primary search unavailable
✔ tool:fallback-search (200ms)
✔ handle-recovered-result (50ms)

The final run may still be successful. If we looked only at the answer, the failed primary search could disappear from the debugging story. The tree preserves both facts:

• the primary path failed;

• the recovery path completed.

That distinction can change the engineering decision. A successful answer produced by a fallback may be acceptable, but a sudden rise in fallback use could still indicate a degraded dependency or an expensive routing change.

3. Repeated siblings expose retries

Retries deserve their own visible shape:

Execution Tree:
✖ tool:fetch-inventory (40ms)
    Error: synthetic 503 from upstream
✖ tool:fetch-inventory (45ms)
    Error: synthetic 503 from upstream
✔ tool:fetch-inventory (60ms)
✔ handle-recovered-result (30ms)

A final success status would hide the cost of reaching success. The repeated tool name makes the retry sequence visible. It also gives a deterministic check something concrete to evaluate: for example, whether fetch-inventory exceeded an allowed call count.

The tree alone does not tell us whether the retry policy was correct. It gives us evidence that the policy was exercised.

4. Parallel siblings expose concurrency

A parallel fixture renders as sibling operations:

Execution Tree:
✔ tool:search-hotels (300ms)
✔ tool:search-flights (200ms)
✔ tool:search-cars (100ms)

The durations are not meant to be added. These steps are siblings, and may overlap. That protects us from a common timeline mistake: assuming each timestamped operation waited for the previous one.

The tree does not prove that concurrency was optimally implemented, but it accurately preserves the structural relationship needed to investigate it.

Trees are a view, not the entire evidence model

It is tempting to turn a readable tree into the only stored artifact. I avoided that because a human-readable view necessarily compresses information.

The underlying trace may include identifiers, timestamps, status, inputs or outputs (subject to capture policy), observations, and metadata. Different questions need different projections:

structured trace
├── tree      -> what path happened?
├── check     -> did an invariant hold?
├── diff      -> what changed between runs?
├── report    -> what should a reviewer read?
└── bundle    -> what evidence can be shared?

An execution tree is the fastest entry point, not a substitute for checks or analysis.

Turn a suspicious shape into a deterministic check

Suppose the retry tree reveals that an inventory tool can run three times. If the intended policy permits at most two calls, encode that expectation rather than relying on future visual inspection.

At the CLI level, a trajectory check can require tools and fail on recorded observations:

npx agent-inspect check travel-planner \
  --dir .agent-inspect \
  --preset trajectory \
  --required-tool search-flights \
  --fail-on-observation failed

For richer rules, AgentInspect exposes an experimental TraceContract API that can express tool requirements, forbidden tools, maximum calls, ordering, run status, duration, model allowlists, and token ceilings. Because that API is beta in the referenced release, pin the version and test the exact semantics before using it as a CI gate.

The important workflow is broader than one API:

1. inspect the tree;

2. identify a stable behavioral invariant;

3. encode it as a deterministic check;

4. keep human judgment for context-dependent questions.

What the tree cannot tell you

A clean tree does not prove that an answer is correct. A required retrieval step may return irrelevant documents. A model call may produce unsupported claims. A tool can succeed technically while returning stale data.

Execution trees are strongest for structural questions:

• Which operations ran?

• Which operation owned a failure?

• Was a fallback or retry used?

• Which work happened as siblings?

• Where did a run stop?

Use semantic evaluators, domain tests, and human review for content quality. The most reliable agent debugging workflow combines these layers rather than asking one visualization to answer every question.

Debug the path, not only the answer

The final response is what the user sees, but the execution path is what the engineer can improve. A tree turns that path from an inferred narrative into a concrete artifact.

That is the design principle behind AgentInspect’s local view: preserve causal structure, expose unsuccessful work even when recovery succeeds, and make suspicious patterns easy to convert into repeatable checks.

You can explore the exact release used here on GitHub. If you try it, start with a synthetic failure-and-fallback fixture. A perfect happy path is the least interesting test of a debugger.

Again.

The final answer is wrong, but the logs look fine:

tool call started
model call started
tool call completed
model call completed
fallback used
error: timeout

Which tool caused the timeout? Did the model answer before retrieval finished? Was the fallback expected? Did the agent call the same tool twice?

This is where console.log stops feeling like debugging and starts feeling like archaeology.

I kept hitting this problem while building TypeScript AI agents. Once the flow moved beyond a single model call, the debugging loop became a system:

plan → retrieve → rank → generate → validate → maybe retry → maybe hand off

Flat logs lost the structure. Output only tests could miss a bad path that happened to produce a plausible answer. Model graded evals helped with semantic quality, but they were a poor fit for every deterministic CI rule. And raw traces were too risky to paste casually into issues or pull requests.

So I built agent-inspect.

AgentInspect is a local evidence debugger and trajectory-test toolkit for TypeScript AI agents.

It turns one local trace into three things: a readable execution tree, a deterministic regression gate, and a derived Evidence v2 bundle that you can review before sharing.

No account. No collector. No default upload. Metadata only by default.

one local JSONL trace
├─ Debug   → view · report · explain
├─ Prevent → check · contract · CI
└─ Share   → redact · bundle · verify

The bug is often in the trajectory

A support agent can return a plausible answer after doing almost everything wrong.

The healthy path might be:

plan-request
└─ retrieve_policy
   └─ rank-results
      └─ generate_answer
         └─ policyShown: passed

The regression might be:

generate_answer          <- answered before retrieval
retrieve_policy
retrieve_policy          <- duplicate call
search_docs              <- wrong tool, failed
policyShown: failed

An output only test may pass. A flat log may contain every event. Neither makes the wrong path obvious.

The final answer is only one fact about the run. Tool choice, ordering, repetition, completion, duration, token usage, and observed outcomes are facts too. Together, those facts form the agent's trajectory.

That trajectory should be inspectable. It should also be testable.

Make execution boundaries explicit

You can start with manual instrumentation:

import { inspectRun, observeOutcome, step } from "agent-inspect";

const answer = await inspectRun(
  "support-agent",
  async () => {
    const policy = await step(
      "retrieve_policy",
      () => retrievePolicy(),
      {
        type: "tool",
        metadata: { toolName: "retrieve_policy" },
      },
    );

    const result = await step(
      "generate_answer",
      () => draftAnswer(policy),
      {
        type: "llm",
        metadata: { model: "your-model" },
      },
    );

    await observeOutcome("policyShown", {
      expectation: "The answer cites a retrieved policy",
      status: "passed",
      method: "custom",
    });

    return result;
  },
  { traceDir: ".agent-inspect" },
);

The wrapper records those boundaries as local JSONL while preserving the application's return value and errors. Raw prompts and model outputs are not required for the core workflow.

If your application already emits structured logs or uses AI SDK, OpenAI Agents JS, LangChain, or LangGraph you can use an adapter or reader instead of wrapping every step manually.

Get a useful result without an API key

The shortest path uses a generated synthetic demo:

npm install agent-inspect
npx agent-inspect init --yes
node examples/agent-inspect-demo.mjs
npx agent-inspect list --dir .agent-inspect

init writes a small config and demo into your project. The demo does not call a model or upload a trace.

Copy the run ID printed by list, then use the same local artifact for the three jobs below.

1. Debug: read the execution tree

npx agent-inspect view <run-id> --dir .agent-inspect --summary
npx agent-inspect report <run-id> --dir .agent-inspect
npx agent-inspect explain <run-id> --dir .agent-inspect

The tree restores the structure that flat logs lose: nested steps, tool and model calls, durations, safe metadata, errors, and observed outcomes. explain summarizes local trace facts deterministically; its default path makes no provider call.

The useful question changes from:

Did the run fail?

to:

Where did the passing and failing trajectories first diverge?

That distinction matters when the visible answer looks fine but the agent skipped a required retrieval, safety, or validation step.

2. Prevent: turn the path into a CI gate

Some agent quality questions are subjective. Helpfulness, tone, and open ended answer quality can benefit from model graded evaluation.

But many regressions are structural:

• Was retrieve_policy called?

• Did the forbidden search_docs tool appear?

• Did generation happen before retrieval?

• Did the run complete?

• Did an observed outcome fail?

• Did the agent exceed a duration or token budget?

Those checks do not need another model. They can be deterministic:

npx agent-inspect check <run-id> --dir .agent-inspect \
  --preset trajectory \
  --required-tool retrieve_policy \
  --forbidden-tool search_docs \
  --fail-on-observation failed

The preset and explicit shorthand rules are additive. A healthy run exits 0; a trajectory-rule failure exits 1.

For the committed regression fixture, the result is concrete:

Check status: fail
Summary: 2 failed, 0 warning(s), 0 error(s)

- outcome.status: Observed outcome count 1 matched [failed].
- tool.usage: Forbidden tool search_docs appeared.

Same trace. Same rules. Same verdict. No model judge and no provider call in the check path.

That makes it suitable for a normal CI step. If your test fixture writes a trace to a stable path:

- name: Run deterministic agent fixture
  run: node run-agent-fixture.mjs

- name: Check agent trajectory
  run: |
    npx agent-inspect check .agent-inspect/ci-run.jsonl \
      --preset trajectory \
      --required-tool retrieve_policy \
      --fail-on-observation failed \
      --evidence-on fail

--evidence-on fail writes local Evidence for triage when the check fails. It does not upload the artifact.

When CLI flags outgrow one command, the Beta TraceContract API expresses the same expectations in TypeScript:

import { openTraceFile } from "agent-inspect/readers";
import {
  defineTraceContract,
  evaluateTraceContractRead,
} from "agent-inspect/checks";

const read = await openTraceFile("./.agent-inspect/demo-regression.jsonl");

const contract = defineTraceContract({
  run: { requireCompleted: true },
  tools: {
    required: ["retrieve_policy"],
    forbidden: ["search_docs"],
  },
  observations: { failOn: ["failed"] },
});

const result = evaluateTraceContractRead(read, contract);
if (result.status !== "pass") process.exitCode = 1;

The principle is simple: use deterministic trace facts for structural CI rules, and reserve model grading for semantic quality.

3. Share: derive reviewable Evidence v2

A failing trace is often the best debugging artifact. It can also contain prompts, tool arguments, retrieved documents, customer identifiers, error messages, or secrets.

The collaboration strategy should not be “paste the raw trace into Slack.”

AgentInspect keeps the source trace read-only and creates a derived bundle:

npx agent-inspect verify-safe <run-id> --dir .agent-inspect
npx agent-inspect bundle <run-id> --dir .agent-inspect \
  --profile share \
  --out ./evidence
npx agent-inspect bundle verify ./evidence

The bundle can include:

evidence.html          self-contained offline review surface
evidence.json          versioned manifest and SHA-256 file hashes
trace.jsonl            redacted derived trace
check-results.json     deterministic findings
redaction-report.json  detector summary without secret values
summary.md             human-readable overview

bundle verify checks the manifest, listed files, hashes, assessment, and provenance offline.

This is an integrity check. It is not a signature or a compliance certificate.

The wording matters: the artifact is share-checked, not “certified safe.” verify-safe and redaction are best-effort controls. Review the generated HTML and safety results before attaching a bundle to a pull request, incident, or public issue.

Use the capture path that matches your stack

The local evidence model is not tied to one agent framework.

The root package is enough for custom capture, the CLI, deterministic checks, and Evidence. Optional packages add only the integration you need.

Optional: give a coding assistant the same facts

The Preview MCP path exposes configured local evidence through bounded, read-only tools:

npx agent-inspect mcp configure --client cursor

The command is a dry run by default, so you can review the generated configuration before enabling it.

A connected coding assistant can then investigate the same TraceFacts used by the CLI: What failed first? Which required tool was missing? What changed between the passing and failing runs?

This is not replay, an auto fix engine, or a hidden upload path. It is optional read-only access to explicitly configured local evidence.

Where it fits—and where it does not

AgentInspect owns the laptop to pull request evidence loop:

capture locally
→ understand the path
→ fail CI on structural drift
→ derive reviewable evidence

It complements hosted observability and evaluation platforms. Use hosted tools when you need production dashboards, long term retention, fleet wide alerting, team wide trace search, hosted datasets, or prompt management.

Use AgentInspect when you need to inspect one TypeScript agent run immediately, enforce deterministic trajectory expectations in CI, compare a passing and failing local run, or hand off a redacted, hash-verifiable artifact.

The boundary is intentional. AgentInspect is not:

• A maintainer hosted SaaS or production APM replacement

• A hosted trace retention or prompt-management service

• An LLM as judge or dataset platform by default

• A replay or automatic remediation engine

• A chain of thought recorder

• A compliance certification tool

Try the complete loop

The current release is 6.17.4, requires Node.js 20 or newer, uses persisted schema 1.0, and is MIT licensed. Legacy v0.1 and v0.2 traces remain readable.

npm install agent-inspect
npx agent-inspect init --yes
node examples/agent-inspect-demo.mjs
npx agent-inspect list --dir .agent-inspect

Then inspect, check, and derive Evidence from the run:

npx agent-inspect view <run-id> --dir .agent-inspect --summary
npx agent-inspect check <run-id> --dir .agent-inspect --preset trajectory
npx agent-inspect bundle <run-id> --dir .agent-inspect \
  --profile share \
  --out ./evidence
npx agent-inspect bundle verify ./evidence

• Documentation

• npm package

• GitHub repository

• Keyless Debug / Prevent / Share starter

• Framework starters

• Discussions

One local trace should be able to tell you what the agent did, prove that the regression stays fixed, and give a teammate evidence they can review without making upload the price of admission.

How you can help

AgentInspect is open source and MIT licensed. If this workflow is useful to you:

• Star the repository so more TypeScript agent developers can find it.

• Try the keyless demo and open an issue if anything feels confusing.

• Add a starter or recipe for your stack.

• Share a real debugging workflow you want the project to support.

• Pick up a good first issue or contribute documentation.

Most of all, leave a comment below. How do you debug agent runs today? Which trajectory rule would you put in CI first? And if AgentInspect does not fit your workflow, tell me why that feedback is just as useful.