Armature Shows MCP Session Analytics

Product analytics (and evals) for agent sessions on your MCP is a Show HN project from Armature, founded by Theodore and Louis, that adds session analytics and evals around Model Context Protocol servers. It deals with a very specific blind spot: MCP builders can see tool calls, but not always the agent session that produced them. The useful takeaway is simple: session reconstruction turns mysterious agent behavior into product debugging evidence, and later into review evidence. For Claude Code users, that also helps when you implement code review habits for ai-generated code without pretending every agent trace is safe to collect.

Model Context Protocol is an open protocol for connecting AI agents to external tools, data, and services through a standard client-server interface. In practice, that means an agent in Claude Code, Anthropic's coding agent, or Claude, Anysphere's AI code editor, can ask an MCP server to read a repo, search docs, update an issue, or call an internal service.

Armature sits on the server side of that exchange. As of August 2026, its founders say you wrap an MCP server with a small SDK layer in TypeScript, Python, or Go, then view reconstructed sessions in a dashboard. The interesting claim is not the wrapper. It is the reconstructed context around the call: what the user asked the agent to do, what the agent was trying, which use cases cluster together, and which failures repeat.

That is why developers cared. Raw MCP logs answer “what endpoint was called?” Armature is trying to answer “what was the agent actually doing?”

Read the session, not just the tool call

An MCP tool call is usually too small to explain a bug. A call like search_docs({ query: "billing webhook retries" }) tells you the agent searched, but not whether the user was debugging production, writing tests, or asking for architecture advice.

Armature's pitch is that it reconstructs the larger session behind those calls. That can make an MCP dashboard feel less like request analytics and more like reading a replay of the agent's work: the user intent, the agent path, and the point where the integration stopped helping.

The trap is assuming “session” means perfect truth. Agents summarize, omit, and sometimes expose only the context the host client makes available. If a product says it captures what the agent thought, ask exactly what artifact that means: model reasoning text, tool-planning metadata, client-visible messages, or inferred steps from the trace.

A concrete example: suppose your docs MCP gets hundreds of searches for pnpm workspace filter. A normal log says the search endpoint is popular. A reconstructed session might show that agents are repeatedly failing to run tests in one package because your internal monorepo guide never explains pnpm --filter @acme/payments test.

Compare logs, sessions, and review notes

Armature is best understood as a third layer next to raw logs and human review notes. It does not replace either one.

Verdict: raw logs win for low-level debugging, Armature wins when the MCP server owner needs product analytics on agent behavior, and manual review notes win for merge decisions. The useful pattern is to connect them lightly: logs prove what happened, sessions explain why, and review notes decide what to change.

This is also where the agentic coding story gets more concrete. A lot of ai coding governance talk stays abstract. Session analytics gives you examples from real agent work, which is much more useful for the related training topic than another policy paragraph.

Keep the first integration narrow

The safest first version is read-only. Wrap one MCP server that already exposes low-risk data, such as public docs, internal runbooks with no secrets, or synthetic test fixtures. Do not start with production databases, customer records, or write-capable tools.

The “three lines of code” claim is plausible as an instrumentation wrapper, not as a complete security review. The hard part is not adding an SDK import. The hard part is deciding what context may leave your process, what gets redacted client side, who can view the dashboard, and how long traces are retained.

Here is a small decision table you can copy before connecting a real server.

Permission-boundary note: treat the analytics wrapper as an observer, not an authority. It should not grant new MCP powers. In Claude Code, keep any hook boundary separate: a PreToolUse hook can block risky writes, while session analytics can explain why the agent tried the action in the first place.

Turn sessions into review evidence

The practical move is not to review every token of every agent session. It is to sample the sessions that produced code changes, then attach one short receipt to the pull request.

This is the one place where the search question matters: what are the best ways to implement code review habits for ai-generated code? Start with evidence that is already close to the work. A reconstructed MCP session can show the prompt goal, the tools used, the errors hit, and whether the agent had the right repository context.

Use a small receipt, not a ceremony:

AI code review receipt

Change: <PR or branch name>
Agent goal: <one sentence from the session>
MCP tools used: <docs search, repo search, issue lookup, etc.>
Risky permissions: <none / read-only / write attempted>
Tests the agent ran: <commands or “not shown”>
Reviewer checked:
- [ ] The final diff matches the stated goal.
- [ ] The agent did not rely on stale or unrelated context.
- [ ] Any failed tool calls are understood.
- [ ] Generated tests fail for the right reason before the fix, if applicable.
- [ ] No secrets, customer data, or private prompts are needed to justify the merge.
Decision: <merge / revise / split / reject>

For a Claude Code repo, this pairs well with a tiny slash-command workflow: /review-agent-session <session-id> opens the trace, extracts the goal and tools used, and leaves the human reviewer to judge the diff. Keep durable repository rules in CLAUDE.md concise. Put the receipt in the PR, because code review happens where the code changes.

Common questions

• How is Armature getting the model's thoughts?

  Armature says it reconstructs the session behind MCP tool calls, including what the user asked and what the agent thought. The important caveat is that “thought” needs a precise product definition. Before sending sensitive work through any analytics layer, ask whether it captures explicit reasoning text, client-visible messages, tool metadata, inferred steps, or some mix of those artifacts.

• Does it really take three lines of code?

  It may take only a few lines to wrap an MCP server with an SDK, but that is not the whole integration. The real work is choosing the first server, testing redaction, setting dashboard access, and confirming that observability does not change permissions. Treat “three lines” as instrumentation setup, not as the security boundary.

• Can I disable collection or control what leaves my MCP server?

  You should require that before connecting sensitive systems. The Show HN discussion specifically raised client-side redaction, configurability, open-source redaction code, disabling collection, and what data leaves the process. Those are the right questions. If the answers are unclear, use only a sandbox or public-docs MCP server until they are documented.

• What are the best ways to implement code review habits for ai-generated code?

  The best way is to review generated code with the same standards as human code, plus one extra receipt for agent context. Keep the receipt short: goal, tools used, permissions touched, tests run, and unresolved failures. Session analytics helps because it gives reviewers evidence without asking them to replay an entire chat by memory.

• Does this replace ai coding training for teams?

  No, it gives training better raw material. Instead of teaching abstract agentic coding rules, you can inspect repeated failures from real MCP sessions and turn them into examples. That is especially useful when engineering team ai adoption stalls because nobody can explain what the agent was trying to do when it made a bad change.

Best ways to use this research

• Best for: MCP server maintainers who need to understand agent use cases, not just count endpoint calls.
• Best first artifact: A read-only session trace from a docs or sandbox MCP server, paired with the short AI code review receipt above.
• Best comparison angle: Compare Armature against raw MCP logs and manual PR notes; each answers a different question.
• Best safety boundary: Start with observation only. Do not let analytics instrumentation expand tool permissions, write access, or data scope.
• Best next question: Ask what exactly is captured, how redaction is configured, and whether collection can be disabled per server or environment.

For a nearby story on the same product surface, see Armature Adds Analytics for MCP Sessions.

Further reading

• Product analytics (and evals) for agent sessions on your MCP — source
• Model Context Protocol — specification
• Claude Code hooks — Anthropic documentation
• Claude — Agent

Try one safe trace

Pick one read-only MCP server and capture one non-sensitive agent session from start to finish. If the trace helps explain a real failure without exposing private data, you have a useful observability path; if it does not, fix the boundary before adding more servers.

One methodology lens

One useful way to read this through our methodology is the Plan step: delegate first-pass decomposition and dependency mapping, review the sequencing and assumptions, and keep ownership of scope and priorities. If that split is still fuzzy, the workflow usually is too.