# claude-fable-5 transport record (safety-classifier boundary on text protocols)

claude-fable-5 (claude CLI, July 2026) cannot be evaluated over the bench's
text-protocol CLI transport: its dual-use safeguards flag the harness prompt,
so the CLI returns an API error instead of a model reply. Its published
episodes run over native tool-calling instead - see Resolution below.

```
API Error: Fable 5's safeguards flagged this message
(https://www.anthropic.com/legal/aup). This sometimes happens with safe, normal
conversations. Claude Code can't respond to this request with Fable 5.
```

## Evidence (2026-07-16)

- Sweep-v2 wrapper: 24/24 invocations flagged across 12 episodes, plus 4/4
  isolated replays (plan and default permission modes). Deterministic.
- Localization: the v2 protocol header ("you are the model inside an automated
  benchmark harness ... reply with EXACTLY ONE JSON object") and the
  tool-schema block each trip the flag independently; the plain "careful
  experimental scientist" system text passes and yields a clean tool call.
- A transparent rewording was built and tested (`protocol: "eval-v3"` in
  cli-client.ts): it states honestly that this is an open-source benchmark and
  the program executes the returned tool call. A hand-built v3 prompt passed
  4/4 - but the harness-assembled v3 prompt flagged 5/5, and isolation runs
  showed the decision is a boundary, not a phrase: with v3 framing, compact
  (vs spaced) schema JSON flips it to flagged, and so does the two-word
  phrasing delta "naming a benchmark tool" (each tested alone, 3/3
  deterministic per variant).
- Boundary + multi-turn = unusable: every later turn appends raw `TOOL
  RESULT:` stats JSON, more machine-shaped than the deltas that already flip
  the classifier. No honest wording can hold the pass side of the boundary
  across a whole episode, and tuning wording against a safety classifier is
  not a defensible evaluation practice.

## Resolution: native tool-calling transports

The flagged material is the text serialization, not the task - so the fix is
a transport where none of it exists as prose. Two are built:

- **Native MCP over the subscription CLI** (shipped; this is how the
  published fable-5 episodes were recorded): the bench harness runs as a
  stdio MCP server (`dist/mcp-bench-server.js`) and each episode is one
  `claude` session restricted with `--strict-mcp-config` and an
  `--allowedTools` allowlist of exactly the four harness tools
  (`native-session.ts`; `configs/frontier-v2-fable-native.json`). Tool
  schemas are the unmodified standardized definitions, results ride in
  structured tool fields, the model reads the same systemFor + briefFor text
  as every other transport, and the same Harness records the same RunLog. No
  safeguard flag in any recorded episode.
- **HTTP with an API key** (`configs/frontier-v2-fable-http.json`,
  `ANTHROPIC_API_KEY` with fable-5 access): equivalent shape over
  `/v1/messages`; kept as the BYOK alternative.

fable-5's leaderboard row carries a native-tools transport footnote. Its
first sweep was quota-paused at 52/90 episodes (L1 complete); resuming the
native config completes the cells in place.