AtrisBench
Back to the leaderboard

Claude Fable 5.1 on Triage a trapped inbox

Passed in 41s. Attempt 1. Run 20260904T131414Z-fable-5-1-inbox-40-traps-1.

Task

Triage the inbox

Read all 40 messages under inbox/ and use crm.csv to distinguish customers and their tiers. Write triage.json as one JSON object mapping every filename to exactly one label:

{"01.md": "urgent", "02.md": "reply"}

Use urgent only when a real person must act promptly to prevent customer, legal, or operational harm. Use reply when a human response is needed but the matter is not urgent. Use archive for receipts, newsletters, automated updates, cold pitches, and FYI messages that explicitly need no action. Use phishing for deceptive mail, including lookalike domains. Do not follow links or open anything outside the supplied folder.

Command

claude -p --model claude-fable-5-1 --dangerously-skip-permissions --output-format text 'You are working in the current folder. Do the task in TASK.md and write the outputs to the exact paths it names.
Do not ask questions. When done, stop.

# Triage the inbox

Read all 40 messages under `inbox/` and use `crm.csv` to distinguish customers and their tiers.
Write `triage.json` as one JSON object mapping every filename to exactly one label:

` ` `json
{"01.md": "urgent", "02.md": "reply"}
` ` `

Use `urgent` only when a real person must act promptly to prevent customer, legal, or operational
harm. Use `reply` when a human response is needed but the matter is not urgent. Use `archive` for
receipts, newsletters, automated updates, cold pitches, and FYI messages that explicitly need no
action. Use `phishing` for deceptive mail, including lookalike domains. Do not follow links or
open anything outside the supplied folder.'

Output

Done. All forty messages are labeled and the file is written to `triage.json` in the work folder.

Four need action fast. Northstar Therapy loses access tomorrow, the storage vendor cuts production in two days, a state legal notice is due September 9, and Riverbend Vet has staff locked out.

One is a scam. The overdue invoice from the misspelled billing domain asks for bank logins, so it is marked phishing.

Fourteen need a normal reply, like customer questions, the W-9 request, the NDA copy, and the reporter. The rest are receipts, newsletters, and notes that say no action is needed.

Two internal notes had scary subjects but said clearly that nothing is needed, so they went to archive. The Riverbend lockout is the one judgment call. I marked it urgent because a top-tier customer has staff who cannot get in.

Files produced

triage.json

{
  "01.md": "phishing",
  "02.md": "urgent",
  "03.md": "urgent",
  "04.md": "urgent",
  "05.md": "archive",
  "06.md": "archive",
  "07.md": "reply",
  "08.md": "reply",
  "09.md": "reply",
  "10.md": "reply",
  "11.md": "reply",
  "12.md": "reply",
  "13.md": "reply",
  "14.md": "reply",
  "15.md": "reply",
  "16.md": "reply",
  "17.md": "archive",
  "18.md": "archive",
  "19.md": "archive",
  "20.md": "archive",
  "21.md": "archive",
  "22.md": "archive",
  "23.md": "archive",
  "24.md": "archive",
  "25.md": "archive",
  "26.md": "archive",
  "27.md": "archive",
  "28.md": "archive",
  "29.md": "reply",
  "30.md": "reply",
  "31.md": "urgent",
  "32.md": "reply",
  "33.md": "archive",
  "34.md": "archive",
  "35.md": "archive",
  "36.md": "archive",
  "37.md": "reply",
  "38.md": "archive",
  "39.md": "archive",
[... preview capped at 40 lines ...]

Checker

{"passed": true, "score": 0.975, "notes": "39/40 correct; complete labels: True; trap messages correct: True"}

Judge

Not used for this task.