First multi-prompt result: Qwen 6-bit TTS = 49 (vs 82 LFU) + per-prompt schema

TTS grade for qwen3.6-35b-a3b-6bit-mlx: 49/100 Critical (same model that
scored 82 on LFU). File doesn't parse + bounded-concurrency is fake
(1 worker + inner semaphore = real concurrency 1). Per-task signal:
strong on data-structures, weak on async-pipeline work.

Schema: prompt_id + PILLARS_BY_PROMPT so each entry uses its own 5 pillars.
TODO_submission_tool.md sketches the grade-as-a-tool idea for later.

Co-Authored-By: Claude <noreply@anthropic.com>

TODO_submission_tool.md

@@ -0,0 +1,46 @@
+# TODO: a submission/grading tool (so grading is one command, not a manual pipeline)
+
+## The idea (from the user)
+Right now grading a model output is a manual multi-step dance:
+1. paste output → save to `outputs/<name>.py`
+2. run it, see if tests pass
+3. manually audit 5 pillars
+4. hand-write a JSON entry with tok/sec/tokens/TTFT/score/bugs/patch
+5. append to `data/benchmark_history.json`
+6. re-run `generate_dashboard.py` + redeploy
+
+The user wants a **tool** (likely an MCP server you can call from your editor/agent,
+or a CLI) where you say:
+> "grade this .py, model name = X, tok/sec = Y, tokens = Z, TTFT = W"
+… and it runs the tests, captures pass/fail + crash output, and stages the JSON entry
+(so I/the agent then do the actual *audit* — the subjective 5-pillar scoring + bug writeup —
+on top of the auto-collected facts).
+
+## What it should auto-collect (deterministic, no judgment)
+- [ ] Run the file; capture: parse OK? tests pass? stderr/crash output?
+- [ ] Static scan: `__slots__` present? `time.monotonic()` used? any `min/max/sorted/heapq`?
+- [ ] File metrics: line count, token-ish count
+- [ ] Append a **draft** entry to the JSON with `total_score: null` + `verdict: "pending"`
+      and the auto-fields filled, so the human/agent only fills the subjective parts.
+
+## What stays human/agent (the actual audit — can't be automated honestly)
+- The 5-pillar scores (0–20 each)
+- The critical-bugs list + the patch code
+- The verdict + best-for recommendation
+- The `prompt_id` (which exam: lfu / tts / mcp / rust / data / automation)
+
+## Two build options
+1. **CLI** (`./grade.sh outputs/foo.py --model "Qwen 6-bit" --tok 69 --tokens 4000 --ttft 0.9 --prompt tts`):
+   simplest, runs anywhere, no MCP setup. Prints the draft JSON entry + a summary.
+2. **MCP server** (`grade_tool`, `list_results_tool`, `regenerate_dashboard_tool`):
+   callable from Claude Code / your agent so you can grade from inside a chat. Needs the MCP
+   server running (the same pattern as your joplin/vault MCPs). More powerful but more setup.
+
+## Recommended path
+Start with the **CLI** (fast to build, works today, no LM-Studio/LiteLLM dependency).
+Promote to an MCP server later once you've got LiteLLM set up for the MCP-prompt testing —
+then the same MCP host can serve both the grading tool AND be the endpoint you test against.
+
+## Status
+Not started. Build after the multi-prompt schema (`prompt_id`) is in place, since the tool
+will need to tag which prompt an output is for.