7 entries (30→37 total). Muse Glimmer 28B (GGUF) avg 80.7 — the strongest
model in the benchmark, 6/7 prompts Minor Logic Flaws:
lfu 76 | webhook 81 | automation 89 | rust 85 | data 88 | tts 58 | mcp 88
Standout results:
- rust 85 (KAT 36, Qwen3-Coder 54) — real tokio channels (mpsc::channel, not
hallucinated mpsc::bounded), two-tier CancellationToken, zero clippy lints;
one-line E0507 compile fix.
- automation 89 — first model to print a correct summary (98/2/0/100);
atomic temp+fsync+rename checkpointing.
- data 88 edges out Gemma-26B's 86; mcp 88 sets the bar on a new prompt.
Only weak spot: tts 58 (backpressure raises instead of awaits, like Qwen3-Coder).
Captured via the native /api/v1/chat fix (real tok/sec + TTFT). Slow
deep-thinker: ~17-19 t/s, 5-9 min/prompt, ~5-9k tokens incl. reasoning.
Also gitignore checkpoint.json (automation test runtime artifact).
Co-Authored-By: Claude <noreply@anthropic.com>
webhook.py: HMAC-signed receiver (X-Gitea-Signature), validates ref==main,
one-concurrent-deploy lock, no request data reaches shell.
deploy-webhook.sh: installs llm-bench-webhook systemd service (runs as
aygea, in docker group), generates + stores secret in .webhook.secret.
deploy.sh: port read from compose (now 31415).
Installed on mewtwo: listening 0.0.0.0:41798, enabled for boot.
Gitea webhook target: http://10.0.0.22:41798/hook
Co-Authored-By: Claude <noreply@anthropic.com>
adminandClaude