Grade muse-glimmer-28b full 7-prompt battery — new benchmark leader
7 entries (30→37 total). Muse Glimmer 28B (GGUF) avg 80.7 — the strongest model in the benchmark, 6/7 prompts Minor Logic Flaws: lfu 76 | webhook 81 | automation 89 | rust 85 | data 88 | tts 58 | mcp 88 Standout results: - rust 85 (KAT 36, Qwen3-Coder 54) — real tokio channels (mpsc::channel, not hallucinated mpsc::bounded), two-tier CancellationToken, zero clippy lints; one-line E0507 compile fix. - automation 89 — first model to print a correct summary (98/2/0/100); atomic temp+fsync+rename checkpointing. - data 88 edges out Gemma-26B's 86; mcp 88 sets the bar on a new prompt. Only weak spot: tts 58 (backpressure raises instead of awaits, like Qwen3-Coder). Captured via the native /api/v1/chat fix (real tok/sec + TTFT). Slow deep-thinker: ~17-19 t/s, 5-9 min/prompt, ~5-9k tokens incl. reasoning. Also gitignore checkpoint.json (automation test runtime artifact). Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -1,13 +1,13 @@
|
||||
{
|
||||
"model_instance_id": "google/gemma-4-26b-a4b",
|
||||
"model_instance_id": "meta/muse-glimmer",
|
||||
"stats": {
|
||||
"input_tokens": 718,
|
||||
"total_output_tokens": 5153,
|
||||
"reasoning_output_tokens": 1919,
|
||||
"tokens_per_second": 53.47535329547263,
|
||||
"time_to_first_token_seconds": 1.127
|
||||
"input_tokens": 495,
|
||||
"total_output_tokens": 4749,
|
||||
"reasoning_output_tokens": 3228,
|
||||
"tokens_per_second": 20.201965201572563,
|
||||
"time_to_first_token_seconds": 2.571259
|
||||
},
|
||||
"response_id": "resp_d8fafc117fa4e51b927d8cd531ded49984ae59e49c39d3eb",
|
||||
"response_id": "resp_a37c5d6bb520db613250881b4919be086d4aed4a8f3a078a",
|
||||
"_output_types": [
|
||||
"reasoning",
|
||||
"message"
|
||||
|
||||
Reference in New Issue
Block a user