Add Cloud-Offload Strategy panel (local backup for cloud credits)
Runnability-first split: TRUSTED/VERIFY (runs clean) vs AVOID (crashes). Only 5/10 local models produce runnable code. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -169,6 +169,16 @@ ul.findings-notes li:last-child{border-bottom:none}
|
||||
ul.findings-notes li::before{content:"▸";position:absolute;left:0;color:var(--mag)}
|
||||
ul.findings-notes code{background:rgba(0,255,200,0.1);color:var(--cyan);padding:1px 5px;border-radius:3px;font-family:'Fira Code',monospace;font-size:.8rem}
|
||||
@media(max-width:900px){.findings{grid-template-columns:repeat(2,1fr)}}
|
||||
/* cloud-offload strategy */
|
||||
.offload-panel{border:1px solid rgba(0,255,200,0.18)}
|
||||
.offload-intro{color:var(--ink);font-size:.86rem;line-height:1.55;margin-bottom:16px;padding:12px 14px;background:rgba(0,0,0,0.25);border-left:3px solid var(--cyan);border-radius:0 5px 5px 0}
|
||||
.tiers{display:grid;grid-template-columns:1fr 1fr;gap:14px}
|
||||
.tier-h{font-family:'Fira Code',monospace;font-size:.74rem;letter-spacing:.08em;text-transform:uppercase;margin-bottom:8px;padding-bottom:6px;border-bottom:1px solid currentColor}
|
||||
.tier-list{display:flex;flex-direction:column;gap:6px}
|
||||
.tier-list .empty{color:var(--dim);font-size:.8rem;padding:8px}
|
||||
a.tr{grid-template-columns:1fr auto auto auto auto !important}
|
||||
.tier-badge{font-size:.6rem;letter-spacing:.08em;padding:1px 6px;border:1px solid currentColor;border-radius:3px;font-family:'Fira Code',monospace}
|
||||
@media(max-width:760px){.tiers{grid-template-columns:1fr}}
|
||||
footer{color:var(--dim);font-size:.74rem;margin-top:40px;border-top:1px solid rgba(255,255,255,0.06);padding-top:14px;text-align:center}
|
||||
@media (prefers-reduced-motion: reduce){*{animation:none!important;transition:none!important}}
|
||||
"""
|
||||
@@ -395,6 +405,52 @@ def render_dashboard(data):
|
||||
</ul>
|
||||
</div>"""
|
||||
|
||||
# ---- Cloud-offload strategy panel (the actual goal: a backup for cloud credits) ----
|
||||
# A model is a useful offload ONLY if its code runs. Rank runnability-first.
|
||||
ranked = sorted(local, key=lambda m: (-int(m.get("tests_pass", False)), -m["total_score"]))
|
||||
runs = [m for m in local if m.get("tests_pass")]
|
||||
crashers = [m for m in local if not m.get("tests_pass")]
|
||||
fast_clean = [m for m in local if m.get("tests_pass") and m.get("tok_sec") and not m.get("speed_caveat")]
|
||||
fast_clean.sort(key=lambda m: -m["tok_sec"])
|
||||
|
||||
def _tier_row(m, badge, color):
|
||||
t = f"{m['tok_sec']:.0f} t/s" if m.get("tok_sec") else "—"
|
||||
return (f'<a class="tr" href="pages/{esc(m["id"])}.html">'
|
||||
f'<span class="tr-n">{esc(m["model_name"][:24])}</span>'
|
||||
f'<span class="tr-q mono">{esc(m["quant"])}</span>'
|
||||
f'<span class="tr-s mono">{t}</span>'
|
||||
f'<span class="tr-sc mono" style="color:{color}">{m["total_score"]}</span>'
|
||||
f'<span class="tier-badge" style="color:{color};border-color:{color}">{badge}</span></a>')
|
||||
|
||||
tier_rows = []
|
||||
if runs:
|
||||
# top runner(s) = recommended offload
|
||||
for m in runs:
|
||||
tier_rows.append(_tier_row(m, "TRUSTED" if m["total_score"] >= 78 else "VERIFY", verdict_meta(m["verdict"])[0]))
|
||||
crash_rows = [_tier_row(m, "AVOID", "var(--red)") for m in crashers]
|
||||
|
||||
offload_panel = f"""
|
||||
<div class="panel offload-panel" style="margin-bottom:26px">
|
||||
<h2>▮ CLOUD-OFFLOAD STRATEGY — what to actually run when rationing cloud credits</h2>
|
||||
<div class="offload-intro">
|
||||
Goal: a <b>trustworthy local backup</b> for when your cloud-LLM plan runs low. A model only earns offload
|
||||
duty if its code <b>runs</b> — a high score that crashes wastes your time <i>and</i> still costs a cloud call afterward.
|
||||
Only <b class="mono" style="color:var(--lime)">{len(runs)}/{len(local)}</b> local models produced runnable code;
|
||||
<b class="mono" style="color:var(--red)">{len(crashers)}/{len(local)}</b> crashed.
|
||||
</div>
|
||||
<div class="tiers">
|
||||
<div class="tier"><div class="tier-h" style="color:var(--lime)">✓ RUNS — offload candidates (verify output, then ship)</div>
|
||||
<div class="tier-list">{''.join(tier_rows) if tier_rows else '<div class="empty">none</div>'}</div></div>
|
||||
<div class="tier"><div class="tier-h" style="color:var(--red)">✗ CRASHES — do NOT offload (wasted cycle + cloud call anyway)</div>
|
||||
<div class="tier-list">{''.join(crash_rows) if crash_rows else '<div class="empty">none</div>'}</div></div>
|
||||
</div>
|
||||
<ul class="findings-notes" style="margin-top:14px">
|
||||
<li><b>Rule of thumb:</b> route a task to local only if the top runner above handles it; if the local output doesn't run on first try, <b>stop and use the cloud model</b> — debugging bad local code usually costs more than the API call you tried to save.</li>
|
||||
<li><b>Best pure-offload picks</b> (runs clean + fast + ≥good): {", ".join(f'<a href="pages/{esc(m["id"])}.html" style="color:var(--lime)">{esc(m["quant"])}</a>' for m in fast_clean[:3]) or "none yet"}.</li>
|
||||
<li><b>Don't trust the score alone.</b> The 4-bit Qwen is the fastest local model (83 t/s) but crashes — speed is meaningless if the code doesn't run.</li>
|
||||
</ul>
|
||||
</div>"""
|
||||
|
||||
body = f"""
|
||||
{head_html("LLM Benchmark Suite")}
|
||||
<header class="hud-bar">
|
||||
@@ -416,6 +472,7 @@ def render_dashboard(data):
|
||||
<div style="color:var(--dim);font-size:.72rem;margin-top:8px">Local models only — cloud baseline (DeepSeek) excluded from the speed axis. Bars flagged ⚠ have suspected GPU-offload / inference issues (not representative of the model).</div>
|
||||
</div>
|
||||
{findings_panel}
|
||||
{offload_panel}
|
||||
<div class="panel" style="margin-bottom:26px">
|
||||
<h2>▮ LEADERBOARD</h2>
|
||||
<div style="overflow-x:auto">
|
||||
|
||||
Reference in New Issue
Block a user