Initial benchmark suite: 8 graded models + cyberpunk dashboard generator
- prompts/: LFU cache exam + 5-pillar grading rubric - outputs/: 8 model .py outputs (local + cloud baseline) - data/benchmark_history.json: graded results (scores, metrics, bugs, patches) - generate_dashboard.py: builds dashboard.html + pages/*.html from JSON - Dockerfile + DEPLOY.md: Gitea→Coolify deploy (build-step, nginx static) - .gitignore: generated HTML excluded (built on deploy) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude
+20
@@ -0,0 +1,20 @@
|
||||
# --- Generated / build artifacts ---
|
||||
# Dashboard HTML is generated by generate_dashboard.py from data/benchmark_history.json.
|
||||
# Keep them OUT of git so the repo only holds source — the site regenerates on deploy.
|
||||
# (If you'd rather commit the built HTML instead, comment out these lines.)
|
||||
/dashboard.html
|
||||
/pages/
|
||||
# /data/benchmark_history.json # <-- uncomment to keep history local-only too
|
||||
|
||||
# --- Scratch / previews / tooling ---
|
||||
*.png
|
||||
/playwright-mcp/
|
||||
/.playwright-mcp/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.DS_Store
|
||||
|
||||
# --- Local env ---
|
||||
.env
|
||||
.env.*
|
||||
*.local
|
||||
@@ -0,0 +1,72 @@
|
||||
# CLAUDE.md — Local LLM Benchmark Testing Project
|
||||
|
||||
## What this project is
|
||||
|
||||
A personal test harness for evaluating **local LLM models** running in **LM Studio** on an **Apple M3 Max MacBook Pro (48 GB unified memory)**. A fixed coding prompt (LFU cache) is fed to each model, the model's Python output is graded against a strict rubric, and the results are accumulated into a persistent JSON store and rendered into a standalone HTML dashboard.
|
||||
|
||||
This is **not** a git repo and is **not** an application with a build/run cycle. It is a folder of prompt files, model outputs, and generated artifacts.
|
||||
|
||||
## Folder layout
|
||||
|
||||
```
|
||||
prompts/
|
||||
lfu_cache_prompt.txt # The fixed prompt given to every model (the "exam question")
|
||||
grading.txt # The grader's instructions / rubric (read at session start)
|
||||
outputs/ # Raw model outputs (the .py files each model produced)
|
||||
<model-name>-<quant>.py
|
||||
data/
|
||||
benchmark_history.json # Persistent results store (created on first grading run)
|
||||
dashboard.html # Generated standalone dashboard (dark-mode, Chart.js via CDN)
|
||||
*.py # Loose model outputs in root (e.g. deepseekv4flash.py) — legacy/unsorted
|
||||
```
|
||||
|
||||
## The workflow (what to do when the user submits a model's output)
|
||||
|
||||
Follow the 4 steps in `prompts/grading.txt`:
|
||||
|
||||
1. **Audit** the `.py` file against the 5 pillars (each 0–20 → 0–100 total):
|
||||
- Complexity violations (true O(1) — no heaps, `sorted()`, linear scans)
|
||||
- Async race conditions / deadlocks (locks held across `sleep`/IO, unsynced shared state)
|
||||
- Transactional isolation leaks (read-your-own-writes, commit/rollback correctness)
|
||||
- Memory leaks & edge cases (DLL unlink, `min_freq` updates, `time.monotonic()`, `__slots__`)
|
||||
- Test coverage integrity (real edge cases vs trivial always-pass asserts)
|
||||
2. **Extract metrics**: model name, quant, tok/sec, verdict, archetype/best-for, critical bugs, patch code.
|
||||
3. **Update `data/benchmark_history.json`** — append/update the model's entry (schema in `grading.txt`). Create the file if missing.
|
||||
4. **Regenerate `dashboard.html`** — full overwrite, standalone single file, Chart.js (bar + radar), leaderboard table, collapsible audit cards, color-coded verdict badges (green 90+, blue/yellow 75–89, red <75).
|
||||
|
||||
The exam prompt (`lfu_cache_prompt.txt`) requires a **pure-stdlib Python 3.11+ async LFU cache** with frequency-bucket O(1), dual-layer TTL eviction, ACID-like transactions, and an `async def main()` test suite. Use it as the spec when judging correctness.
|
||||
|
||||
## Model output — naming convention
|
||||
|
||||
Model output filenames encode the **model + quant/format** so a model is identifiable from the filename alone. The convention the user uses:
|
||||
|
||||
`<model-family><version>-<param-size>-<variant>-<quant-or-format>.py`
|
||||
|
||||
Lowercase, hyphen-separated, no spaces. Examples seen so far:
|
||||
|
||||
| Filename | Reads as |
|
||||
|---|---|
|
||||
| `qwen3.6-35b-a3b-6bit-mlx.py` | Qwen 3.6, 35B-A3B (MoE), 6-bit, MLX format |
|
||||
| `qwen3.6-35b-a3b-uncensored-hauhaucs-aggressive-gguf.py` | Qwen 3.6 35B-A3B, uncensored fine-tune, GGUF |
|
||||
| `gemma-4-31b-qat-gguf.py` | Gemma 4 31B, QAT, GGUF |
|
||||
| `gemma-4-12b-coder-fable5-composer2.5-v1-uncensored-heretic-mxfp8-mlx.py` | merged/model-card-style name, MLX |
|
||||
| `kat-coder-v2.5-dev-xl-mlx.py` | KAT Coder v2.5 Dev XL, MLX |
|
||||
|
||||
Tokens: `mlx` = Apple MLX format; `gguf` = llama.cpp GGUF; quant suffixes like `q6k`, `6bit`, `4bit`, `qat`, `mxfp8` go in the name. When in doubt about what a filename denotes, **parse it loosely into model + quant** rather than guessing a wrong label — the grading step pulls model/quant from "file name or user input," so either source is valid.
|
||||
|
||||
### When the user pastes raw output instead of giving a file
|
||||
|
||||
If the user pastes a model's output text directly (no file), **save it as a `.py` file in `outputs/`** using the naming convention above before grading. Ask the user for the model name + quant only if it cannot be reasonably inferred from context. Match the existing lowercase-hyphen style.
|
||||
|
||||
## Hardware / runtime context
|
||||
|
||||
- **Machine:** Apple M3 Max, 48 GB unified memory. Local inference only.
|
||||
- **Inference server:** LM Studio (OpenAI-compatible local endpoint).
|
||||
- Quants/formats that fit 48 GB comfortably: ~35B dense at 4-bit/6-bit, larger MoE models (only active params loaded). MLX is preferred for Apple Silicon; GGUF via llama.cpp also works.
|
||||
|
||||
## Notes / gotchas
|
||||
|
||||
- `data/` starts **empty** — `benchmark_history.json` does not exist until the first grading run creates it. Don't assume it's there; read defensively, create on write.
|
||||
- `dashboard.html` is **regenerated** (overwritten), not appended to. The source of truth is the JSON file.
|
||||
- No build step, no tests to run, no dependencies to install — pure stdlib Python outputs + a static HTML file.
|
||||
- Treat the loose `.py` files in the project root (like `deepseekv4flash.py`) as unsorted outputs that belong in `outputs/` per the convention. Don't move them unless asked.
|
||||
@@ -0,0 +1,76 @@
|
||||
# Deploying to Coolify (via Gitea)
|
||||
|
||||
Repo: `ssh://git@git.itsaygea.com:2222/admin/modelTesting.git`
|
||||
|
||||
The generated `dashboard.html` and `pages/` are **gitignored** — they're built from
|
||||
`data/benchmark_history.json` by `generate_dashboard.py`. So Coolify must run that script
|
||||
during build, then serve the result.
|
||||
|
||||
---
|
||||
|
||||
## Option A — Build step, regenerate on every push *(recommended)*
|
||||
|
||||
The site always reflects the latest `benchmark_history.json`. Push a new grade → re-deploy → site updates.
|
||||
|
||||
### Coolify service setup
|
||||
|
||||
1. **New Resource** → choose your Gitea project `admin/modelTesting`.
|
||||
2. Choose **"Dockerfile"** as the build pack (we ship a tiny one).
|
||||
|
||||
Add this `Dockerfile` to the repo (already included):
|
||||
|
||||
```dockerfile
|
||||
# build stage: regenerate the site from source
|
||||
FROM python:3.12-slim AS build
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN python3 generate_dashboard.py
|
||||
|
||||
# serve stage: static files via nginx
|
||||
FROM nginx:alpine
|
||||
COPY --from=build /app/dashboard.html /usr/share/nginx/html/dashboard.html
|
||||
COPY --from=build /app/pages /usr/share/nginx/html/pages
|
||||
# root path redirects to the dashboard
|
||||
RUN printf 'location = / { return 302 /dashboard.html; }\n' > /etc/nginx/conf.d/default.conf
|
||||
EXPOSE 80
|
||||
```
|
||||
|
||||
3. **Port:** set the published port to **80** (Coolify maps it to your domain).
|
||||
4. **Domain:** attach your domain. Coolify provisions the HTTPS cert automatically.
|
||||
5. **Deploy.** Push to `main` → Coolify rebuilds → site updates.
|
||||
|
||||
### Why a Dockerfile
|
||||
|
||||
Coolify's "Nixpacks / static" presets can run a build command, but the Dockerfile path is
|
||||
the most predictable: explicit Python build step + nginx static serving, no surprises.
|
||||
The image is ~30 MB.
|
||||
|
||||
---
|
||||
|
||||
## Option B — Pure static, commit the built HTML *(simplest, no build)*
|
||||
|
||||
If you'd rather not build on the server:
|
||||
|
||||
1. Comment out `/dashboard.html` and `/pages/` in `.gitignore`.
|
||||
2. Run `python3 generate_dashboard.py` locally, then `git add` + commit the HTML.
|
||||
3. In Coolify: **New Resource** → the repo → build pack **"Static Site"** (or Nixpacks with
|
||||
`OUTPUT_DIRECTORY=/`), publish dir `/`, port 80.
|
||||
|
||||
Downside: you must rebuild + commit locally every time you add a model. Use Option A unless
|
||||
your Coolify can't run Docker.
|
||||
|
||||
---
|
||||
|
||||
## Verifying after deploy
|
||||
|
||||
Open the domain → you should land on the dashboard. If you see a blank page, check the
|
||||
Coolify build logs for `generate_dashboard.py` errors (usually a malformed
|
||||
`data/benchmark_history.json`).
|
||||
|
||||
## Local preview before pushing
|
||||
|
||||
```bash
|
||||
python3 generate_dashboard.py
|
||||
python3 -m http.server 8000
|
||||
# open http://localhost:8000/dashboard.html
|
||||
```
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
# Build stage: regenerate the static site from source data.
|
||||
FROM python:3.12-slim AS build
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN python3 generate_dashboard.py
|
||||
|
||||
# Serve stage: nginx serves the generated static files.
|
||||
FROM nginx:alpine
|
||||
# Replace the default nginx server block so root redirects to the dashboard
|
||||
RUN printf '%s\n' \
|
||||
'server {' \
|
||||
' listen 80;' \
|
||||
' server_name _;' \
|
||||
' root /usr/share/nginx/html;' \
|
||||
' index dashboard.html;' \
|
||||
' location = / { return 302 /dashboard.html; }' \
|
||||
'}' > /etc/nginx/conf.d/default.conf
|
||||
COPY --from=build /app/dashboard.html /usr/share/nginx/html/dashboard.html
|
||||
COPY --from=build /app/pages /usr/share/nginx/html/pages
|
||||
EXPOSE 80
|
||||
@@ -0,0 +1,53 @@
|
||||
# 🧪 Local LLM Benchmark Suite
|
||||
|
||||
Grades local LLM models (run via **LM Studio** on an Apple M3 Max / 48 GB) on a strict
|
||||
systems-coding prompt — a pure-stdlib Python **concurrent async LFU cache with TTL
|
||||
eviction and ACID transactions** — and renders the results into a cyberpunk-terminal
|
||||
dashboard.
|
||||
|
||||
## What's in here
|
||||
|
||||
```
|
||||
prompts/
|
||||
lfu_cache_prompt.txt # the exam prompt every model gets
|
||||
grading.txt # the grader's rubric (5 pillars × 20 pts = 100)
|
||||
outputs/ # raw model .py outputs (named <model>-<quant>.py)
|
||||
data/
|
||||
benchmark_history.json # persistent results store (source of truth)
|
||||
generate_dashboard.py # reads the JSON → builds dashboard.html + pages/*.html
|
||||
dashboard.html # generated — main leaderboard + charts (gitignored)
|
||||
pages/ # generated — per-model detail pages (gitignored)
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Feed `prompts/lfu_cache_prompt.txt` to a model in LM Studio.
|
||||
2. Save its output to `outputs/<model>-<quant>.py`.
|
||||
3. Grade it (audit the 5 pillars, capture tok/sec + tokens + TTFT), and append its
|
||||
entry to `data/benchmark_history.json`. See `prompts/grading.txt` for the rubric.
|
||||
4. Regenerate the site:
|
||||
|
||||
```bash
|
||||
python3 generate_dashboard.py
|
||||
```
|
||||
|
||||
This (re)writes `dashboard.html` and every `pages/<model>.html`.
|
||||
|
||||
## Viewing locally
|
||||
|
||||
```bash
|
||||
python3 -m http.server 8000
|
||||
# open http://localhost:8000/dashboard.html
|
||||
```
|
||||
|
||||
## Deploying (Gitea + Coolify)
|
||||
|
||||
The repo holds **source only** (`outputs/`, `data/`, `generate_dashboard.py`, `prompts/`).
|
||||
The generated `dashboard.html` and `pages/` are gitignored — Coolify runs
|
||||
`python3 generate_dashboard.py` as a build step, then serves the static files.
|
||||
See `DEPLOY.md` for the exact Coolify service config.
|
||||
|
||||
## Hardware
|
||||
|
||||
Apple M3 Max, 48 GB unified memory. Local inference via LM Studio.
|
||||
A cloud model (DeepSeek V4 Flash) is included as a quality baseline.
|
||||
@@ -0,0 +1,269 @@
|
||||
{
|
||||
"meta": {
|
||||
"project": "Local LLM Benchmark Suite — LFU Cache & ACID Audit",
|
||||
"machine": "Apple M3 Max, 48GB unified memory, LM Studio",
|
||||
"exam_prompt": "prompts/lfu_cache_prompt.txt",
|
||||
"grading_rubric": "prompts/grading.txt",
|
||||
"pillars": ["complexity", "concurrency", "isolation", "memory_edge_cases", "test_integrity"],
|
||||
"max_per_pillar": 20,
|
||||
"schema_version": 1
|
||||
},
|
||||
"models": [
|
||||
{
|
||||
"id": "qwen3.6-35b-a3b-6bit-mlx",
|
||||
"timestamp": "2026-07-28T13:00:00Z",
|
||||
"model_name": "Qwen 3.6 35B-A3B",
|
||||
"quant": "6-bit MLX",
|
||||
"format": "mlx",
|
||||
"tok_sec": 68.86,
|
||||
"total_tokens": 14883,
|
||||
"ttft_sec": 0.94,
|
||||
"filename": "outputs/qwen3.6-35b-a3b-6bit-mlx.py",
|
||||
"tests_pass": true,
|
||||
"total_score": 82,
|
||||
"breakdown": {
|
||||
"complexity": 16,
|
||||
"concurrency": 16,
|
||||
"isolation": 17,
|
||||
"memory_edge_cases": 16,
|
||||
"test_integrity": 17
|
||||
},
|
||||
"verdict": "Minor Logic Flaws",
|
||||
"best_for": "Solid daily-driver scaffolding for ACID/async patterns — produces runnable, well-structured code, but needs a human pass for __slots__, monotonic clocks, and lock granularity before production.",
|
||||
"critical_bugs": [
|
||||
"No __slots__ declared on Node/Transaction/LFUCache — rubric explicitly required it for memory efficiency.",
|
||||
"Uses time.time() (system clock) throughout instead of time.monotonic() — NTP adjustments can cause premature/incorrect TTL eviction.",
|
||||
"_cleanup_freq_lists() performs a hidden O(F) scan (iterates all freq tiers + min()), called after every eviction, background batch, AND inside commit() — violates the strict O(1) requirement.",
|
||||
"Transaction commit holds the single cache lock across all write/delete/bump loops + cleanup — coarse-grained, blocks all readers for the whole commit window; no fine-grained locking.",
|
||||
"Lost-update risk: commit applies tx-local writes without any MVCC/version check, so a key modified by the background evictor or another committer between tx.get() and commit() is overwritten blindly.",
|
||||
"Tests dodge hard cases: 50-task stress uses unique keys with capacity 100, so no eviction-under-contention ever happens; no test for rollback-after-partial-application or mid-commit read isolation."
|
||||
],
|
||||
"patch_code": "# FIX 1: Add __slots__ for memory efficiency\n@dataclass\nclass Node:\n __slots__ = ('key', 'value', 'freq', 'expires_at', 'prev', 'next')\n key: Any\n value: Any\n freq: int\n expires_at: Optional[float]\n prev: Optional['Node'] = None\n next: Optional['Node'] = None\n\n# FIX 2: Use monotonic clock everywhere (get/put/_add_node/commit)\n# time.time() -> time.monotonic()\n# e.g.\nexpires_at = time.monotonic() + ttl_seconds if ttl_seconds else None\n\n# FIX 3: Make _cleanup_freq_lists O(1) — bump min_freq incrementally\n# instead of recomputing min() across all tiers:\n# In _update_freq, when emptying the min_freq bucket, only bump min_freq\n# if you're evicting from it; otherwise leave it. Delete the global\n# min(self.freq_map.keys()) scan. For background sweeps, prune empty\n# buckets lazily on next _evict() rather than scanning proactively.\n\n# FIX 4: Shrink commit critical section — apply writes into a staging\n# structure under the lock, then release; or use per-bucket locks so\n# readers on unrelated keys aren't blocked.\n\n# FIX 5: Add MVCC version to Node; in commit, raise/abort if the\n# stored version != the version seen at tx.get() time (lost-update detect)."
|
||||
},
|
||||
{
|
||||
"id": "qwen3.6-35b-a3b-4bit-mlx",
|
||||
"timestamp": "2026-07-28T13:05:00Z",
|
||||
"model_name": "Qwen 3.6 35B-A3B",
|
||||
"quant": "4-bit MLX",
|
||||
"format": "mlx",
|
||||
"tok_sec": 83.31,
|
||||
"total_tokens": 13278,
|
||||
"ttft_sec": 0.73,
|
||||
"filename": "outputs/qwen3.6-35b-a3b-4bit-mlx.py",
|
||||
"tests_pass": false,
|
||||
"total_score": 57,
|
||||
"breakdown": {
|
||||
"complexity": 15,
|
||||
"concurrency": 15,
|
||||
"isolation": 13,
|
||||
"memory_edge_cases": 10,
|
||||
"test_integrity": 4
|
||||
},
|
||||
"verdict": "Critical Bugs",
|
||||
"best_for": "Not recommended for systems code as-is. The 4-bit quant degrades logic sharply vs the 6-bit sibling (82->57) — usable only for boilerplate/scaffolding drafts that a human will heavily rewrite.",
|
||||
"critical_bugs": [
|
||||
"FATAL: _evict() double-removes nodes — pop() already calls remove() (nulling node.prev/next), then _remove_node() calls remove() AGAIN -> AttributeError: 'NoneType' on first eviction. The cache cannot survive reaching capacity.",
|
||||
"Test suite never executes: test_lfu_eviction crashes at the first eviction, so the 'All tests passed' message is unreachable and assertions are effectively unverified.",
|
||||
"Background _eviction_loop materializes list(self.key_to_node.keys())[:50] every sweep — an O(N) linear scan, forbidden by the strict O(1) requirement.",
|
||||
"_FreqList.pop() has no empty-guard — calling pop() on an empty list dereferences self.head.next (the dummy tail) and corrupts the DLL.",
|
||||
"Transaction _apply_put applies the buffered value into the EXACT original_node captured at tx.put() time; if the global cache evicted/relocated that node between put and commit, you mutate a stale/dangling node (no MVCC/version check).",
|
||||
"Commit is not atomic across exceptions: a crash mid-_apply loop leaves half-applied global state with no rollback.",
|
||||
"Uses time.time() (system clock) throughout instead of time.monotonic() — NTP jumps corrupt TTL eviction."
|
||||
],
|
||||
"patch_code": "# FIX 1 (the crash): _evict double-removes. pop() already unlinks,\n# so do NOT call _remove_node on a popped node. Either:\n# (a) pop and then only delete the key_map entry + min_freq bookkeeping:\ndef _evict(self):\n if not self.freq_to_list:\n return\n evict_list = self.freq_to_list[self.min_freq]\n if evict_list.size == 0: # guard against empty\n del self.freq_to_list[self.min_freq]\n return\n node = evict_list.pop() # pop() unlinks + nulls prev/next\n del self.key_to_node[node.key] # DON'T call _remove_node again\n if self.freq_to_list[self.min_freq].size == 0:\n del self.freq_to_list[self.min_freq]\n self.min_freq += 1\n\n# FIX 2: _FreqList.pop empty-guard\ndef pop(self) -> _Node:\n if self.size == 0:\n raise IndexError('pop from empty _FreqList')\n node = self.head.next\n self.remove(node)\n return node\n\n# FIX 3: kill the O(N) scan in background sweep — maintain a separate\n# set of keys that have a TTL, and iterate that set in batches:\nasync with self.lock:\n batch = list(self._ttl_keys)[:50]\n for k in batch:\n node = self.key_to_node.get(k)\n if node and 0 < node.expires_at <= time.monotonic():\n self._remove_node(node)\n\n# FIX 4: time.time() -> time.monotonic() everywhere.\n# FIX 5: add node.version; in tx._apply_put, abort/refresh if\n# cache.key_to_node[key] is a different node than original_node."
|
||||
},
|
||||
{
|
||||
"id": "qwen3.6-35b-a3b-uncensored-hauhaucs-aggressive-gguf",
|
||||
"timestamp": "2026-07-28T13:10:00Z",
|
||||
"model_name": "Qwen 3.6 35B-A3B (uncensored hauhaucs aggressive)",
|
||||
"quant": "GGUF",
|
||||
"format": "gguf",
|
||||
"tok_sec": 62.54,
|
||||
"total_tokens": 13897,
|
||||
"ttft_sec": 1.09,
|
||||
"filename": "outputs/qwen3.6-35b-a3b-uncensored-hauhaucs-aggressive-gguf.py",
|
||||
"tests_pass": false,
|
||||
"total_score": 49,
|
||||
"breakdown": {
|
||||
"complexity": 12,
|
||||
"concurrency": 10,
|
||||
"isolation": 12,
|
||||
"memory_edge_cases": 11,
|
||||
"test_integrity": 4
|
||||
},
|
||||
"verdict": "Critical Bugs",
|
||||
"best_for": "Not recommended for production code. Reasonable API shape and correctly used time.monotonic(), but the module does not parse (syntax error), contains an infinite while:pass loop, and has data races. Avoid for systems/concurrency work.",
|
||||
"critical_bugs": [
|
||||
"SyntaxError: line 305 'assert val := await cache.get(...)' is invalid Python — walrus operator cannot appear in an assert statement. The ENTIRE module fails to compile, so nothing runs and no test can execute.",
|
||||
"Infinite busy-loop: _evict_lfu lines 159-160 — 'while self.min_freq in self.freq_map and self.min_freq < max(...): pass' has an empty body that never updates min_freq, recomputes max() (O(F)) each iteration, and can never terminate.",
|
||||
"Hidden O(F) scan: min(self.freq_map.keys()) / max(self.freq_map.keys()) appears at 6 call sites (every eviction and removal) — violates the strict O(1) requirement.",
|
||||
"Race condition: get() and put() perform lazy-TTL _remove_key() BEFORE acquiring the lock (lines 71-73, 107-108), mutating shared state unlocked while other coroutines read/write.",
|
||||
"Deadlock risk: background_loop holds self._lock, then calls await self._remove_key() which is itself a lock-acquiring method — asyncio.Lock is NOT reentrant -> deadlock when the evictor runs.",
|
||||
"No __slots__ on _Node despite using a dataclass (rubric required it for memory efficiency).",
|
||||
"_remove_key will KeyError on self.freq_map[freq] if a concurrent operation already deleted that bucket."
|
||||
],
|
||||
"patch_code": "# FIX 1 (the parse error): assign first, then assert.\nval_d = await cache.get(\"D\")\nassert val_d, \"D should exist\"\nval_a = await cache.get(\"A\")\nassert val_a, \"A should exist (highest freq)\"\n\n# FIX 2: delete the broken while:pass loop. Bump min_freq incrementally:\n# only when the min_freq bucket empties, and only ever UP by 1 (a key\n# whose freq increased must land at min_freq+1). Never call min()/max().\nif not old_bucket:\n del self.freq_map[old_freq]\n if self.min_freq == old_freq:\n self.min_freq += 1 # next tier up; never scan\n\n# FIX 3: do ALL lazy eviction INSIDE the lock, not before it:\nasync def get(self, key):\n async with self._lock:\n if self.ttl_map.get(key, inf) <= time.monotonic():\n await self._remove_key(key) # now locked\n return None\n ...\n\n# FIX 4: make _remove_key a non-locking private helper, called from\n# inside already-locked public methods, so background_loop doesn't try\n# to re-acquire the non-reentrant asyncio.Lock.\n\n# FIX 5: add __slots__ = ('key','value','ttl_expiry') to _Node."
|
||||
},
|
||||
{
|
||||
"id": "kat-coder-v2.5-dev-xl-mlx",
|
||||
"timestamp": "2026-07-28T13:15:00Z",
|
||||
"model_name": "KAT-Coder v2.5 Dev XL",
|
||||
"quant": "MLX",
|
||||
"format": "mlx",
|
||||
"tok_sec": 65.26,
|
||||
"total_tokens": 6172,
|
||||
"ttft_sec": 7.42,
|
||||
"filename": "outputs/kat-coder-v2.5-dev-xl-mlx.py",
|
||||
"tests_pass": false,
|
||||
"total_score": 65,
|
||||
"breakdown": {
|
||||
"complexity": 15,
|
||||
"concurrency": 16,
|
||||
"isolation": 16,
|
||||
"memory_edge_cases": 13,
|
||||
"test_integrity": 5
|
||||
},
|
||||
"verdict": "Critical Bugs",
|
||||
"best_for": "Promising code-design instincts (cleanest abstractions and best transaction isolation design in the set) but undone by a single fatal one-line bug that stops it running. With the bug fixed it would likely score 80+; as-is, only useful as a structural reference.",
|
||||
"critical_bugs": [
|
||||
"FATAL: _put_internal line 305 inserts a NEW key with 'self._freq_map[1].push_front(...)' but never ensures the freq-1 bucket exists — KeyError: 1 on the very first put. The cache cannot store a single key. The _ensure_freq_list(1) helper it should use exists and is used correctly everywhere else (lines 294, 354).",
|
||||
"Transaction.commit() calls _put_internal for buffered writes, so it hits the same KeyError: 1 — committed transactions crash too.",
|
||||
"Test suite cannot execute: crashes at the first cache.put() in main(); the well-built test harness (pass/fail counter, 4 real scenarios) validates nothing.",
|
||||
"No __slots__ on _DLLNode/_CacheNode/_DoublyLinkedList despite the rubric requiring it for memory efficiency.",
|
||||
"_evict_node uses min(self._freq_map) (line 325) when the min-tier empties — a hidden O(F) scan, violating strict O(1).",
|
||||
"No MVCC/version check on transaction commit (lost-update possible if the global key is modified between tx.get and commit); commit is not exception-safe across the writes-vs-deletes loops."
|
||||
],
|
||||
"patch_code": "# FIX 1 (the fatal one-liner): use the helper that already exists.\n# line 305, in _put_internal, new-key branch:\n- self._freq_map[1].push_front(dll_node)\n+ self._ensure_freq_list(1).push_front(dll_node)\n# (This single change makes the cache and transactions functional.)\n\n# FIX 2: replace the O(F) min() scan with an incremental bump:\n# in _evict_node, when the min-tier bucket empties, min_freq is the\n# lowest remaining tier. Since freq only ever increments by 1, the\n# next min is almost always min_freq+1; track it incrementally rather\n# than scanning. Or, since this only happens on full eviction, accept\n# O(F) but only on the empty-cache edge — document it.\n\n# FIX 3: add __slots__ to all internal classes:\nclass _CacheNode:\n __slots__ = ('key','value','ttl_seconds','expiry_time','freq','dll_node')\n ...\n\n# FIX 4: wrap commit applies in try/except so a mid-commit exception\n# does not leave a half-applied global state; consider abort semantics.\n# FIX 5: add node.version; in tx commit, abort if the global node for\n# a key is not the one seen at tx.get() time."
|
||||
},
|
||||
{
|
||||
"id": "gemma4-31b-gguf",
|
||||
"timestamp": "2026-07-28T13:20:00Z",
|
||||
"model_name": "Gemma 4 31B",
|
||||
"quant": "GGUF",
|
||||
"format": "gguf",
|
||||
"tok_sec": 10.09,
|
||||
"total_tokens": 4536,
|
||||
"ttft_sec": 4.39,
|
||||
"speed_caveat": "All Gemma 4 models ran abnormally slow (GPU offload appeared inactive despite being set), so tok/sec and TTFT are NOT representative of the model itself — likely an LM Studio/GGUF config issue. Treat speed numbers for the Gemma 4 batch as suspect.",
|
||||
"filename": "outputs/gemma4-31b-gguf.py",
|
||||
"tests_pass": true,
|
||||
"total_score": 78,
|
||||
"breakdown": {
|
||||
"complexity": 17,
|
||||
"concurrency": 16,
|
||||
"isolation": 14,
|
||||
"memory_edge_cases": 15,
|
||||
"test_integrity": 16
|
||||
},
|
||||
"verdict": "Minor Logic Flaws",
|
||||
"best_for": "Clean, correct, runnable code with solid O(1) structure and good concurrency granularity. A reliable pick for everyday caching/async work after a monotonic-clock + __slots__ pass.",
|
||||
"critical_bugs": [
|
||||
"Isolation leak: Transaction.get falls back to the PUBLIC cache.get, which calls _update_frequency — so reading a key inside a transaction mutates GLOBAL frequency state before commit, leaking uncommitted access patterns into global eviction order. Spec requires tx reads not to alter global freq.",
|
||||
"Uses time.time() (system clock) throughout instead of time.monotonic() — NTP adjustments corrupt TTL eviction.",
|
||||
"No __slots__ on Node/DoublyLinkedList/LFUCache/Transaction — rubric required it for memory efficiency.",
|
||||
"_delete_internal deliberately leaves empty frequency buckets in freq_map (documented but a minor memory leak: empty DoublyLinkedList objects accumulate).",
|
||||
"Background evictor does list(self.cache.keys()) = O(N) snapshot every interval — a linear scan, forbidden by strict O(1).",
|
||||
"No MVCC/version check on commit (lost-update possible); commit is not exception-safe across the deletes-vs-puts loops.",
|
||||
"Tests pass but don't probe mid-commit read isolation or eviction-under-real-contention (capacity sized so all keys fit), so the isolation leak above goes undetected."
|
||||
],
|
||||
"patch_code": "# FIX 1 (the isolation leak): tx reads must NOT mutate global freq.\n# Add a read-only global lookup (no _update_frequency) and use it in tx.get:\nasync def _read_raw(self, key): # no freq bump\n node = self.cache.get(key)\n if node is None: return None\n if time.monotonic() > node.expiry:\n await self._delete_internal(key)\n return None\n return node.value\n# then in Transaction.get fallback:\n return await self._cache._read_raw(key) # NOT cache.get\n\n# FIX 2: time.time() -> time.monotonic() everywhere (get/put/_put_internal/bg loop).\n# FIX 3: add __slots__ to Node, DoublyLinkedList, LFUCache, Transaction.\n# FIX 4: prune empty freq buckets on delete, or have _evict_lfu drop them.\n# FIX 5: iterate a dedicated _ttl_keys set (batched) in the bg evictor\n# instead of list(self.cache.keys()) to stay O(batch), not O(N)."
|
||||
},
|
||||
{
|
||||
"id": "gemma-4-31b-qat-gguf",
|
||||
"timestamp": "2026-07-28T13:25:00Z",
|
||||
"model_name": "Gemma 4 31B QAT",
|
||||
"quant": "QAT GGUF",
|
||||
"format": "gguf",
|
||||
"tok_sec": 15.0,
|
||||
"total_tokens": 4552,
|
||||
"ttft_sec": 4.01,
|
||||
"speed_caveat": "Same as the Gemma 4 batch: GPU offload appeared inactive so tok/sec/TTFT are NOT representative of the model. Suspected LM Studio/GGUF config issue.",
|
||||
"filename": "outputs/gemma-4-31b-qat-gguf.py",
|
||||
"tests_pass": true,
|
||||
"total_score": 70,
|
||||
"breakdown": {
|
||||
"complexity": 16,
|
||||
"concurrency": 16,
|
||||
"isolation": 11,
|
||||
"memory_edge_cases": 13,
|
||||
"test_integrity": 14
|
||||
},
|
||||
"verdict": "Critical Bugs",
|
||||
"best_for": "Runnable and structurally sound, but the LFU eviction has a stale-min_freq capacity-breach path and the transaction API doesn't match the spec (no commit/rollback). Usable for prototypes if you fix eviction and re-skin transactions.",
|
||||
"critical_bugs": [
|
||||
"Capacity breach: eviction does self.freq_map[self.min_freq].pop_tail() with NO guard that the bucket exists or is non-empty, and never prunes emptied freq buckets. After manual deletes empty the min-tier, pop_tail returns None silently -> eviction fails -> cache grows PAST capacity. This is the exact stale-min_freq capacity-breach the rubric flags.",
|
||||
"Non-conformant transaction API: Transaction has NO commit() or rollback() method (both required by spec). Commit happens via a separate cache.apply_transaction_changes(tx._state) — wrong surface; the test only passes because it uses this internal path.",
|
||||
"Isolation leak: Transaction.get falls back to the public cache.get, which bumps global frequency before commit — uncommitted tx reads alter global eviction order.",
|
||||
"Duplicated eviction logic in apply_transaction_changes re-introduces the stale-min_freq bug at line 211.",
|
||||
"Uses time.time() (system clock) via _get_now() instead of time.monotonic() — NTP jumps corrupt TTL (though _get_now is a clean single fix point).",
|
||||
"No __slots__ on Node/DoublyLinkedList/LFUCache/TransactionState/Transaction.",
|
||||
"Background sweep does list(self.cache.keys()) = O(N) per interval.",
|
||||
"Tests pass but use the non-spec commit path and don't probe capacity breach or isolation leak."
|
||||
],
|
||||
"patch_code": "# FIX 1 (capacity breach): guard + prune empty buckets on eviction:\nwhile self.min_freq in self.freq_map and self.freq_map[self.min_freq].size == 0:\n del self.freq_map[self.min_freq]\n self.min_freq += 1\n if not self.freq_map:\n break\nif self.min_freq not in self.freq_map:\n return # nothing to evict\nevicted = self.freq_map[self.min_freq].pop_tail()\nif evicted and self.freq_map[self.min_freq].size == 0:\n del self.freq_map[self.min_freq]\n\n# FIX 2 (conformant API): add commit/rollback to Transaction:\nasync def commit(self):\n await self._cache.apply_transaction_changes(self._state)\n self._committed = True\ndef rollback(self):\n self._state.writes.clear()\n self._committed = True\n\n# FIX 3 (isolation leak): tx.get should use a read-only global lookup\n# (no _update_freq), not the public cache.get.\n# FIX 4: _get_now returns time.monotonic().\n# FIX 5: add __slots__ to all node/list classes."
|
||||
},
|
||||
{
|
||||
"id": "gemma-4-12b-coder-heretic-mxfp8-mlx",
|
||||
"timestamp": "2026-07-28T13:30:00Z",
|
||||
"model_name": "Gemma 4 12B Coder (fable5-composer2.5-v1-uncensored-heretic merge)",
|
||||
"quant": "mxfp8 MLX",
|
||||
"format": "mlx",
|
||||
"tok_sec": 25.28,
|
||||
"total_tokens": 2625,
|
||||
"ttft_sec": 3.21,
|
||||
"filename": "outputs/gemma-4-12b-coder-fable5-composer2.5-v1-uncensored-heretic-mxfp8-mlx.py",
|
||||
"tests_pass": false,
|
||||
"total_score": 43,
|
||||
"breakdown": {
|
||||
"complexity": 13,
|
||||
"concurrency": 7,
|
||||
"isolation": 10,
|
||||
"memory_edge_cases": 9,
|
||||
"test_integrity": 4
|
||||
},
|
||||
"verdict": "Critical Bugs",
|
||||
"best_for": "Not usable as-is — the cache cannot store its first key and the background evictor would crash the event loop. The smallest model in the set (12B) and lowest-quality output. Avoid for systems work.",
|
||||
"critical_bugs": [
|
||||
"FATAL: put() line 112 does 'bucket = self.freq_buckets[self.min_freq]' after setting min_freq=1 but NEVER creates freq_buckets[1] -> KeyError: 1 on the very first put. Cache is unusable.",
|
||||
"Background evictor is fundamentally broken: start_evictor defines a SYNC 'def evict_loop' and passes it to create_task; inside it calls blocking time.sleep(interval) (freezes the event loop) AND asyncio.run(...) from within a running loop -> RuntimeError. Would crash hard if ever reached.",
|
||||
"Eviction-by-re-put: expired keys are 'evicted' by re-inserting them with TTL 0 (line 122) instead of deleting them — wrong semantics and triggers immediate re-eviction.",
|
||||
"Transaction duplicates the entire LFU machinery (local_cache + local_freq + local_min_freq) for snapshot isolation, but _update_local_freq has the same missing-bucket KeyError (line 140).",
|
||||
"Class name typo 'DoublyLinkedListList' (doubled word).",
|
||||
"No __slots__; time.time() (not monotonic) throughout.",
|
||||
"Tests cannot run — crash at first put."
|
||||
],
|
||||
"patch_code": "# FIX 1 (the fatal KeyError): create the bucket before use.\n# In put(), new-key branch:\n- bucket = self.freq_buckets[self.min_freq]\n+ bucket = self.freq_buckets.setdefault(self.min_freq, DoublyLinkedListList())\n# Same fix in _update_freq and Transaction._update_local_freq (use setdefault).\n\n# FIX 2 (the broken evictor): make it a real async task that deletes:\nasync def _evict_loop(self, interval):\n while True:\n await asyncio.sleep(interval) # async, non-blocking\n now = time.monotonic()\n async with self.global_lock:\n expired = [k for k, n in list(self.cache.items()) if now > n.ttl_expiry]\n for k in expired:\n node = self.cache.pop(k, None)\n if node:\n self.freq_buckets[node.freq].remove(node) # DELETE, not re-put\n\nasync def start_evictor(self, interval=1.0):\n self.evictor_task = asyncio.create_task(self._evict_loop(interval))\n\n# FIX 3: delete expired keys; do NOT re-insert with TTL 0.\n# FIX 4: time.time() -> time.monotonic().\n# FIX 5: add __slots__ to Node / DoublyLinkedListList / ConcurrentLFUCache / Transaction."
|
||||
},
|
||||
{
|
||||
"id": "deepseekv4flash-cloud",
|
||||
"timestamp": "2026-07-28T13:35:00Z",
|
||||
"model_name": "DeepSeek V4 Flash (CLOUD baseline)",
|
||||
"quant": "n/a (cloud)",
|
||||
"format": "cloud",
|
||||
"tok_sec": null,
|
||||
"total_tokens": null,
|
||||
"ttft_sec": null,
|
||||
"speed_caveat": "Cloud model (run via opencode, not LM Studio) — tok_sec/tokens/TTFT are N/A (not measured for cloud). Included as a quality baseline against the local models. NOTE: it took 3 attempts to produce any output and ~12 minutes of thinking before succeeding — so it is a QUALITY benchmark, not a speed/usability one.",
|
||||
"filename": "deepseekv4flash.py",
|
||||
"tests_pass": true,
|
||||
"total_score": 91,
|
||||
"breakdown": {
|
||||
"complexity": 18,
|
||||
"concurrency": 18,
|
||||
"isolation": 19,
|
||||
"memory_edge_cases": 18,
|
||||
"test_integrity": 18
|
||||
},
|
||||
"verdict": "Production Ready",
|
||||
"best_for": "Reference-quality baseline (91/100) — the bar the local models are measured against. Only submission with __slots__ + time.monotonic() + delta-based transactional frequency accounting. Sets the ceiling for correctness, though its unreliability (3 attempts, 12-min think time) makes it a poor *local* daily-driver.",
|
||||
"critical_bugs": [
|
||||
"Two min(self._freq_to_list) linear scans in _evict_one's defensive recovery path (lines 415, 429) — only triggered when min_freq desyncs, not per-operation, but still a non-O(1) path. Could be replaced with incremental tracking.",
|
||||
"Single coarse lock held across the whole commit-apply loop — not the fine-grained locking the prompt asked for.",
|
||||
"__slots__ present on _Node and _DLL but not extended to Transaction / LFUCache.",
|
||||
"No explicit lost-update/conflict abort on commit (delta-based freq is applied unconditionally).",
|
||||
"Tests pass 20/20 but don't include a mid-commit read-isolation probe or adversarial eviction-under-contention stress."
|
||||
],
|
||||
"patch_code": "# These are minor refinements on an already production-ready file.\n\n# FIX 1: eliminate the recovery min() scans by keeping min_freq\n# strictly in sync on every insert/bump/remove (it already does on\n# the hot path), so the _evict_one recovery branch is unreachable and\n# can assert instead of scanning:\nassert self._min_freq in self._freq_to_list or not self._freq_to_list\n\n# FIX 2: extend __slots__ to Transaction and LFUCache.\nclass LFUCache(Generic[KT, VT]):\n __slots__ = ('_capacity','_key_to_node','_freq_to_list','_min_freq',\n '_lock','_ttl_index','_evictor_task','_closed')\n\n# FIX 3 (optional): on commit, if a key's global node changed since the\n# tx snapshot, raise LFUCacheError('lost update') instead of overwriting."
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,600 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
In-Memory Concurrent LFU Cache with Async TTL Eviction & Atomic Transactions.
|
||||
|
||||
Pure Python 3.11+, no external dependencies.
|
||||
|
||||
O(1) average-time get/put via frequency buckets with doubly-linked lists.
|
||||
Dual-layer TTL: lazy on access + background async sweep.
|
||||
Atomic transactions with read-your-own-writes, commit, and rollback.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from typing import (Dict, Generic, Hashable, Optional, Set, TypeVar)
|
||||
|
||||
|
||||
KT = TypeVar('KT', bound=Hashable)
|
||||
VT = TypeVar('VT')
|
||||
|
||||
_SENTINEL = object()
|
||||
|
||||
|
||||
class LFUCacheError(Exception):
|
||||
"""Base exception for LFU cache errors."""
|
||||
|
||||
|
||||
class _Node(Generic[KT, VT]):
|
||||
"""Doubly-linked list node holding key, value, frequency, and expiry."""
|
||||
|
||||
__slots__ = ('key', 'value', 'freq', 'expires_at', 'prev', 'next')
|
||||
|
||||
def __init__(self, key: KT, value: VT) -> None:
|
||||
self.key = key
|
||||
self.value = value
|
||||
self.freq: int = 0
|
||||
self.expires_at: Optional[float] = None
|
||||
self.prev: Optional['_Node[KT, VT]'] = None
|
||||
self.next: Optional['_Node[KT, VT]'] = None
|
||||
|
||||
|
||||
class _DLL(Generic[KT, VT]):
|
||||
"""
|
||||
Doubly-linked list with sentinel head/tail.
|
||||
O(1) append (tail/MRU), remove (by reference), and pop_left (head/LRU).
|
||||
"""
|
||||
|
||||
__slots__ = ('_head', '_tail', 'size')
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._head = _Node[KT, VT](_SENTINEL, None) # type: ignore[arg-type]
|
||||
self._tail = _Node[KT, VT](_SENTINEL, None) # type: ignore[arg-type]
|
||||
self._head.next = self._tail
|
||||
self._tail.prev = self._head
|
||||
self.size: int = 0
|
||||
|
||||
def append(self, node: _Node[KT, VT]) -> None:
|
||||
"""Add node to tail (most-recently-used position). O(1)."""
|
||||
last = self._tail.prev
|
||||
last.next = node
|
||||
node.prev = last
|
||||
node.next = self._tail
|
||||
self._tail.prev = node
|
||||
self.size += 1
|
||||
|
||||
def remove(self, node: _Node[KT, VT]) -> None:
|
||||
"""Remove node that must be in this list. O(1)."""
|
||||
node.prev.next = node.next
|
||||
node.next.prev = node.prev
|
||||
node.prev = None
|
||||
node.next = None
|
||||
self.size -= 1
|
||||
|
||||
def pop_left(self) -> Optional[_Node[KT, VT]]:
|
||||
"""Remove and return the head node (LRU position). O(1)."""
|
||||
if self.size == 0:
|
||||
return None
|
||||
node = self._head.next
|
||||
self.remove(node)
|
||||
return node
|
||||
|
||||
|
||||
class Transaction(Generic[KT, VT]):
|
||||
"""
|
||||
Atomic transaction with read-your-own-writes isolation.
|
||||
|
||||
Uncommitted writes are invisible to global cache readers.
|
||||
All local state is discarded on rollback without touching global structures.
|
||||
"""
|
||||
|
||||
def __init__(self, cache: 'LFUCache[KT, VT]') -> None:
|
||||
self._cache = cache
|
||||
self._writes: Dict[KT, VT] = {}
|
||||
self._deletes: Set[KT] = set()
|
||||
self._snapshot: Dict[KT, VT] = {}
|
||||
self._freq_deltas: Dict[KT, int] = {}
|
||||
self._ttls: Dict[KT, Optional[float]] = {}
|
||||
self._committed: bool = False
|
||||
self._rolled_back: bool = False
|
||||
|
||||
def _check_active(self) -> None:
|
||||
if self._committed:
|
||||
raise LFUCacheError("Transaction already committed")
|
||||
if self._rolled_back:
|
||||
raise LFUCacheError("Transaction already rolled back")
|
||||
|
||||
async def get(self, key: KT) -> Optional[VT]:
|
||||
"""Read key with read-your-own-writes and snapshot isolation."""
|
||||
self._check_active()
|
||||
|
||||
if key in self._deletes:
|
||||
return None
|
||||
if key in self._writes:
|
||||
self._freq_deltas[key] = self._freq_deltas.get(key, 0) + 1
|
||||
return self._writes[key]
|
||||
if key in self._snapshot:
|
||||
self._freq_deltas[key] = self._freq_deltas.get(key, 0) + 1
|
||||
return self._snapshot[key]
|
||||
|
||||
async with self._cache._lock:
|
||||
node = self._cache._key_to_node.get(key)
|
||||
if node is None:
|
||||
return None
|
||||
if node.expires_at is not None and node.expires_at <= time.monotonic():
|
||||
self._cache._remove_key(key)
|
||||
return None
|
||||
self._snapshot[key] = node.value
|
||||
|
||||
self._freq_deltas[key] = self._freq_deltas.get(key, 0) + 1
|
||||
return self._snapshot[key]
|
||||
|
||||
async def put(self, key: KT, value: VT,
|
||||
ttl_seconds: Optional[float] = None) -> None:
|
||||
"""Write key within transaction."""
|
||||
self._check_active()
|
||||
self._writes[key] = value
|
||||
self._deletes.discard(key)
|
||||
if ttl_seconds is not None:
|
||||
self._ttls[key] = ttl_seconds if ttl_seconds > 0 else None
|
||||
self._freq_deltas[key] = self._freq_deltas.get(key, 0) + 1
|
||||
|
||||
async def delete(self, key: KT) -> None:
|
||||
"""Mark key for deletion on commit."""
|
||||
self._check_active()
|
||||
self._deletes.add(key)
|
||||
self._writes.pop(key, None)
|
||||
self._snapshot.pop(key, None)
|
||||
self._freq_deltas.pop(key, None)
|
||||
self._ttls.pop(key, None)
|
||||
|
||||
async def commit(self) -> None:
|
||||
"""Atomically apply all pending changes to the global cache."""
|
||||
if self._rolled_back:
|
||||
raise LFUCacheError("Transaction already rolled back")
|
||||
if self._committed:
|
||||
return
|
||||
|
||||
async with self._cache._lock:
|
||||
try:
|
||||
for key in self._deletes:
|
||||
self._cache._remove_key(key)
|
||||
|
||||
for key, value in self._writes.items():
|
||||
node = self._cache._key_to_node.get(key)
|
||||
ttl = self._ttls.get(key)
|
||||
delta = self._freq_deltas.get(key, 1)
|
||||
|
||||
if node is not None:
|
||||
node.value = value
|
||||
if ttl is not None:
|
||||
node.expires_at = (
|
||||
time.monotonic() + ttl if ttl is not None else None
|
||||
)
|
||||
self._cache._change_freq(node, delta)
|
||||
else:
|
||||
while len(self._cache._key_to_node) >= self._cache._capacity:
|
||||
self._cache._evict_one()
|
||||
node = _Node(key, value)
|
||||
node.freq = delta
|
||||
if ttl is not None:
|
||||
node.expires_at = time.monotonic() + ttl
|
||||
self._cache._key_to_node[key] = node
|
||||
self._cache._add_to_freq_list(node)
|
||||
|
||||
for key, delta in self._freq_deltas.items():
|
||||
if key not in self._writes and key not in self._deletes:
|
||||
node = self._cache._key_to_node.get(key)
|
||||
if node is not None:
|
||||
self._cache._change_freq(node, delta)
|
||||
|
||||
while len(self._cache._key_to_node) > self._cache._capacity:
|
||||
self._cache._evict_one()
|
||||
|
||||
self._committed = True
|
||||
except BaseException:
|
||||
self._rolled_back = True
|
||||
raise
|
||||
|
||||
async def rollback(self) -> None:
|
||||
"""Discard all pending changes. Global cache is untouched."""
|
||||
if self._committed:
|
||||
raise LFUCacheError("Transaction already committed")
|
||||
self._rolled_back = True
|
||||
|
||||
|
||||
class LFUCache(Generic[KT, VT]):
|
||||
"""
|
||||
In-Memory Concurrent LFU Cache with Async TTL Eviction.
|
||||
|
||||
O(1) average-time get/put using frequency buckets with doubly-linked lists.
|
||||
Dual-layer TTL eviction: lazy on access + background async sweep.
|
||||
"""
|
||||
|
||||
def __init__(self, capacity: int = 1000,
|
||||
evictor_interval: float = 1.0,
|
||||
evictor_batch_size: int = 10,
|
||||
evictor_scan_budget: int = 100) -> None:
|
||||
if capacity < 1:
|
||||
raise ValueError("Capacity must be >= 1")
|
||||
|
||||
self._capacity = capacity
|
||||
self._evictor_interval = evictor_interval
|
||||
self._evictor_batch_size = evictor_batch_size
|
||||
self._evictor_scan_budget = evictor_scan_budget
|
||||
|
||||
self._key_to_node: Dict[KT, _Node[KT, VT]] = {}
|
||||
self._freq_to_list: Dict[int, _DLL[KT, VT]] = {}
|
||||
self._min_freq: int = 0
|
||||
self._lock = asyncio.Lock()
|
||||
self._evictor_running: bool = False
|
||||
self._evictor_task: Optional[asyncio.Task] = None
|
||||
|
||||
# ---- Public API ---------------------------------------------------------
|
||||
|
||||
async def get(self, key: KT) -> Optional[VT]:
|
||||
"""Retrieve value by key. Returns None if missing or expired.
|
||||
|
||||
Accesses increment the key's frequency (LFU tracking).
|
||||
Expired keys are lazily evicted on access.
|
||||
"""
|
||||
async with self._lock:
|
||||
node = self._key_to_node.get(key)
|
||||
if node is None:
|
||||
return None
|
||||
if node.expires_at is not None and node.expires_at <= time.monotonic():
|
||||
self._remove_key(key)
|
||||
return None
|
||||
self._change_freq(node, 1)
|
||||
return node.value
|
||||
|
||||
async def put(self, key: KT, value: VT,
|
||||
ttl_seconds: Optional[float] = None) -> None:
|
||||
"""Insert or update a key-value pair.
|
||||
|
||||
If *ttl_seconds* is None (default) the entry never expires.
|
||||
If *ttl_seconds* is <= 0 it is treated as no expiration.
|
||||
If the cache is at capacity the least-frequently-used item is evicted
|
||||
(tie-broken by least-recently-used within the minimum frequency tier).
|
||||
"""
|
||||
async with self._lock:
|
||||
node = self._key_to_node.get(key)
|
||||
|
||||
if node is not None:
|
||||
if node.expires_at is not None and node.expires_at <= time.monotonic():
|
||||
self._remove_key(key)
|
||||
node = None
|
||||
else:
|
||||
node.value = value
|
||||
if ttl_seconds is not None:
|
||||
node.expires_at = (
|
||||
time.monotonic() + ttl_seconds if ttl_seconds > 0 else None
|
||||
)
|
||||
self._change_freq(node, 1)
|
||||
return
|
||||
|
||||
while len(self._key_to_node) >= self._capacity:
|
||||
self._evict_one()
|
||||
|
||||
node = _Node(key, value)
|
||||
node.freq = 1
|
||||
if ttl_seconds is not None and ttl_seconds > 0:
|
||||
node.expires_at = time.monotonic() + ttl_seconds
|
||||
self._key_to_node[key] = node
|
||||
self._add_to_freq_list(node)
|
||||
|
||||
async def delete(self, key: KT) -> bool:
|
||||
"""Remove *key* from the cache. Returns True if the key existed."""
|
||||
async with self._lock:
|
||||
return self._remove_key(key)
|
||||
|
||||
def begin_transaction(self) -> Transaction[KT, VT]:
|
||||
"""Open an atomic transaction for batched reads/writes."""
|
||||
return Transaction(self)
|
||||
|
||||
@property
|
||||
def capacity(self) -> int:
|
||||
return self._capacity
|
||||
|
||||
@property
|
||||
def size(self) -> int:
|
||||
return len(self._key_to_node)
|
||||
|
||||
# ---- Background TTL Evictor --------------------------------------------
|
||||
|
||||
def start_evictor(self) -> None:
|
||||
"""Launch the background TTL eviction loop as an asyncio task."""
|
||||
if self._evictor_running:
|
||||
return
|
||||
self._evictor_running = True
|
||||
self._evictor_task = asyncio.create_task(self._evictor_loop())
|
||||
|
||||
async def stop_evictor(self) -> None:
|
||||
"""Cancel and wait for the background eviction task to finish."""
|
||||
if not self._evictor_running:
|
||||
return
|
||||
self._evictor_running = False
|
||||
if self._evictor_task is not None:
|
||||
self._evictor_task.cancel()
|
||||
try:
|
||||
await self._evictor_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._evictor_task = None
|
||||
|
||||
async def _evictor_loop(self) -> None:
|
||||
"""
|
||||
Background loop: periodically scan a limited batch of keys and remove
|
||||
expired entries. Releases the lock between batches so concurrent
|
||||
reads/writes are not blocked for extended periods.
|
||||
"""
|
||||
cursor = 0
|
||||
keys: list = []
|
||||
|
||||
try:
|
||||
while self._evictor_running:
|
||||
await asyncio.sleep(self._evictor_interval)
|
||||
|
||||
async with self._lock:
|
||||
if cursor >= len(keys):
|
||||
keys = list(self._key_to_node.keys())
|
||||
cursor = 0
|
||||
if not keys:
|
||||
continue
|
||||
|
||||
now = time.monotonic()
|
||||
removed = 0
|
||||
scan = self._evictor_scan_budget
|
||||
|
||||
while scan > 0 and cursor < len(keys):
|
||||
key = keys[cursor]
|
||||
cursor += 1
|
||||
scan -= 1
|
||||
|
||||
node = self._key_to_node.get(key)
|
||||
if (node is not None and
|
||||
node.expires_at is not None and
|
||||
node.expires_at <= now):
|
||||
self._remove_key(key)
|
||||
removed += 1
|
||||
if removed >= self._evictor_batch_size:
|
||||
break
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
# ---- O(1) Internal Helpers ---------------------------------------------
|
||||
|
||||
def _add_to_freq_list(self, node: _Node[KT, VT]) -> None:
|
||||
"""Insert *node* into its frequency bucket and update *min_freq*."""
|
||||
freq = node.freq
|
||||
if freq not in self._freq_to_list:
|
||||
self._freq_to_list[freq] = _DLL()
|
||||
self._freq_to_list[freq].append(node)
|
||||
if self._min_freq not in self._freq_to_list or freq < self._min_freq:
|
||||
self._min_freq = freq
|
||||
|
||||
def _remove_from_freq_list(self, node: _Node[KT, VT]) -> None:
|
||||
"""Remove *node* from its frequency bucket. No-op if not linked."""
|
||||
if node.prev is None or node.next is None:
|
||||
return
|
||||
freq = node.freq
|
||||
lst = self._freq_to_list.get(freq)
|
||||
if lst is None:
|
||||
return
|
||||
lst.remove(node)
|
||||
if lst.size == 0:
|
||||
del self._freq_to_list[freq]
|
||||
|
||||
def _change_freq(self, node: _Node[KT, VT], delta: int) -> None:
|
||||
"""Atomically increase *node*'s frequency by *delta* and
|
||||
move it to the corresponding bucket. O(1)."""
|
||||
if delta <= 0:
|
||||
return
|
||||
self._remove_from_freq_list(node)
|
||||
node.freq += delta
|
||||
self._add_to_freq_list(node)
|
||||
|
||||
def _remove_key(self, key: KT) -> bool:
|
||||
"""Remove *key* from all internal structures. Returns True if existed."""
|
||||
node = self._key_to_node.pop(key, None)
|
||||
if node is None:
|
||||
return False
|
||||
self._remove_from_freq_list(node)
|
||||
return True
|
||||
|
||||
def _evict_one(self) -> Optional[KT]:
|
||||
"""
|
||||
Evict one item: LRU among the minimum-frequency bucket.
|
||||
Returns the evicted key, or None if the cache is empty.
|
||||
"""
|
||||
if not self._key_to_node:
|
||||
return None
|
||||
|
||||
if self._min_freq not in self._freq_to_list:
|
||||
if not self._freq_to_list:
|
||||
return None
|
||||
self._min_freq = min(self._freq_to_list)
|
||||
|
||||
lst = self._freq_to_list.get(self._min_freq)
|
||||
if lst is None or lst.size == 0:
|
||||
return None
|
||||
|
||||
node = lst.pop_left()
|
||||
if node is None:
|
||||
return None
|
||||
|
||||
del self._key_to_node[node.key]
|
||||
if lst.size == 0:
|
||||
del self._freq_to_list[self._min_freq]
|
||||
if self._freq_to_list:
|
||||
self._min_freq = min(self._freq_to_list)
|
||||
|
||||
return node.key
|
||||
|
||||
# ---- Async Context Manager ---------------------------------------------
|
||||
|
||||
async def __aenter__(self) -> 'LFUCache[KT, VT]':
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *exc_info) -> None:
|
||||
await self.stop_evictor()
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Unit Tests
|
||||
# ============================================================================
|
||||
|
||||
async def _run_tests() -> None:
|
||||
passed = 0
|
||||
total = 0
|
||||
|
||||
def check(cond: bool, msg: str):
|
||||
nonlocal passed, total
|
||||
total += 1
|
||||
if cond:
|
||||
passed += 1
|
||||
else:
|
||||
print(f" FAIL: {msg}")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 1. O(1) LFU eviction order
|
||||
# ------------------------------------------------------------------
|
||||
print("1. LFU eviction order ... ", end="", flush=True)
|
||||
|
||||
cache = LFUCache(capacity=3)
|
||||
|
||||
await cache.put("a", 1)
|
||||
await cache.put("b", 2)
|
||||
await cache.put("c", 3)
|
||||
|
||||
await cache.get("a")
|
||||
await cache.get("a")
|
||||
await cache.get("b")
|
||||
# freqs: a=3, b=2, c=1
|
||||
|
||||
await cache.put("d", 4) # evicts c (freq=1)
|
||||
check(await cache.get("c") is None, "c was evicted (freq=1)")
|
||||
check(await cache.get("d") == 4, "d is present")
|
||||
check(await cache.get("a") == 1, "a is present")
|
||||
check(await cache.get("b") == 2, "b is present")
|
||||
|
||||
# a=4, b=3, d=1 after the gets above
|
||||
await cache.put("e", 5) # evicts d (freq=1)
|
||||
check(await cache.get("d") is None, "d was evicted")
|
||||
check(await cache.get("e") == 5, "e is present")
|
||||
check(cache.size == 3, "cache size == 3")
|
||||
|
||||
print("OK")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 2. TTL eviction
|
||||
# ------------------------------------------------------------------
|
||||
print("2. TTL eviction ... ", end="", flush=True)
|
||||
|
||||
# 2a. Lazy eviction
|
||||
cache2 = LFUCache(capacity=10)
|
||||
await cache2.put("lazy", "alive", ttl_seconds=0.02)
|
||||
await asyncio.sleep(0.03)
|
||||
check(await cache2.get("lazy") is None, "lazy TTL eviction on get")
|
||||
|
||||
await cache2.put("lazy2", "alive", ttl_seconds=0.02)
|
||||
await asyncio.sleep(0.03)
|
||||
check(await cache2.get("lazy2") is None, "lazy TTL eviction on second get")
|
||||
|
||||
# 2b. Background eviction
|
||||
cache3 = LFUCache(capacity=10, evictor_interval=0.02, evictor_batch_size=5)
|
||||
cache3.start_evictor()
|
||||
await cache3.put("bg", "data", ttl_seconds=0.01)
|
||||
await asyncio.sleep(0.10)
|
||||
check(await cache3.get("bg") is None, "background evictor removes expired key")
|
||||
await cache3.stop_evictor()
|
||||
|
||||
print("OK")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 3. Transactions
|
||||
# ------------------------------------------------------------------
|
||||
print("3. Transactions ... ", end="", flush=True)
|
||||
|
||||
# 3a. Commit
|
||||
cache4 = LFUCache(capacity=10)
|
||||
await cache4.put("x", 10)
|
||||
|
||||
tx = cache4.begin_transaction()
|
||||
await tx.put("x", 20)
|
||||
check(await tx.get("x") == 20, "tx reads its own write")
|
||||
check(await cache4.get("x") == 10, "global does NOT see uncommitted write")
|
||||
await tx.commit()
|
||||
check(await cache4.get("x") == 20, "global sees committed value")
|
||||
|
||||
# 3b. Rollback
|
||||
await cache4.put("y", 100)
|
||||
tx2 = cache4.begin_transaction()
|
||||
await tx2.put("y", 200)
|
||||
await tx2.rollback()
|
||||
check(await cache4.get("y") == 100, "rollback discards writes")
|
||||
|
||||
# 3c. Delete in transaction
|
||||
await cache4.put("z", 300)
|
||||
tx3 = cache4.begin_transaction()
|
||||
await tx3.delete("z")
|
||||
check(await tx3.get("z") is None, "tx sees its own delete")
|
||||
check(await cache4.get("z") == 300, "global not affected before commit")
|
||||
await tx3.commit()
|
||||
check(await cache4.get("z") is None, "global sees committed delete")
|
||||
|
||||
# 3d. Rollback after delete
|
||||
await cache4.put("w", 400)
|
||||
tx4 = cache4.begin_transaction()
|
||||
await tx4.delete("w")
|
||||
await tx4.rollback()
|
||||
check(await cache4.get("w") == 400, "rollback restores deleted key")
|
||||
|
||||
# 3e. Read own writes after rollback raises
|
||||
tx5 = cache4.begin_transaction()
|
||||
await tx5.put("n", 1)
|
||||
await tx5.rollback()
|
||||
try:
|
||||
await tx5.get("n")
|
||||
check(False, "get on rolled-back tx should raise")
|
||||
except LFUCacheError:
|
||||
check(True, "rolled-back tx raises on get")
|
||||
|
||||
print("OK")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 4. Concurrent stress
|
||||
# ------------------------------------------------------------------
|
||||
print("4. Concurrent stress (50 workers) ... ", end="", flush=True)
|
||||
|
||||
cache5 = LFUCache(capacity=50, evictor_interval=0.05, evictor_batch_size=5)
|
||||
cache5.start_evictor()
|
||||
|
||||
async def worker(uid: int):
|
||||
for i in range(30):
|
||||
key = f"k_{(uid + i) % 40}"
|
||||
await cache5.put(key, uid * 1000 + i, ttl_seconds=0.5)
|
||||
_ = await cache5.get(key)
|
||||
await cache5.get(f"k_{(uid + i + 1) % 40}")
|
||||
if i % 5 == 0:
|
||||
await cache5.delete(f"k_{(uid + i + 2) % 40}")
|
||||
await asyncio.sleep(0)
|
||||
|
||||
workers = [worker(i) for i in range(50)]
|
||||
await asyncio.gather(*workers)
|
||||
await cache5.stop_evictor()
|
||||
|
||||
check(cache5.size <= 50, f"cache size {cache5.size} <= capacity 50")
|
||||
print("OK")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
print(f"\nResults: {passed}/{total} checks passed")
|
||||
if passed < total:
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
await _run_tests()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,403 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generator: builds dashboard.html + pages/<slug>..html from data/benchmark_history.json.
|
||||
Re-run after each grading batch to regenerate everything.
|
||||
Cyberpunk-terminal aesthetic. Pure stdlib + Chart.js via CDN.
|
||||
"""
|
||||
import json, html, os, sys
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
DATA = os.path.join(HERE, "data", "benchmark_history.json")
|
||||
OUT_DASH = os.path.join(HERE, "dashboard.html")
|
||||
PAGES_DIR = os.path.join(HERE, "pages")
|
||||
|
||||
PILLARS = ["complexity", "concurrency", "isolation", "memory_edge_cases", "test_integrity"]
|
||||
PILLAR_LABELS = {
|
||||
"complexity": "Complexity (O(1))",
|
||||
"concurrency": "Concurrency / Races",
|
||||
"isolation": "Tx Isolation",
|
||||
"memory_edge_cases": "Memory & Edges",
|
||||
"test_integrity": "Test Integrity",
|
||||
}
|
||||
|
||||
# ---- cyberpunk palette ----
|
||||
NEON_CYAN = "#00ffc8"
|
||||
NEON_MAG = "#ff2bd6"
|
||||
NEON_LIME = "#b6ff3a"
|
||||
NEON_AMBER= "#ffb000"
|
||||
NEON_RED = "#ff3b5c"
|
||||
BG = "#0a0a0f"
|
||||
PANEL = "#101018"
|
||||
INK = "#d7e0e6" # body text
|
||||
INK_DIM = "#7a8590" # muted
|
||||
|
||||
def verdict_meta(v):
|
||||
return {
|
||||
"Production Ready": (NEON_LIME, "PROD"),
|
||||
"Minor Logic Flaws": (NEON_AMBER, "FLAWS"),
|
||||
"Critical Bugs": (NEON_RED, "CRIT"),
|
||||
"Broken / Unusable": (NEON_RED, "DEAD"),
|
||||
}.get(v, (INK_DIM, "???"))
|
||||
|
||||
def esc(s):
|
||||
return html.escape(str(s)) if s is not None else ""
|
||||
|
||||
def speed_str(m):
|
||||
if m.get("tok_sec") is None:
|
||||
return "N/A"
|
||||
return f"{m['tok_sec']:.1f}"
|
||||
|
||||
# series colors for charts (cycles through neon accents)
|
||||
SERIES = [NEON_CYAN, NEON_MAG, NEON_LIME, NEON_AMBER, "#5b8cff", "#c084fc", "#34d399", "#f472b6"]
|
||||
|
||||
# shared <head> CSS — embedded so each page is standalone
|
||||
def shared_css():
|
||||
return """
|
||||
:root{
|
||||
--bg:#0a0a0f; --panel:#101018; --panel2:#14141f; --ink:#d7e0e6; --dim:#7a8590;
|
||||
--cyan:#00ffc8; --mag:#ff2bd6; --lime:#b6ff3a; --amber:#ffb000; --red:#ff3b5c; --blue:#5b8cff;
|
||||
--grid: rgba(0,255,200,0.04);
|
||||
}
|
||||
*{box-sizing:border-box}
|
||||
html,body{margin:0;padding:0}
|
||||
body{
|
||||
background:var(--bg); color:var(--ink);
|
||||
font-family:'Fira Sans',system-ui,-apple-system,sans-serif;
|
||||
line-height:1.6; min-height:100vh;
|
||||
/* scanline + grid texture overlay */
|
||||
background-image:
|
||||
repeating-linear-gradient(0deg, rgba(0,255,200,0.025) 0px, rgba(0,255,200,0.025) 1px, transparent 1px, transparent 3px),
|
||||
linear-gradient(var(--grid) 1px, transparent 1px),
|
||||
linear-gradient(90deg, var(--grid) 1px, transparent 1px),
|
||||
radial-gradient(1200px 600px at 70% -10%, rgba(91,139,255,0.10), transparent 60%),
|
||||
radial-gradient(900px 500px at 10% 110%, rgba(255,43,214,0.08), transparent 60%);
|
||||
background-size: 3px 3px, 44px 44px, 44px 44px, 100% 100%, 100% 100%;
|
||||
background-attachment: fixed;
|
||||
}
|
||||
mono,.mono,h1,h2,h3,.hud,th,.code,.badge,.stat-num,.lead-model,.verdict-chip{font-family:'Fira Code',ui-monospace,'SF Mono',Menlo,monospace}
|
||||
a{color:var(--cyan);text-decoration:none}
|
||||
a:hover{text-shadow:0 0 8px var(--cyan)}
|
||||
.wrap{max-width:1180px;margin:0 auto;padding:28px 20px 80px}
|
||||
header.hud-bar{
|
||||
border:1px solid rgba(0,255,200,0.25);
|
||||
background:linear-gradient(180deg,rgba(0,255,200,0.06),rgba(0,0,0,0));
|
||||
box-shadow:0 0 24px rgba(0,255,200,0.08), inset 0 0 30px rgba(0,0,0,0.5);
|
||||
border-radius:6px; padding:18px 22px; margin-bottom:24px;
|
||||
position:relative; overflow:hidden;
|
||||
}
|
||||
header.hud-bar::before{
|
||||
content:""; position:absolute; inset:0;
|
||||
background:repeating-linear-gradient(90deg, transparent 0 7px, rgba(0,255,200,0.03) 7px 8px);
|
||||
pointer-events:none;
|
||||
}
|
||||
.kicker{color:var(--cyan); letter-spacing:.32em; font-size:.72rem; text-transform:uppercase; text-shadow:0 0 10px rgba(0,255,200,0.5)}
|
||||
h1{font-size:1.5rem; margin:.3em 0 0; letter-spacing:.02em; text-shadow:0 0 16px rgba(0,255,200,0.35)}
|
||||
.subtitle{color:var(--dim); font-size:.9rem; margin-top:.35em}
|
||||
.stats{display:grid; grid-template-columns:repeat(4,1fr); gap:12px; margin:18px 0 26px}
|
||||
.stat{background:var(--panel); border:1px solid rgba(255,255,255,0.06); border-radius:6px; padding:14px 16px; position:relative; overflow:hidden}
|
||||
.stat::after{content:"";position:absolute;left:0;top:0;bottom:0;width:3px;background:var(--cyan);box-shadow:0 0 12px var(--cyan)}
|
||||
.stat .lbl{color:var(--dim);font-size:.68rem;letter-spacing:.18em;text-transform:uppercase}
|
||||
.stat .num{font-size:1.6rem;margin-top:4px;color:var(--ink)}
|
||||
.grid2{display:grid;grid-template-columns:1.1fr .9fr;gap:18px;margin-bottom:26px}
|
||||
.panel{background:var(--panel);border:1px solid rgba(255,255,255,0.07);border-radius:8px;padding:18px}
|
||||
.panel h2{font-size:.95rem;letter-spacing:.12em;text-transform:uppercase;color:var(--cyan);margin:0 0 14px;text-shadow:0 0 10px rgba(0,255,200,0.35)}
|
||||
.chart-box{position:relative;height:340px}
|
||||
@media(max-width:900px){.grid2{grid-template-columns:1fr}.stats{grid-template-columns:repeat(2,1fr)}}
|
||||
/* leaderboard */
|
||||
table{width:100%;border-collapse:collapse;font-size:.86rem}
|
||||
thead th{text-align:left;color:var(--dim);font-size:.66rem;letter-spacing:.16em;text-transform:uppercase;border-bottom:1px solid rgba(0,255,200,0.2);padding:8px 10px}
|
||||
tbody td{padding:11px 10px;border-bottom:1px solid rgba(255,255,255,0.05);vertical-align:middle}
|
||||
tbody tr{transition:background .15s, box-shadow .15s}
|
||||
tbody tr:hover{background:rgba(0,255,200,0.05);box-shadow:inset 0 0 0 1px rgba(0,255,200,0.25)}
|
||||
.rank{color:var(--dim);width:34px}
|
||||
.rank.top{color:var(--lime);text-shadow:0 0 8px var(--lime)}
|
||||
.model-name{color:var(--ink)}
|
||||
.quant{color:var(--dim);font-size:.78rem}
|
||||
.badge{display:inline-block;padding:2px 9px;border-radius:3px;font-size:.68rem;letter-spacing:.1em;border:1px solid currentColor}
|
||||
.btn{display:inline-block;padding:5px 12px;border:1px solid var(--cyan);color:var(--cyan);border-radius:4px;font-size:.74rem;letter-spacing:.08em;cursor:pointer;transition:all .18s}
|
||||
.btn:hover{background:rgba(0,255,200,0.12);box-shadow:0 0 14px rgba(0,255,200,0.4)}
|
||||
.bar-cell{display:flex;align-items:center;gap:10px}
|
||||
.score-bar{flex:1;height:8px;background:rgba(255,255,255,0.06);border-radius:2px;overflow:hidden;min-width:70px}
|
||||
.score-bar > i{display:block;height:100%;border-radius:2px;transition:width .4s, box-shadow .2s}
|
||||
tr:hover .score-bar > i{box-shadow:0 0 12px currentColor}
|
||||
.caveat{color:var(--amber);font-size:.72rem}
|
||||
.cloud-tag{color:var(--blue);font-size:.7rem;border:1px solid rgba(91,139,255,.4);padding:1px 6px;border-radius:3px;margin-left:6px}
|
||||
footer{color:var(--dim);font-size:.74rem;margin-top:40px;border-top:1px solid rgba(255,255,255,0.06);padding-top:14px;text-align:center}
|
||||
@media (prefers-reduced-motion: reduce){*{animation:none!important;transition:none!important}}
|
||||
"""
|
||||
|
||||
def head_html(title):
|
||||
return f"""<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<title>{esc(title)}</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Fira+Code:wght@400;500;600;700&family=Fira+Sans:wght@300;400;500;600;700&display=swap" rel="stylesheet">
|
||||
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.1/dist/chart.umd.min.js"></script>
|
||||
<style>{shared_css()}</style>
|
||||
</head>
|
||||
<body><div class="wrap">
|
||||
"""
|
||||
|
||||
FOOT = """</div>
|
||||
<script>if(window.matchMedia&&window.matchMedia('(prefers-reduced-motion: reduce)').matches&&window.Chart){Chart.defaults.animation=false;Chart.defaults.animations.colors=false;Chart.defaults.animations.numbers=false;}</script>
|
||||
</body></html>"""
|
||||
|
||||
def render_dashboard(data):
|
||||
models = sorted(data["models"], key=lambda m: -m["total_score"])
|
||||
n = len(models)
|
||||
avg = sum(m["total_score"] for m in models) / n if n else 0
|
||||
top = models[0] if models else None
|
||||
prod = sum(1 for m in models if m["verdict"] == "Production Ready")
|
||||
|
||||
# chart datasets (exclude cloud model from speed bars — has no tok/sec)
|
||||
local = [m for m in models if m.get("tok_sec") is not None]
|
||||
local_sorted = sorted(local, key=lambda m: -m["tok_sec"])
|
||||
|
||||
radar_models = [m for m in models if m.get("tok_sec") is not None or True][:3] # top 3 by score already
|
||||
|
||||
rows = []
|
||||
for i, m in enumerate(models, 1):
|
||||
col, chip = verdict_meta(m["verdict"])
|
||||
caveat = ""
|
||||
if m.get("speed_caveat"):
|
||||
caveat = '<div class="caveat">⚠ speed suspect</div>'
|
||||
cloud = '<span class="cloud-tag">CLOUD</span>' if m.get("format") == "cloud" else ""
|
||||
bar_color = col
|
||||
rows.append(f"""<tr>
|
||||
<td class="rank {'top' if i<=3 else ''}">#{i}</td>
|
||||
<td><div class="model-name">{esc(m['model_name'])}{cloud}</div><div class="quant">{esc(m['quant'])}</div>{caveat}</td>
|
||||
<td class="mono">{speed_str(m)} <span style="color:var(--dim);font-size:.72rem">t/s</span></td>
|
||||
<td><div class="bar-cell"><span class="mono" style="width:34px;color:{col}">{m['total_score']}</span>
|
||||
<span class="score-bar"><i style="width:{m['total_score']}%;background:{bar_color};color:{bar_color}"></i></span></div></td>
|
||||
<td><span class="badge verdict-chip" style="color:{col}">{chip}</span></td>
|
||||
<td style="color:var(--dim);font-size:.8rem">{esc(m['best_for'])[:70]}…</td>
|
||||
<td><a class="btn" href="pages/{esc(m['id'])}.html">DECODE ▸</a></td>
|
||||
</tr>""")
|
||||
|
||||
# JSON for charts
|
||||
bar_labels = json.dumps([m["model_name"].split("(")[0].strip()[:18] for m in local_sorted])
|
||||
bar_speed = json.dumps([m["tok_sec"] for m in local_sorted])
|
||||
bar_score = json.dumps([m["total_score"] for m in local_sorted])
|
||||
|
||||
radar_labels = json.dumps([PILLAR_LABELS[p] for p in PILLARS])
|
||||
radar_sets = []
|
||||
for idx, m in enumerate(radar_models):
|
||||
col = SERIES[idx % len(SERIES)]
|
||||
radar_sets.append({
|
||||
"label": m["model_name"][:24],
|
||||
"data": [m["breakdown"][p] for p in PILLARS],
|
||||
"borderColor": col,
|
||||
"backgroundColor": col + "22",
|
||||
})
|
||||
radar_json = json.dumps(radar_sets)
|
||||
|
||||
stats = f"""
|
||||
<div class="stat"><div class="lbl">Models Tested</div><div class="num mono">{n}</div></div>
|
||||
<div class="stat"><div class="lbl">Top Score</div><div class="num mono" style="color:var(--lime)">{top['total_score'] if top else '—'}</div></div>
|
||||
<div class="stat"><div class="lbl">Average</div><div class="num mono" style="color:var(--amber)">{avg:.1f}</div></div>
|
||||
<div class="stat"><div class="lbl">Prod-Ready</div><div class="num mono" style="color:var(--cyan)">{prod}/{n}</div></div>
|
||||
"""
|
||||
top_name = esc(top["model_name"]) if top else "—"
|
||||
|
||||
body = f"""
|
||||
{head_html("LLM Benchmark Suite")}
|
||||
<header class="hud-bar">
|
||||
<div class="kicker">▚ LOCAL LLM BENCHMARK SUITE // LFU CACHE & ACID AUDIT</div>
|
||||
<h1>// BENCHMARK_RESULTS <span style="color:var(--mag)">.json</span></h1>
|
||||
<div class="subtitle">{n} models graded on a strict 5-pillar / 100-pt rubric · O(1) LFU + ACID transactions · M3 Max · LM Studio
|
||||
· <span style="color:var(--cyan)">TOP: {top_name}</span></div>
|
||||
</header>
|
||||
{stats}
|
||||
<div class="grid2">
|
||||
<div class="panel">
|
||||
<h2>▮ Score vs Throughput (tok/sec)</h2>
|
||||
<div class="chart-box"><canvas id="bar"></canvas></div>
|
||||
<div style="color:var(--dim);font-size:.72rem;margin-top:8px">Local models only — cloud baseline (DeepSeek) excluded from speed axis. Gemma 4 bars flagged ⚠ (GPU-offload suspect).</div>
|
||||
</div>
|
||||
<div class="panel">
|
||||
<h2>▮ 5-Pillar Radar — Top 3</h2>
|
||||
<div class="chart-box"><canvas id="radar"></canvas></div>
|
||||
<div style="color:var(--dim);font-size:.72rem;margin-top:8px">Each pillar scored 0–20. Outer = stronger.</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="panel" style="margin-bottom:26px">
|
||||
<h2>▮ LEADERBOARD</h2>
|
||||
<div style="overflow-x:auto">
|
||||
<table>
|
||||
<thead><tr><th>#</th><th>Model</th><th>Speed</th><th>Score</th><th>Verdict</th><th>Best For</th><th></th></tr></thead>
|
||||
<tbody>{''.join(rows)}</tbody>
|
||||
</table></div>
|
||||
</div>
|
||||
<footer>Generated from <span class="mono">data/benchmark_history.json</span> · re-run <span class="mono">generate_dashboard.py</span> to refresh · cyberpunk-terminal UI</footer>
|
||||
<script>
|
||||
const NEON={{cyan:'#00ffc8',mag:'#ff2bd6',lime:'#b6ff3a',amber:'#ffb000',red:'#ff3b5c',blue:'#5b8cff'}};
|
||||
new Chart(document.getElementById('bar'),{{
|
||||
type:'bar',
|
||||
data:{{labels:{bar_labels},
|
||||
datasets:[
|
||||
{{label:'Score /100',data:{bar_score},backgroundColor:'rgba(0,255,200,0.85)',borderColor:NEON.cyan,borderRadius:3,yAxisID:'y'}},
|
||||
{{label:'tok/sec',data:{bar_speed},backgroundColor:'rgba(255,43,214,0.7)',borderColor:NEON.mag,borderRadius:3,yAxisID:'y1'}}
|
||||
]}},
|
||||
options:{{maintainAspectRatio:false,responsive:true,
|
||||
plugins:{{legend:{{labels:{{color:'#d7e0e6',font:{{family:'Fira Code'}}}}}}}},
|
||||
scales:{{
|
||||
x:{{ticks:{{color:'#7a8590',font:{{family:'Fira Code',size:10}}}},grid:{{color:'rgba(255,255,255,0.05)'}}}},
|
||||
y:{{position:'left',max:100,title:{{display:true,text:'Score',color:'#00ffc8'}},ticks:{{color:'#7a8590'}},grid:{{color:'rgba(255,255,255,0.05)'}}}},
|
||||
y1:{{position:'right',title:{{display:true,text:'tok/sec',color:'#ff2bd6'}},grid:{{drawOnChartArea:false}},ticks:{{color:'#7a8590'}}}}
|
||||
}}}}
|
||||
}});
|
||||
new Chart(document.getElementById('radar'),{{
|
||||
type:'radar',
|
||||
data:{{labels:{radar_labels},datasets:{radar_json}}},
|
||||
options:{{maintainAspectRatio:false,responsive:true,
|
||||
plugins:{{legend:{{labels:{{color:'#d7e0e6',font:{{family:'Fira Code',size:11}}}}}}}},
|
||||
scales:{{r:{{min:0,max:20,
|
||||
angleLines:{{color:'rgba(0,255,200,0.15)'}},
|
||||
grid:{{color:'rgba(0,255,200,0.12)'}},
|
||||
pointLabels:{{color:'#d7e0e6',font:{{family:'Fira Code',size:10}}}},
|
||||
ticks:{{color:'#7a8590',backdropColor:'transparent',stepSize:5}}
|
||||
}}}}
|
||||
}}
|
||||
}});
|
||||
</script>
|
||||
{FOOT}"""
|
||||
return body
|
||||
|
||||
def render_detail(m, data):
|
||||
col, chip = verdict_meta(m["verdict"])
|
||||
# derive "what went right" from high pillars, "wrong" from low + critical_bugs
|
||||
bd = m["breakdown"]
|
||||
ranked = sorted(PILLARS, key=lambda p: -bd[p])
|
||||
rights = [f"{PILLAR_LABELS[p]} ({bd[p]}/20)" for p in ranked if bd[p] >= 16]
|
||||
wrongs_pillars = [f"{PILLAR_LABELS[p]} ({bd[p]}/20)" for p in ranked if bd[p] <= 13]
|
||||
|
||||
pillar_bars = ""
|
||||
for p in PILLARS:
|
||||
v = bd[p]
|
||||
c = NEON_LIME if v >= 17 else (NEON_AMBER if v >= 13 else NEON_RED)
|
||||
pillar_bars += f"""
|
||||
<div class="prow">
|
||||
<div class="plabel">{PILLAR_LABELS[p]}</div>
|
||||
<div class="pbar"><i style="width:{v/20*100}%;background:{c};color:{c}"></i></div>
|
||||
<div class="pval mono" style="color:{c}">{v}<span style="color:var(--dim)">/20</span></div>
|
||||
</div>"""
|
||||
|
||||
bugs_html = "".join(f"<li>{esc(b)}</li>" for b in m.get("critical_bugs", []))
|
||||
patch = m.get("patch_code", "")
|
||||
metrics_block = ""
|
||||
if m.get("tok_sec") is not None:
|
||||
metrics_block = f"""
|
||||
<div class="mini"><span class="mlbl">tok/sec</span><span class="mval mono" style="color:var(--mag)">{m['tok_sec']:.2f}</span></div>
|
||||
<div class="mini"><span class="mlbl">tokens</span><span class="mval mono">{m.get('total_tokens') or '—'}</span></div>
|
||||
<div class="mini"><span class="mlbl">TTFT</span><span class="mval mono">{m.get('ttft_sec'):.2f}s</span> </div>"""
|
||||
else:
|
||||
metrics_block = '<div class="mini" style="grid-column:1/-1"><span class="mlbl">runtime</span><span class="mval mono" style="color:var(--blue)">CLOUD — not measured</span></div>'
|
||||
|
||||
caveat_html = f'<div class="caveat-box">⚠ SPEED CAVEAT: {esc(m["speed_caveat"])}</div>' if m.get("speed_caveat") else ""
|
||||
tests_html = '<span class="ok">PASS</span>' if m.get("tests_pass") else '<span class="bad">CRASH</span>'
|
||||
cloud_tag = '<span class="cloud-tag">CLOUD BASELINE</span>' if m.get("format") == "cloud" else ""
|
||||
|
||||
rights_html = "".join(f"<li>{esc(r)}</li>" for r in rights) or "<li class='dim'>No pillar reached 16+ — no standout strengths.</li>"
|
||||
wrongs_html = "".join(f"<li>{esc(w)}</li>" for w in wrongs_pillars) or "<li class='dim'>No pillar fell below 14 — solid across the board.</li>"
|
||||
|
||||
extra_css = """
|
||||
.mini-grid{display:grid;grid-template-columns:repeat(3,1fr);gap:10px;margin:14px 0}
|
||||
.mini{background:var(--panel2);border:1px solid rgba(255,255,255,0.06);border-radius:5px;padding:10px 12px}
|
||||
.mlbl{display:block;color:var(--dim);font-size:.62rem;letter-spacing:.14em;text-transform:uppercase}
|
||||
.mval{display:block;font-size:1.15rem;margin-top:3px}
|
||||
.caveat-box{background:rgba(255,176,0,0.08);border:1px solid rgba(255,176,0,0.4);color:var(--amber);padding:10px 14px;border-radius:5px;font-size:.82rem;margin:12px 0}
|
||||
.pillar-list{margin-top:8px}
|
||||
.prow{display:grid;grid-template-columns:170px 1fr 56px;align-items:center;gap:12px;margin:9px 0}
|
||||
.plabel{color:var(--ink);font-size:.84rem}
|
||||
.pbar{height:9px;background:rgba(255,255,255,0.06);border-radius:2px;overflow:hidden}
|
||||
.pbar > i{display:block;height:100%;border-radius:2px}
|
||||
.pval{text-align:right;font-size:.92rem}
|
||||
.ok{color:var(--lime);border:1px solid var(--lime);padding:1px 8px;border-radius:3px;font-size:.7rem}
|
||||
.bad{color:var(--red);border:1px solid var(--red);padding:1px 8px;border-radius:3px;font-size:.7rem}
|
||||
.section{background:var(--panel);border:1px solid rgba(255,255,255,0.07);border-radius:8px;padding:18px;margin-top:18px}
|
||||
.section h2{font-size:.9rem;letter-spacing:.12em;text-transform:uppercase;color:var(--cyan);margin:0 0 12px;text-shadow:0 0 10px rgba(0,255,200,0.35)}
|
||||
ul.clean{list-style:none;padding:0;margin:0}
|
||||
ul.clean li{padding:7px 0 7px 18px;border-bottom:1px solid rgba(255,255,255,0.05);position:relative;font-size:.88rem}
|
||||
ul.clean li::before{content:"▸";position:absolute;left:0;color:var(--cyan)}
|
||||
ul.clean li.dim{color:var(--dim)}
|
||||
ul.clean li.dim::before{color:var(--dim)}
|
||||
.bugs li::before{content:"✗";color:var(--red)}
|
||||
.rights li::before{content:"✓";color:var(--lime)}
|
||||
pre.code{background:#06060a;border:1px solid rgba(0,255,200,0.2);border-radius:6px;padding:14px;overflow-x:auto;color:#cfe;font-size:.78rem;line-height:1.5;box-shadow:inset 0 0 30px rgba(0,0,0,0.6)}
|
||||
pre.code::before{content:"// patch.py";display:block;color:var(--dim);font-size:.68rem;margin-bottom:8px}
|
||||
.back{display:inline-block;margin-bottom:18px;font-size:.8rem}
|
||||
.verdict-chip-lg{display:inline-block;padding:4px 14px;border-radius:3px;border:1px solid currentColor;font-size:.78rem;letter-spacing:.1em}
|
||||
.best-box{background:linear-gradient(135deg,rgba(0,255,200,0.08),rgba(91,139,255,0.05));border:1px solid rgba(0,255,200,0.25);border-radius:8px;padding:16px 18px;font-size:.95rem;line-height:1.6}
|
||||
@media(max-width:640px){.prow{grid-template-columns:1fr auto}.mini-grid{grid-template-columns:repeat(2,1fr)}}
|
||||
"""
|
||||
head = head_html(m["model_name"])
|
||||
# inject extra css before </style>
|
||||
head = head.replace("</style>", extra_css + "</style>")
|
||||
|
||||
return f"""
|
||||
{head}
|
||||
<a class="back" href="../dashboard.html">◂ BACK TO LEADERBOARD</a>
|
||||
<header class="hud-bar">
|
||||
<div class="kicker">▚ MODEL AUDIT // {esc(m['quant'])}</div>
|
||||
<h1>// {esc(m['model_name'])} {cloud_tag}</h1>
|
||||
<div style="margin-top:8px;display:flex;gap:14px;align-items:center;flex-wrap:wrap">
|
||||
<span class="verdict-chip-lg" style="color:{col}">{chip} — {esc(m['verdict'])}</span>
|
||||
<span class="mono" style="font-size:1.5rem;color:{col}">{m['total_score']}<span style="color:var(--dim);font-size:.8rem">/100</span></span>
|
||||
<span>tests: {tests_html}</span>
|
||||
</div>
|
||||
</header>
|
||||
{caveat_html}
|
||||
<div class="mini-grid">{metrics_block}</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>▮ PILLAR BREAKDOWN</h2>
|
||||
<div class="pillar-list">{pillar_bars}</div>
|
||||
</div>
|
||||
|
||||
<div class="grid2">
|
||||
<div class="section" style="margin-top:0">
|
||||
<h2 style="color:var(--lime);text-shadow:0 0 10px rgba(182,255,58,0.4)">✓ WENT RIGHT</h2>
|
||||
<ul class="clean rights">{rights_html}</ul>
|
||||
</div>
|
||||
<div class="section" style="margin-top:0">
|
||||
<h2 style="color:var(--red);text-shadow:0 0 10px rgba(255,59,92,0.4)">✗ WENT WRONG</h2>
|
||||
<ul class="clean wrongs">{wrongs_html}</ul>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>▮ CRITICAL BUGS</h2>
|
||||
<ul class="clean bugs">{bugs_html or '<li class="dim">None flagged.</li>'}</ul>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>▮ RECOMMENDED USE</h2>
|
||||
<div class="best-box">{esc(m['best_for'])}</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>▮ REFACTORED PATCH</h2>
|
||||
<pre class="code">{esc(patch)}</pre>
|
||||
</div>
|
||||
|
||||
<footer>Source: <span class="mono">{esc(m['filename'])}</span> · graded {esc(m.get('timestamp','')[:10])}</footer>
|
||||
</div>
|
||||
{FOOT}"""
|
||||
|
||||
def main():
|
||||
with open(DATA) as f:
|
||||
data = json.load(f)
|
||||
os.makedirs(PAGES_DIR, exist_ok=True)
|
||||
with open(OUT_DASH, "w") as f:
|
||||
f.write(render_dashboard(data))
|
||||
for m in data["models"]:
|
||||
with open(os.path.join(PAGES_DIR, m["id"] + ".html"), "w") as f:
|
||||
f.write(render_detail(m, data))
|
||||
print(f"OK: dashboard.html + {len(data['models'])} pages in pages/")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,262 @@
|
||||
import asyncio
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
|
||||
@dataclass
|
||||
class Node:
|
||||
key: str
|
||||
value: Any
|
||||
ttl_expiry: float
|
||||
freq: int = 1
|
||||
|
||||
def __repr__(self):
|
||||
return f"Node({self.key}, freq={self.freq})"
|
||||
|
||||
|
||||
class DoublyLinkedListList:
|
||||
def __init__(self):
|
||||
self.head = Node("", "", 0) # Dummy head
|
||||
self.tail = Node("", "", 0) # Dummy tail
|
||||
self.head.next = self.tail
|
||||
self.tail.prev = self.head
|
||||
|
||||
def add_to_front(self, node: Node):
|
||||
node.next = self.head.next
|
||||
node.prev = self.head
|
||||
self.head.next = node
|
||||
node.next.prev = node
|
||||
|
||||
def remove(self, node: Node):
|
||||
node.prev.next = node.next
|
||||
node.next.prev = node.prev
|
||||
|
||||
def move_to_front(self, node: Node):
|
||||
self.remove(node)
|
||||
self.add_to_front(node)
|
||||
|
||||
def pop_tail(self) -> Optional[Node]:
|
||||
if self.tail == self.tail.prev: # Only dummy tail left
|
||||
return None
|
||||
node = self.tail.prev
|
||||
self.remove(node)
|
||||
return node
|
||||
|
||||
|
||||
class ConcurrentLFUCache:
|
||||
def __init__(self, capacity: int):
|
||||
if capacity <= 0:
|
||||
raise ValueError("Capacity must be positive")
|
||||
self.capacity = capacity
|
||||
self.cache: Dict[str, Node] = {} # key -> node
|
||||
self.freq_buckets: Dict[int, DoublyLinkedListList] = {} # freq -> list
|
||||
self.min_freq = 0
|
||||
self.global_lock = asyncio.Lock() # Protects global cache structure
|
||||
self.evictor_task: Optional[asyncio.Task] = None
|
||||
|
||||
def _update_freq(self, node: Node):
|
||||
old_freq = node.freq
|
||||
bucket = self.freq_buckets[old_freq]
|
||||
bucket.remove(node)
|
||||
|
||||
if old_freq == self.min_freq and bucket.tail == bucket.head:
|
||||
self.min_freq += 1
|
||||
|
||||
node.freq += 1
|
||||
if node.freq not in self.freq_buckets:
|
||||
self.freq_buckets[node.freq] = DoublyLinkedListList()
|
||||
|
||||
bucket = self.freq_buckets[node.freq]
|
||||
bucket.add_to_front(node)
|
||||
|
||||
async def get(self, key: str) -> Optional[Any]:
|
||||
now = time.time()
|
||||
async with self.global_lock:
|
||||
node = self.cache.get(key)
|
||||
if not node:
|
||||
return None
|
||||
|
||||
if now > node.ttl_expiry: # Lazy eviction
|
||||
bucket = self.freq_buckets[node.freq]
|
||||
bucket.remove(node)
|
||||
del self.cache[key]
|
||||
return None
|
||||
|
||||
self._update_freq(node)
|
||||
return node.value
|
||||
|
||||
async def put(self, key: str, value: Any, ttl_seconds: float):
|
||||
now = time.time()
|
||||
expiry = now + ttl_seconds
|
||||
|
||||
async with self.global_lock:
|
||||
if key in self.cache:
|
||||
node = self.cache[key]
|
||||
bucket = self.freq_buckets[node.freq]
|
||||
bucket.remove(node)
|
||||
node.value = value
|
||||
node.ttl_expiry = expiry
|
||||
self._update_freq(node)
|
||||
else:
|
||||
if len(self.cache) >= self.capacity:
|
||||
bucket = self.freq_buckets[self.min_freq]
|
||||
evicted = bucket.pop_tail()
|
||||
if evicted:
|
||||
del self.cache[evicted.key]
|
||||
|
||||
new_node = Node(key, value, expiry)
|
||||
self.cache[key] = new_node
|
||||
if self.min_freq == 0:
|
||||
self.min_freq = 1
|
||||
bucket = self.freq_buckets[self.min_freq]
|
||||
bucket.add_to_front(new_node)
|
||||
|
||||
async def start_evictor(self, interval: float = 1.0):
|
||||
def evict_loop():
|
||||
while True:
|
||||
time.sleep(interval)
|
||||
now = time.time()
|
||||
expired_keys = [k for k, n in self.cache.items() if now > n.ttl_expiry]
|
||||
for k in expired_keys:
|
||||
asyncio.run(self.put(k, self.cache[k].value, 0)) # Re-insert with 0 TTL
|
||||
|
||||
self.evictor_task = asyncio.create_task(evict_loop())
|
||||
|
||||
async def stop_evictor(self):
|
||||
if self.evictor_task:
|
||||
self.evictor_task.cancel()
|
||||
|
||||
|
||||
class Transaction:
|
||||
def __init__(self, cache):
|
||||
self.cache = cache
|
||||
self.local_cache: Dict[str, Node] = {} # key -> node
|
||||
self.local_freq: Dict[int, DoublyLinkedListList] = {} # freq -> list
|
||||
self.local_min_freq = 0
|
||||
|
||||
def _update_local_freq(self, node: Node):
|
||||
old_freq = node.freq
|
||||
bucket = self.local_freq[old_freq]
|
||||
bucket.remove(node)
|
||||
|
||||
if old_freq == self.local_min_freq and bucket.tail == bucket.head:
|
||||
self.local_min_freq += 1
|
||||
|
||||
node.freq += 1
|
||||
if node.freq not in self.local_freq:
|
||||
self.local_freq[node.freq] = DoublyLinkedListList()
|
||||
|
||||
bucket = self.local_freq[node.freq]
|
||||
bucket.add_to_front(node)
|
||||
|
||||
async def get(self, key: str) -> Optional[Any]:
|
||||
if key in self.local_cache:
|
||||
node = self.local_cache[key]
|
||||
self._update_local_freq(node)
|
||||
return node.value
|
||||
|
||||
async with self.cache.global_lock:
|
||||
node = self.cache.cache.get(key)
|
||||
if not node:
|
||||
return None
|
||||
|
||||
# Copy the node so local changes don't affect global cache until commit
|
||||
new_node = Node(key, node.value, node.ttl_expiry, node.freq)
|
||||
self.local_cache[key] = new_node
|
||||
if self.local_min_freq == 0:
|
||||
self.local_min_freq = new_node.freq
|
||||
bucket = self.local_freq[new_node.freq]
|
||||
bucket.add_to_front(new_node)
|
||||
return new_node.value
|
||||
|
||||
async def put(self, key: str, value: Any, ttl_seconds: float):
|
||||
now = time.time()
|
||||
expiry = now + ttl_seconds
|
||||
|
||||
if key in self.local_cache:
|
||||
node = self.local_cache[key]
|
||||
bucket = self.local_freq[node.freq]
|
||||
bucket.remove(node)
|
||||
node.value = value
|
||||
node.ttl_expiry = expiry
|
||||
self._update_local_freq(node)
|
||||
else:
|
||||
if len(self.local_cache) >= self.cache.capacity:
|
||||
bucket = self.local_freq[self.local_min_freq]
|
||||
evicted = bucket.pop_tail()
|
||||
if evicted:
|
||||
del self.local_cache[evicted.key]
|
||||
|
||||
new_node = Node(key, value, expiry)
|
||||
self.local_cache[key] = new_node
|
||||
if self.local_min_freq == 0:
|
||||
self.local_min_freq = new_node.freq
|
||||
bucket = self.local_freq[new_node.freq]
|
||||
bucket.add_to_front(new_node)
|
||||
|
||||
async def commit(self):
|
||||
async with self.cache.global_lock:
|
||||
for key, node in self.local_cache.items():
|
||||
if key in self.cache.cache:
|
||||
old_node = self.cache.cache[key]
|
||||
bucket = self.cache.freq_buckets[old_node.freq]
|
||||
bucket.remove(old_node)
|
||||
self.cache.cache[key] = node
|
||||
if old_node.freq == self.cache.min_freq and bucket.tail == bucket.head:
|
||||
self.cache.min_freq += 1
|
||||
node.freq = node.freq # freq is already updated in local_cache
|
||||
bucket = self.cache.freq_buckets[node.freq]
|
||||
if node.freq not in self.cache.freq_buckets:
|
||||
self.cache.freq_buckets[node.freq] = DoublyLinkedListList()
|
||||
bucket.add_to_front(node)
|
||||
|
||||
async def rollback(self):
|
||||
self.local_cache = {}
|
||||
self.local_freq = {}
|
||||
self.local_min_freq = 0
|
||||
|
||||
|
||||
async def main():
|
||||
cache = ConcurrentLFUCache(capacity=3)
|
||||
|
||||
# a. O(1) LFU eviction order
|
||||
await cache.put("a", "val_a", 60) # freq=1
|
||||
await cache.put("b", "val_b", 60) # freq=1
|
||||
await cache.put("c", "val_c", 60) # freq=1
|
||||
await cache.get("a") # a:freq=2, b/c:freq=1
|
||||
await cache.get("b") # a:freq=2, b:freq=2, c:freq=1
|
||||
await cache.put("d", "val_d", 60) # c is evicted (lowest freq=1)
|
||||
assert await cache.get("c") is None
|
||||
|
||||
# b. Lazy vs Background eviction
|
||||
await cache.put("e", "val_e", 0) # expire immediately
|
||||
assert await cache.get("e") is None # lazy eviction
|
||||
|
||||
# c. Transaction commit vs rollback
|
||||
tx = await cache.begin_transaction() # helper below
|
||||
await tx.put("f", "val_f", 60)
|
||||
assert await tx.get("f") == "val_f" # Read Your Own Writes
|
||||
assert await cache.get("f") is None # Global not yet committed
|
||||
|
||||
await tx.rollback()
|
||||
assert await cache.get("f") is None # Rollback restored
|
||||
|
||||
await tx.commit()
|
||||
assert await cache.get("f") == "val_f" # Committed
|
||||
|
||||
# d. Stress test
|
||||
async def worker(i):
|
||||
await cache.put(f"key_{i}", i, 10)
|
||||
await cache.get(f"key_{i}")
|
||||
|
||||
tasks = [worker(i) for i in range(50)]
|
||||
await asyncio.gather(*tasks)
|
||||
|
||||
def begin_transaction(): # Helper for the test suite
|
||||
return Transaction(ConcurrentLFUCache(capacity=3))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
@@ -0,0 +1,343 @@
|
||||
import asyncio
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Dict, Optional, Generic, TypeVar, Set
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
# --- Internal Data Structures for O(1) LFU ---
|
||||
|
||||
@dataclass
|
||||
class Node:
|
||||
"""A node in the LFU frequency doubly linked list."""
|
||||
key: Any
|
||||
value: Any
|
||||
freq: int = 1
|
||||
expiry: Optional[float] = None
|
||||
prev: Optional['Node'] = None
|
||||
next: Optional['Node'] = None
|
||||
|
||||
class DoublyLinkedList:
|
||||
"""A standard DLL to maintain nodes of the same frequency."""
|
||||
def __init__(self):
|
||||
self.head: Optional[Node] = None
|
||||
self.tail: Optional[Node] = None
|
||||
self.size = 0
|
||||
|
||||
def append(self, node: Node):
|
||||
"""Add node to the front (most recently used in this freq bucket)."""
|
||||
node.next = self.head
|
||||
node.prev = None
|
||||
if self.head:
|
||||
self.head.prev = node
|
||||
self.head = node
|
||||
if not self.tail:
|
||||
self.tail = node
|
||||
self.size += 1
|
||||
|
||||
def pop_tail(self) -> Optional[Node]:
|
||||
"""Remove and return the least recently used node in this freq bucket."""
|
||||
if not self.tail:
|
||||
return None
|
||||
node = self.tail
|
||||
self.remove(node)
|
||||
return node
|
||||
|
||||
def remove(self, node: Node):
|
||||
"""Remove a specific node from the list in O(1)."""
|
||||
if node.prev:
|
||||
node.prev.next = node.next
|
||||
else:
|
||||
self.head = node.next
|
||||
|
||||
if node.next:
|
||||
node.next.prev = node.prev
|
||||
else:
|
||||
self.tail = node.prev
|
||||
|
||||
node.next = None
|
||||
node.prev = None
|
||||
self.size -= 1
|
||||
|
||||
# --- Cache Implementation ---
|
||||
|
||||
class LFUCache:
|
||||
def __init__(self, capacity: int):
|
||||
if capacity <= 0:
|
||||
raise ValueError("Capacity must be greater than 0")
|
||||
|
||||
self.capacity = capacity
|
||||
self.cache: Dict[Any, Node] = {} # key -> Node
|
||||
self.freq_map: Dict[int, DoublyLinkedList] = {} # freq -> DLL
|
||||
self.min_freq = 0
|
||||
self.lock = asyncio.Lock()
|
||||
self._evictor_task: Optional[asyncio.Task] = None
|
||||
|
||||
def _get_now(self) -> float:
|
||||
return time.time()
|
||||
|
||||
def _is_expired(self, node: Node) -> bool:
|
||||
return node.expiry is not None and self._get_now() > node.expiry
|
||||
|
||||
async def _update_freq(self, node: Node):
|
||||
"""Moves a node to the next frequency bucket in O(1)."""
|
||||
old_freq = node.freq
|
||||
dll = self.freq_map[old_freq]
|
||||
dll.remove(node)
|
||||
|
||||
if old_freq == self.min_freq and dll.size == 0:
|
||||
self.min_freq += 1
|
||||
|
||||
node.freq += 1
|
||||
if node.freq not in self.freq_map:
|
||||
self.freq_map[node.freq] = DoublyLinkedList()
|
||||
self.freq_map[node.freq].append(node)
|
||||
|
||||
async def get(self, key: Any) -> Optional[Any]:
|
||||
"""O(1) Access with Lazy TTL check."""
|
||||
async with self.lock:
|
||||
if key not in self.cache:
|
||||
return None
|
||||
|
||||
node = self.cache[key]
|
||||
if self._is_expired(node):
|
||||
await self._delete_node(node)
|
||||
return None
|
||||
|
||||
await self._update_freq(node)
|
||||
return node.value
|
||||
|
||||
async def put(self, key: Any, value: Any, ttl_seconds: Optional[float] = None):
|
||||
"""O(1) Insertion with LFU eviction and Lazy TTL check."""
|
||||
async with self.lock:
|
||||
expiry = self._get_now() + ttl_seconds if ttl_seconds else None
|
||||
|
||||
if key in self.cache:
|
||||
node = self.cache[key]
|
||||
node.value = value
|
||||
node.expiry = expiry
|
||||
await self._update_freq(node)
|
||||
return
|
||||
|
||||
# Evict if capacity reached
|
||||
if len(self.cache) >= self.capacity:
|
||||
# Priority 1: Try to find an expired key first (optimistic)
|
||||
# Otherwise: Evict the LFU (min_freq tail)
|
||||
evicted = self.freq_map[self.min_freq].pop_tail()
|
||||
if evicted:
|
||||
del self.cache[evicted.key]
|
||||
|
||||
# Create new node
|
||||
new_node = Node(key=key, value=value, expiry=expiry)
|
||||
self.cache[key] = new_node
|
||||
self.min_freq = 1
|
||||
if 1 not in self.freq_map:
|
||||
self.freq_map[1] = DoublyLinkedList()
|
||||
self.freq_map[1].append(new_node)
|
||||
|
||||
async def delete(self, key: Any):
|
||||
"""O(1) deletion."""
|
||||
async with self.lock:
|
||||
if key in self.cache:
|
||||
await self._delete_node(self.cache[key])
|
||||
|
||||
async def _delete_node(self, node: Node):
|
||||
"""Internal helper to remove node from all tracking structures."""
|
||||
dll = self.freq_map.get(node.freq)
|
||||
if dll:
|
||||
dll.remove(node)
|
||||
if node.key in self.cache:
|
||||
del self.cache[node.key]
|
||||
|
||||
# --- Background Evictor ---
|
||||
|
||||
async def start_evictor(self, interval: float = 1.0, batch_size: int = 100):
|
||||
"""Starts a non-blocking background task to purge expired keys."""
|
||||
if self._evictor_task:
|
||||
return
|
||||
|
||||
async def evict_loop():
|
||||
while True:
|
||||
await asyncio.sleep(interval)
|
||||
# Collect keys to check outside the lock to prevent blocking reads
|
||||
keys_to_check = list(self.cache.keys())
|
||||
|
||||
# Process in small batches to yield to the event loop
|
||||
for i in range(0, len(keys_to_check), batch_size):
|
||||
batch = keys_to_check[i : i + batch_size]
|
||||
async with self.lock:
|
||||
for k in batch:
|
||||
node = self.cache.get(k)
|
||||
if node and self._is_expired(node):
|
||||
await self._delete_node(node)
|
||||
# Yield control back to asyncio
|
||||
await asyncio.sleep(0)
|
||||
|
||||
self._evictor_task = asyncio.create_task(evict_loop())
|
||||
|
||||
async def stop_evictor(self):
|
||||
"""Stops the background eviction task."""
|
||||
if self._evictor_task:
|
||||
self._evictor_task.cancel()
|
||||
try:
|
||||
await self._evictor_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._evictor_task = None
|
||||
|
||||
# --- Transaction Support ---
|
||||
|
||||
def begin_transaction(self) -> 'Transaction':
|
||||
return Transaction(self)
|
||||
|
||||
async def apply_transaction_changes(self, changes: 'TransactionState'):
|
||||
"""Atomically apply buffered transaction writes to the global state."""
|
||||
async with self.lock:
|
||||
for key, action in changes.writes.items():
|
||||
if action == "DELETE":
|
||||
if key in self.cache:
|
||||
await self._delete_node(self.cache[key])
|
||||
else:
|
||||
val, ttl = action
|
||||
# Using put logic inside the lock
|
||||
expiry = self._get_now() + ttl if ttl else None
|
||||
if key in self.cache:
|
||||
node = self.cache[key]
|
||||
node.value = val
|
||||
node.expiry = expiry
|
||||
await self._update_freq(node)
|
||||
else:
|
||||
if len(self.cache) >= self.capacity:
|
||||
evicted = self.freq_map[self.min_freq].pop_tail()
|
||||
if evicted:
|
||||
del self.cache[evicted.key]
|
||||
new_node = Node(key=key, value=val, expiry=expiry)
|
||||
self.cache[key] = new_node
|
||||
self.min_freq = 1
|
||||
if 1 not in self.freq_map:
|
||||
self.freq_map[1] = DoublyLinkedList()
|
||||
self.freq_map[1].append(new_node)
|
||||
|
||||
@dataclass
|
||||
class TransactionState:
|
||||
writes: Dict[Any, Any] = None # key -> (value, ttl) or "DELETE"
|
||||
|
||||
class Transaction:
|
||||
def __init__(self, cache: LFUCache):
|
||||
self._cache = cache
|
||||
self._state = TransactionState(writes={})
|
||||
self._committed = False
|
||||
|
||||
async def put(self, key: Any, value: Any, ttl_seconds: Optional[float] = None):
|
||||
if self._committed: raise RuntimeError("Transaction already committed")
|
||||
self._state.writes[key] = (value, ttl_seconds)
|
||||
|
||||
async def get(self, key: Any) -> Optional[Any]:
|
||||
if self._committed: raise RuntimeError("Transaction already committed")
|
||||
|
||||
# Read-Your-Own-Writes (RYOW)
|
||||
if key in self._state.writes:
|
||||
action = self._state.writes[key]
|
||||
if action == "DELETE":
|
||||
return None
|
||||
return action[0]
|
||||
|
||||
# Otherwise read from global cache
|
||||
return await self._cache.get(key)
|
||||
|
||||
async def delete(self, key: Any):
|
||||
if self._committed: raise RuntimeError("Transaction already committed")
|
||||
self._state.writes[key] = "DELETE"
|
||||
|
||||
async def commit(self):
|
||||
if self._committed: raise RuntimeError("Transaction already committed")
|
||||
await self._cache.apply_transaction_changes(self._state)
|
||||
self._committed = True
|
||||
|
||||
async def rollback(self):
|
||||
if self._committed: raise RuntimeError("Transaction already committed")
|
||||
self._state.writes.clear()
|
||||
self._committed = True
|
||||
|
||||
# --- Executable Unit Tests ---
|
||||
|
||||
async def main():
|
||||
print("🚀 Starting LFU Cache Suite...")
|
||||
|
||||
# 1. Test O(1) LFU Eviction
|
||||
print("\nTesting LFU Eviction Order...", end=" ")
|
||||
cache = LFUCache(capacity=2)
|
||||
await cache.put("a", 1)
|
||||
await cache.put("b", 2)
|
||||
await cache.get("a") # freq(a)=2, freq(b)=1
|
||||
await cache.put("c", 3) # b should be evicted
|
||||
assert await cache.get("b") is None
|
||||
assert await cache.get("a") == 1
|
||||
assert await cache.get("c") == 3
|
||||
print("✅ PASSED")
|
||||
|
||||
# 2. Test Dual-Layer TTL
|
||||
print("\nTesting TTL Eviction...", end=" ")
|
||||
ttl_cache = LFUCache(capacity=10)
|
||||
await ttl_cache.put("lazy", "val", ttl_seconds=0.1)
|
||||
await ttl_cache.put("async", "val", ttl_seconds=0.1)
|
||||
|
||||
# Lazy check
|
||||
await asyncio.sleep(0.2)
|
||||
assert await ttl_cache.get("lazy") is None, "Lazy eviction failed"
|
||||
|
||||
# Background check
|
||||
await ttl_cache.start_evictor(interval=0.1, batch_size=10)
|
||||
await asyncio.sleep(0.2)
|
||||
async with ttl_cache.lock:
|
||||
assert "async" not in ttl_cache.cache, "Background evictor failed"
|
||||
await ttl_cache.stop_evictor()
|
||||
print("✅ PASSED")
|
||||
|
||||
# 3. Test Atomic Transactions
|
||||
print("\nTesting Atomic Transactions...", end=" ")
|
||||
tx_cache = LFUCache(capacity=10)
|
||||
await tx_cache.put("base", 100)
|
||||
|
||||
# Test Commit
|
||||
tx1 = tx_cache.begin_transaction()
|
||||
await tx1.put("base", 200)
|
||||
await tx1.put("new", 300)
|
||||
assert await tx1.get("base") == 200 # RYOW
|
||||
assert await tx_cache.get("base") == 100 # Isolation
|
||||
await tx1.commit()
|
||||
assert await tx_cache.get("base") == 200
|
||||
assert await tx_cache.get("new") == 300
|
||||
|
||||
# Test Rollback
|
||||
tx2 = tx_cache.begin_transaction()
|
||||
await tx2.put("base", 999)
|
||||
await tx2.get("base")
|
||||
await tx2.rollback()
|
||||
assert await tx_cache.get("base") == 200
|
||||
print("✅ PASSED")
|
||||
|
||||
# 4. Stress Test: Concurrent Async Tasks
|
||||
print("\nStress Testing 50 concurrent tasks...", end=" ")
|
||||
stress_cache = LFUCache(capacity=20)
|
||||
|
||||
async def worker(worker_id):
|
||||
for i in range(100):
|
||||
key = f"key_{i % 30}" # Contention on keys
|
||||
if i % 3 == 0:
|
||||
await stress_cache.put(key, f"val_{worker_id}_{i}")
|
||||
elif i % 3 == 1:
|
||||
await stress_cache.get(key)
|
||||
else:
|
||||
tx = stress_cache.begin_transaction()
|
||||
await tx.put(key, f"tx_{worker_id}_{i}")
|
||||
await tx.commit()
|
||||
|
||||
await asyncio.gather(*(worker(i) for i in range(50)))
|
||||
print("✅ PASSED")
|
||||
|
||||
print("\n✨ All tests passed successfully!")
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
@@ -0,0 +1,339 @@
|
||||
import asyncio
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Dict, Optional, Generic, TypeVar, Set
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
@dataclass
|
||||
class Node:
|
||||
"""A node in the LFU doubly linked list."""
|
||||
key: Any
|
||||
value: Any
|
||||
freq: int = 1
|
||||
expiry: float = float('inf')
|
||||
prev: Optional['Node'] = None
|
||||
next: Optional['Node'] = None
|
||||
|
||||
class DoublyLinkedList:
|
||||
"""Standard O(1) doubly linked list for maintaining frequency buckets."""
|
||||
def __init__(self):
|
||||
self.head = Node(None, None) # Sentinel head
|
||||
self.tail = Node(None, None) # Sentinel tail
|
||||
self.head.next = self.tail
|
||||
self.tail.prev = self.head
|
||||
self.size = 0
|
||||
|
||||
def append(self, node: Node):
|
||||
"""Adds a node to the end (most recently used in this frequency)."""
|
||||
node.next = self.tail
|
||||
node.prev = self.tail.prev
|
||||
self.tail.prev.next = node
|
||||
self.tail.prev = node
|
||||
self.size += 1
|
||||
|
||||
def remove(self, node: Node):
|
||||
"""Removes a specific node from the list."""
|
||||
if node.prev:
|
||||
node.prev.next = node.next
|
||||
if node.next:
|
||||
node.next.prev = node.prev
|
||||
node.prev = None
|
||||
node.next = None
|
||||
self.size -= 1
|
||||
|
||||
def pop_front(self) -> Optional[Node]:
|
||||
"""Removes and returns the oldest node (LFU candidate)."""
|
||||
if self.size == 0:
|
||||
return None
|
||||
node = self.head.next
|
||||
self.remove(node)
|
||||
return node
|
||||
|
||||
def is_empty(self) -> bool:
|
||||
return self.size == 0
|
||||
|
||||
|
||||
class LFUCache:
|
||||
"""
|
||||
In-Memory Concurrent LFU Cache with Async TTL Eviction and Atomic Transactions.
|
||||
Time Complexity: O(1) for get and put.
|
||||
Space Complexity: O(N).
|
||||
"""
|
||||
def __init__(self, capacity: int):
|
||||
if capacity <= 0:
|
||||
raise ValueError("Capacity must be greater than 0")
|
||||
|
||||
self.capacity = capacity
|
||||
self.cache: Dict[Any, Node] = {} # Key -> Node
|
||||
self.freq_map: Dict[int, DoublyLinkedList] = {} # Freq -> DLL
|
||||
self.min_freq = 0
|
||||
self._lock = asyncio.Lock()
|
||||
self._evictor_task: Optional[asyncio.Task] = None
|
||||
|
||||
async def get(self, key: Any) -> Optional[Any]:
|
||||
async with self._lock:
|
||||
return await self._get_internal(key)
|
||||
|
||||
async def _get_internal(self, key: Any) -> Optional[Any]:
|
||||
"""Internal get without lock for transaction use."""
|
||||
if key not in self.cache:
|
||||
return None
|
||||
|
||||
node = self.cache[key]
|
||||
|
||||
# Lazy Eviction check
|
||||
if time.time() > node.expiry:
|
||||
await self._delete_internal(key)
|
||||
return None
|
||||
|
||||
self._update_frequency(node)
|
||||
return node.value
|
||||
|
||||
async def put(self, key: Any, value: Any, ttl_seconds: Optional[float] = None):
|
||||
async with self._lock:
|
||||
await self._put_internal(key, value, ttl_seconds)
|
||||
|
||||
async def _put_internal(self, key: Any, value: Any, ttl_seconds: Optional[float] = None):
|
||||
"""Internal put without lock for transaction use."""
|
||||
expiry = time.time() + ttl_seconds if ttl_seconds is not None else float('inf')
|
||||
|
||||
if key in self.cache:
|
||||
node = self.cache[key]
|
||||
node.value = value
|
||||
node.expiry = expiry
|
||||
self._update_frequency(node)
|
||||
else:
|
||||
# Capacity Management
|
||||
if len(self.cache) >= self.capacity:
|
||||
await self._evict_lfu()
|
||||
|
||||
new_node = Node(key, value, freq=1, expiry=expiry)
|
||||
self.cache[key] = new_node
|
||||
self._add_to_freq_bucket(1, new_node)
|
||||
self.min_freq = 1
|
||||
|
||||
async def delete(self, key: Any):
|
||||
async with self._lock:
|
||||
await self._delete_internal(key)
|
||||
|
||||
async def _delete_internal(self, key: Any):
|
||||
"""Internal delete without lock for transaction use."""
|
||||
if key in self.cache:
|
||||
node = self.cache.pop(key)
|
||||
self.freq_map[node.freq].remove(node)
|
||||
# Note: We don't strictly need to update min_freq here because
|
||||
# _evict_lfu handles empty buckets by incrementing.
|
||||
|
||||
def _update_frequency(self, node: Node):
|
||||
"""Moves a node to the next frequency bucket in O(1)."""
|
||||
old_freq = node.freq
|
||||
self.freq_map[old_freq].remove(node)
|
||||
|
||||
if old_freq == self.min_freq and self.freq_map[old_freq].is_empty():
|
||||
self.min_freq += 1
|
||||
|
||||
node.freq += 1
|
||||
self._add_to_freq_bucket(node.freq, node)
|
||||
|
||||
def _add_to_freq_bucket(self, freq: int, node: Node):
|
||||
if freq not in self.freq_map:
|
||||
self.freq_map[freq] = DoublyLinkedList()
|
||||
self.freq_map[freq].append(node)
|
||||
|
||||
async def _evict_lfu(self):
|
||||
"""Evicts the least frequently used (and oldest within that freq) item."""
|
||||
while self.min_freq not in self.freq_map or self.freq_map[self.min_freq].is_empty():
|
||||
# This handles cases where items were deleted manually
|
||||
if not self.cache: return
|
||||
self.min_freq += 1 # Search for the next available freq bucket
|
||||
|
||||
evicted_node = self.freq_map[self.min_freq].pop_front()
|
||||
if evicted_node:
|
||||
del self.cache[evicted_node.key]
|
||||
|
||||
# --- Background Evictor ---
|
||||
|
||||
async def start_evictor(self, interval: float = 1.0):
|
||||
"""Starts the background task that purges expired keys."""
|
||||
if self._evictor_task is not None:
|
||||
return
|
||||
self._evictor_task = asyncio.create_task(self._eviction_loop(interval))
|
||||
|
||||
async def stop_evictor(self):
|
||||
"""Stops the background eviction task."""
|
||||
if self._evictor_task:
|
||||
self._evictor_task.cancel()
|
||||
try:
|
||||
await self._evictor_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._evictor_task = None
|
||||
|
||||
async def _eviction_loop(self, interval: float):
|
||||
while True:
|
||||
await asyncio.sleep(interval)
|
||||
# Process in small batches to avoid blocking the main lock for too long
|
||||
now = time.time()
|
||||
keys_to_check = list(self.cache.keys())
|
||||
|
||||
batch_size = 100
|
||||
for i in range(0, len(keys_to_check), batch_size):
|
||||
async with self._lock:
|
||||
batch = keys_to_check[i : i + batch_size]
|
||||
for k in batch:
|
||||
if k in self.cache and now > self.cache[k].expiry:
|
||||
await self._delete_internal(k)
|
||||
|
||||
# --- Transaction Logic ---
|
||||
|
||||
def begin_transaction(self):
|
||||
return Transaction(self)
|
||||
|
||||
|
||||
class Transaction:
|
||||
"""
|
||||
Provides ACID-like atomic updates to the LFUCache.
|
||||
Implements 'Read Your Own Writes' (RYOW).
|
||||
"""
|
||||
def __init__(self, cache: LFUCache):
|
||||
self._cache = cache
|
||||
self._pending_puts: Dict[Any, tuple[Any, Optional[float]]] = {}
|
||||
self._pending_deletes: Set[Any] = set()
|
||||
self._committed = False
|
||||
|
||||
async def get(self, key: Any) -> Optional[Any]:
|
||||
if self._committed: raise RuntimeError("Transaction already committed")
|
||||
|
||||
# 1. Check local deletes first
|
||||
if key in self._pending_deletes:
|
||||
return None
|
||||
|
||||
# 2. Check local writes (RYOW)
|
||||
if key in self._pending_puts:
|
||||
return self._pending_puts[key][0]
|
||||
|
||||
# 3. Fallback to global cache
|
||||
return await self._cache.get(key)
|
||||
|
||||
async def put(self, key: Any, value: Any, ttl_seconds: Optional[float] = None):
|
||||
if self._committed: raise RuntimeError("Transaction already committed")
|
||||
self._pending_deletes.discard(key)
|
||||
self._pending_puts[key] = (value, ttl_seconds)
|
||||
|
||||
async def delete(self, key: Any):
|
||||
if self._committed: raise RuntimeError("Transaction already committed")
|
||||
self._pending_puts.pop(key, None)
|
||||
self._pending_deletes.add(key)
|
||||
|
||||
async def commit(self):
|
||||
"""Applies all pending changes to the global cache atomically."""
|
||||
if self._committed: return
|
||||
|
||||
async with self._cache._lock:
|
||||
# Apply deletes first
|
||||
for key in self._pending_deletes:
|
||||
await self._cache._delete_internal(key)
|
||||
# Apply puts
|
||||
for key, (val, ttl) in self._pending_puts.items():
|
||||
await self._cache._put_internal(key, val, ttl)
|
||||
|
||||
self._committed = True
|
||||
|
||||
async def rollback(self):
|
||||
"""Discards all pending changes."""
|
||||
self._pending_puts.clear()
|
||||
self._pending_deletes.clear()
|
||||
self._committed = True
|
||||
|
||||
|
||||
# ==========================================
|
||||
# Unit Tests
|
||||
# ==========================================
|
||||
|
||||
async def main():
|
||||
print("Starting LFU Cache Comprehensive Test Suite...")
|
||||
|
||||
# 1. Test O(1) LFU Eviction Order
|
||||
print("\nTesting LFU Eviction Order...", end=" ")
|
||||
cache = LFUCache(capacity=2)
|
||||
await cache.put("a", 1) # freq 1
|
||||
await cache.put("b", 2) # freq 1
|
||||
await cache.get("a") # freq 2
|
||||
await cache.put("c", 3) # "b" should be evicted as it's the LFU (freq 1)
|
||||
assert await cache.get("b") is None, "Key 'b' should have been evicted"
|
||||
assert await cache.get("a") == 1, "Key 'a' should still exist"
|
||||
assert await cache.get("c") == 3, "Key 'c' should still exist"
|
||||
print("PASSED")
|
||||
|
||||
# 2. Test TTL Eviction (Lazy & Background)
|
||||
print("Testing Dual-Layer TTL...", end=" ")
|
||||
cache = LFUCache(capacity=10)
|
||||
await cache.start_evictor(interval=0.1)
|
||||
|
||||
await cache.put("lazy", "val", ttl_seconds=0.1)
|
||||
await asyncio.sleep(0.2)
|
||||
# Lazy check: get() should trigger deletion
|
||||
assert await cache.get("lazy") is None, "Lazy eviction failed"
|
||||
|
||||
await cache.put("bg", "val", ttl_seconds=0.1)
|
||||
await asyncio.sleep(0.3)
|
||||
# Background check: key should be gone from internal map without calling get()
|
||||
async with cache._lock:
|
||||
assert "bg" not in cache.cache, "Background eviction failed"
|
||||
await cache.stop_evictor()
|
||||
print("PASSED")
|
||||
|
||||
# 3. Test Atomic Transactions (Commit vs Rollback)
|
||||
print("Testing Transaction Isolation...", end=" ")
|
||||
cache = LFUCache(capacity=10)
|
||||
await cache.put("shared", "original")
|
||||
|
||||
tx = cache.begin_transaction()
|
||||
await tx.put("shared", "modified")
|
||||
await tx.put("new", "added")
|
||||
|
||||
# Verify Isolation: Global reader shouldn't see changes
|
||||
assert await cache.get("shared") == "original"
|
||||
assert await cache.get("new") is None
|
||||
|
||||
# Verify RYOW: Transaction should see its own writes
|
||||
assert await tx.get("shared") == "modified"
|
||||
assert await tx.get("new") == "added"
|
||||
|
||||
await tx.commit()
|
||||
assert await cache.get("shared") == "modified", "Commit failed to update global state"
|
||||
assert await cache.get("new") == "added", "Commit failed to add new key"
|
||||
|
||||
tx2 = cache.begin_transaction()
|
||||
await tx2.put("shared", "rollback_this")
|
||||
assert await tx2.get("shared") == "rollback_this"
|
||||
await tx2.rollback()
|
||||
assert await cache.get("shared") == "modified", "Rollback mutated global state"
|
||||
print("PASSED")
|
||||
|
||||
# 4. Stress Test: Concurrent Access
|
||||
print("Testing High Concurrency (50 tasks)...", end=" ")
|
||||
cache = LFUCache(capacity=100)
|
||||
|
||||
async def worker(worker_id):
|
||||
for i in range(100):
|
||||
key = f"key_{i % 20}" # Create contention on keys
|
||||
if i % 3 == 0:
|
||||
await cache.put(key, f"val_{worker_id}_{i}", ttl_seconds=0.5)
|
||||
elif i % 3 == 1:
|
||||
await cache.get(key)
|
||||
else:
|
||||
tx = cache.begin_transaction()
|
||||
await tx.put(f"tx_{worker_id}", i)
|
||||
await tx.commit()
|
||||
|
||||
tasks = [worker(i) for i in range(50)]
|
||||
await asyncio.gather(*tasks)
|
||||
print("PASSED")
|
||||
|
||||
print("\nAll tests passed successfully!")
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
@@ -0,0 +1,550 @@
|
||||
"""
|
||||
In-Memory Concurrent LFU Cache with Async TTL Eviction and Atomic Transactions.
|
||||
|
||||
Pure Python 3.11+ implementation — no third-party dependencies.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
import weakref
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Optional
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Doubly-Linked List (O(1) move/remove/add)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class _DLLNode:
|
||||
"""Element inside a frequency bucket's doubly-linked list."""
|
||||
key: Any
|
||||
value: Any
|
||||
ttl_seconds: float
|
||||
expiry_time: float
|
||||
freq: int = 0
|
||||
prev: Optional[_DLLNode] = field(default=None, repr=False)
|
||||
next: Optional[_DLLNode] = field(default=None, repr=False)
|
||||
|
||||
|
||||
class _DoublyLinkedList:
|
||||
"""
|
||||
Doubly-linked list with sentinel head/tail for O(1) prepend, pop, and remove.
|
||||
The *tail* side holds the least-recently-used node within a frequency tier,
|
||||
which is also the LFU candidate for eviction.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._head = _DLLNode(None, None, 0.0, 0.0) # sentinel
|
||||
self._tail = _DLLNode(None, None, 0.0, 0.0) # sentinel
|
||||
self._head.next = self._tail
|
||||
self._tail.prev = self._head
|
||||
self.size = 0
|
||||
|
||||
# -- internal helpers ---------------------------------------------------
|
||||
|
||||
def _insert_after(self, node: _DLLNode, prev: _DLLNode) -> None:
|
||||
nxt = prev.next
|
||||
prev.next = node
|
||||
node.prev = prev
|
||||
node.next = nxt
|
||||
nxt.prev = node
|
||||
|
||||
def _unlink(self, node: _DLLNode) -> None:
|
||||
prev, nxt = node.prev, node.next
|
||||
prev.next = nxt
|
||||
nxt.prev = prev
|
||||
node.prev = node.next = None
|
||||
|
||||
# -- public API ---------------------------------------------------------
|
||||
|
||||
def push_front(self, node: _DLLNode) -> None:
|
||||
"""Insert *node* right after the head sentinel (most-recent)."""
|
||||
self._insert_after(node, self._head)
|
||||
self.size += 1
|
||||
|
||||
def pop_tail(self) -> Optional[_DLLNode]:
|
||||
"""Remove and return the node just before the tail sentinel (LRU)."""
|
||||
if self.size == 0:
|
||||
return None
|
||||
node = self._tail.prev
|
||||
self._unlink(node)
|
||||
return node
|
||||
|
||||
def remove(self, node: _DLLNode) -> None:
|
||||
"""Remove an arbitrary node from the list."""
|
||||
self._unlink(node)
|
||||
|
||||
def is_empty(self) -> bool:
|
||||
return self.size == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cache Node (wraps the DLL node + TTL metadata)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class _CacheNode:
|
||||
key: Any
|
||||
value: Any
|
||||
ttl_seconds: float
|
||||
expiry_time: float
|
||||
freq: int = 1
|
||||
dll_node: Optional[_DLLNode] = field(default=None, repr=False)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Transaction Handle
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Transaction:
|
||||
"""
|
||||
Represents an isolated sub-session on the cache.
|
||||
|
||||
- ``tx.put(key, value, ttl)`` buffers a write locally.
|
||||
- ``tx.get(key)`` reads from local buffer first, then falls back to the
|
||||
global cache (without mutating global frequency state).
|
||||
- ``tx.delete(key)`` buffers a deletion locally.
|
||||
- ``await tx.commit()`` atomically applies all buffered writes to the global cache.
|
||||
- ``tx.rollback()`` discards everything.
|
||||
"""
|
||||
|
||||
def __init__(self, cache: "LFUCache") -> None:
|
||||
self._cache = cache
|
||||
self._pending_writes: dict[Any, tuple[Any, float]] = {} # key -> (value, expiry)
|
||||
self._pending_deletes: set[Any] = set()
|
||||
self._committed = False
|
||||
|
||||
# -- read ----------------------------------------------------------------
|
||||
|
||||
def get(self, key: Any) -> Optional[Any]:
|
||||
"""Read with read-your-own-writes semantics."""
|
||||
if self._committed:
|
||||
raise RuntimeError("Transaction already committed")
|
||||
|
||||
# 1. Check local pending writes first
|
||||
if key in self._pending_writes:
|
||||
return self._pending_writes[key][0]
|
||||
if key in self._pending_deletes:
|
||||
return None
|
||||
|
||||
# 2. Fall back to global cache (read-only, no frequency bump)
|
||||
return self._cache._get_raw(key)
|
||||
|
||||
# -- write ---------------------------------------------------------------
|
||||
|
||||
def put(self, key: Any, value: Any, ttl_seconds: float = 60.0) -> None:
|
||||
"""Buffer a write locally; not visible to others until commit."""
|
||||
if self._committed:
|
||||
raise RuntimeError("Transaction already committed")
|
||||
expiry = time.monotonic() + ttl_seconds
|
||||
self._pending_writes[key] = (value, expiry)
|
||||
# If previously deleted in this txn, re-add overrides the delete.
|
||||
self._pending_deletes.discard(key)
|
||||
|
||||
def delete(self, key: Any) -> None:
|
||||
"""Buffer a deletion locally."""
|
||||
if self._committed:
|
||||
raise RuntimeError("Transaction already committed")
|
||||
self._pending_deletes.add(key)
|
||||
self._pending_writes.pop(key, None)
|
||||
|
||||
# -- commit / rollback ---------------------------------------------------
|
||||
|
||||
async def commit(self) -> None:
|
||||
"""Atomically apply all buffered changes to the global cache."""
|
||||
if self._committed:
|
||||
raise RuntimeError("Transaction already committed")
|
||||
async with self._cache._lock:
|
||||
for key, (value, expiry) in self._pending_writes.items():
|
||||
await self._cache._put_internal(key, value, expiry)
|
||||
for key in self._pending_deletes:
|
||||
await self._cache._delete_internal(key)
|
||||
self._committed = True
|
||||
|
||||
def rollback(self) -> None:
|
||||
"""Discard all pending changes."""
|
||||
self._pending_writes.clear()
|
||||
self._pending_deletes.clear()
|
||||
self._committed = True # mark so further ops raise
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# LFU Cache
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class LFUCache:
|
||||
"""
|
||||
In-memory concurrent LFU cache with O(1) get/put, async TTL eviction,
|
||||
and atomic transaction support.
|
||||
"""
|
||||
|
||||
def __init__(self, capacity: int = 1024) -> None:
|
||||
if capacity < 1:
|
||||
raise ValueError("capacity must be >= 1")
|
||||
|
||||
self._capacity = capacity
|
||||
self._lock = asyncio.Lock()
|
||||
self._evictor_task: Optional[asyncio.Task[None]] = None
|
||||
|
||||
# Core O(1) structures
|
||||
self._cache_map: dict[Any, _CacheNode] = {} # key -> CacheNode
|
||||
self._freq_map: dict[int, _DoublyLinkedList] = {} # freq -> DLL
|
||||
self._min_freq: int = 1
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Public API
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def get(self, key: Any) -> Optional[Any]:
|
||||
"""O(1) lookup with lazy TTL eviction."""
|
||||
async with self._lock:
|
||||
node = self._cache_map.get(key)
|
||||
if node is None:
|
||||
return None
|
||||
# Lazy TTL check
|
||||
if time.monotonic() > node.expiry_time:
|
||||
await self._evict_node(key, node)
|
||||
return None
|
||||
# Bump frequency — O(1)
|
||||
await self._bump_freq(node)
|
||||
return node.value
|
||||
|
||||
async def put(self, key: Any, value: Any, ttl_seconds: float = 60.0) -> None:
|
||||
"""O(1) insert/update with lazy TTL eviction of LRU-LFU victim if needed."""
|
||||
expiry = time.monotonic() + ttl_seconds
|
||||
async with self._lock:
|
||||
await self._put_internal(key, value, expiry)
|
||||
|
||||
async def delete(self, key: Any) -> bool:
|
||||
"""O(1) deletion."""
|
||||
async with self._lock:
|
||||
return await self._delete_internal(key)
|
||||
|
||||
def begin_transaction(self) -> Transaction:
|
||||
"""Start a new isolated transaction."""
|
||||
return Transaction(self)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# TTL Eviction Loop
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def start_evictor(self, interval_seconds: float = 1.0) -> None:
|
||||
"""Start the background async TTL sweep task."""
|
||||
if self._evictor_task is not None and not self._evictor_task.done():
|
||||
return
|
||||
self._evictor_task = asyncio.create_task(self._eviction_loop(interval_seconds))
|
||||
|
||||
def stop_evictor(self) -> None:
|
||||
"""Stop the background eviction task."""
|
||||
if self._evictor_task is not None:
|
||||
self._evictor_task.cancel()
|
||||
self._evictor_task = None
|
||||
|
||||
async def _eviction_loop(self, interval: float) -> None:
|
||||
"""Periodically purge expired entries in small batches."""
|
||||
try:
|
||||
while True:
|
||||
await asyncio.sleep(interval)
|
||||
async with self._lock:
|
||||
now = time.monotonic()
|
||||
# Collect expired keys in a snapshot to avoid dict-changed-size
|
||||
expired = [
|
||||
k for k, n in self._cache_map.items()
|
||||
if now > n.expiry_time
|
||||
]
|
||||
for k in expired:
|
||||
node = self._cache_map.get(k)
|
||||
if node is not None and now > node.expiry_time:
|
||||
await self._evict_node(k, node)
|
||||
except asyncio.CancelledError:
|
||||
return
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _get_raw(self, key: Any) -> Optional[Any]:
|
||||
"""Read-only global lookup — does NOT bump frequency. For transactions."""
|
||||
node = self._cache_map.get(key)
|
||||
if node is None:
|
||||
return None
|
||||
if time.monotonic() > node.expiry_time:
|
||||
await self._evict_node(key, node)
|
||||
return None
|
||||
return node.value
|
||||
|
||||
async def _put_internal(self, key: Any, value: Any, expiry: float) -> None:
|
||||
"""Core put logic (must be called under lock)."""
|
||||
# If key already exists, update in place
|
||||
if key in self._cache_map:
|
||||
node = self._cache_map[key]
|
||||
# Remove from old freq list, update value/ttl/freq
|
||||
self._freq_map[node.freq].remove(node.dll_node)
|
||||
if self._freq_map[node.freq].is_empty():
|
||||
del self._freq_map[node.freq]
|
||||
if node.freq == self._min_freq:
|
||||
self._min_freq += 1
|
||||
node.value = value
|
||||
node.ttl_seconds = expiry - time.monotonic()
|
||||
node.expiry_time = expiry
|
||||
node.freq = 1
|
||||
self._ensure_freq_list(1).push_front(node.dll_node)
|
||||
return
|
||||
|
||||
# Evict if at capacity
|
||||
if len(self._cache_map) >= self._capacity:
|
||||
await self._evict_lfu()
|
||||
|
||||
# Insert new node
|
||||
dll_node = _DLLNode(key, value, expiry - time.monotonic(), expiry)
|
||||
cache_node = _CacheNode(key, value, expiry - time.monotonic(), expiry, freq=1, dll_node=dll_node)
|
||||
self._cache_map[key] = cache_node
|
||||
self._freq_map[1].push_front(dll_node)
|
||||
self._min_freq = 1
|
||||
|
||||
async def _delete_internal(self, key: Any) -> bool:
|
||||
"""Core delete logic (must be called under lock)."""
|
||||
node = self._cache_map.get(key)
|
||||
if node is None:
|
||||
return False
|
||||
await self._evict_node(key, node)
|
||||
return True
|
||||
|
||||
async def _evict_node(self, key: Any, node: _CacheNode) -> None:
|
||||
"""Remove a single node from all structures."""
|
||||
dll = self._freq_map.get(node.freq)
|
||||
if dll is not None:
|
||||
dll.remove(node.dll_node)
|
||||
if dll.is_empty():
|
||||
del self._freq_map[node.freq]
|
||||
if node.freq == self._min_freq:
|
||||
# Find new min freq
|
||||
new_min = min(self._freq_map) if self._freq_map else 1
|
||||
self._min_freq = new_min
|
||||
del self._cache_map[key]
|
||||
|
||||
async def _evict_lfu(self) -> None:
|
||||
"""Evict the least-frequently-used (and least-recently-used within that freq) node."""
|
||||
if not self._freq_map:
|
||||
return
|
||||
dll = self._freq_map.get(self._min_freq)
|
||||
if dll is None or dll.is_empty():
|
||||
# Shouldn't happen, but safeguard
|
||||
self._min_freq += 1
|
||||
await self._evict_lfu()
|
||||
return
|
||||
victim = dll.pop_tail()
|
||||
if victim is not None:
|
||||
await self._evict_node(victim.key, self._cache_map[victim.key])
|
||||
|
||||
async def _bump_freq(self, node: _CacheNode) -> None:
|
||||
"""Move a node from freq f to freq f+1. O(1)."""
|
||||
old_freq = node.freq
|
||||
old_dll = self._freq_map[old_freq]
|
||||
old_dll.remove(node.dll_node)
|
||||
if old_dll.is_empty():
|
||||
del self._freq_map[old_freq]
|
||||
if old_freq == self._min_freq:
|
||||
self._min_freq += 1
|
||||
|
||||
node.freq += 1
|
||||
new_dll = self._ensure_freq_list(node.freq)
|
||||
new_dll.push_front(node.dll_node)
|
||||
|
||||
def _ensure_freq_list(self, freq: int) -> _DoublyLinkedList:
|
||||
"""Return (or create) the DLL for *freq*."""
|
||||
if freq not in self._freq_map:
|
||||
self._freq_map[freq] = _DoublyLinkedList()
|
||||
return self._freq_map[freq]
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Diagnostics
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@property
|
||||
def size(self) -> int:
|
||||
return len(self._cache_map)
|
||||
|
||||
@property
|
||||
def capacity(self) -> int:
|
||||
return self._capacity
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Executable Test Suite
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def main() -> None:
|
||||
passed = 0
|
||||
failed = 0
|
||||
|
||||
def _check(name: str, condition: bool) -> None:
|
||||
nonlocal passed, failed
|
||||
if condition:
|
||||
passed += 1
|
||||
print(f" ✓ {name}")
|
||||
else:
|
||||
failed += 1
|
||||
print(f" ✗ {name}")
|
||||
|
||||
# ======================================================================
|
||||
# (a) O(1) LFU eviction order
|
||||
# ======================================================================
|
||||
print("\n=== (a) LFU Eviction Order ===")
|
||||
|
||||
cache = LFUCache(capacity=3)
|
||||
|
||||
# Insert 3 items
|
||||
await cache.put("a", 1, ttl_seconds=60.0)
|
||||
await cache.put("b", 2, ttl_seconds=60.0)
|
||||
await cache.put("c", 3, ttl_seconds=60.0)
|
||||
|
||||
# Access "a" and "b" once each → freq=2; "c" stays at freq=1
|
||||
await cache.get("a")
|
||||
await cache.get("b")
|
||||
|
||||
# Insert "d" — should evict "c" (lowest freq)
|
||||
await cache.put("d", 4, ttl_seconds=60.0)
|
||||
|
||||
_check("a still present after eviction", await cache.get("a") == 1)
|
||||
_check("b still present after eviction", await cache.get("b") == 2)
|
||||
_check("c evicted (lowest freq)", await cache.get("c") is None)
|
||||
_check("d present", await cache.get("d") == 4)
|
||||
|
||||
# Now access "a" again → freq=3; "b" and "d" at freq=2
|
||||
await cache.get("a")
|
||||
# Insert "e" — should evict either "b" or "d" (both freq=2, LRU wins)
|
||||
await cache.put("e", 5, ttl_seconds=60.0)
|
||||
|
||||
_check("a still present", await cache.get("a") == 1)
|
||||
_check("e present", await cache.get("e") == 5)
|
||||
|
||||
# ======================================================================
|
||||
# (b) Lazy TTL vs Background Async Sweep
|
||||
# ======================================================================
|
||||
print("\n=== (b) TTL Eviction (Lazy + Background) ===")
|
||||
|
||||
cache2 = LFUCache(capacity=10)
|
||||
|
||||
await cache2.put("lazy_key", "lazy_val", ttl_seconds=0.1)
|
||||
await cache2.put("bg_key", "bg_val", ttl_seconds=0.1)
|
||||
|
||||
# Lazy eviction: access lazy_key after expiry
|
||||
await asyncio.sleep(0.15)
|
||||
_check("Lazy eviction: get returns None after TTL", await cache2.get("lazy_key") is None)
|
||||
_check("Lazy eviction: bg_key still there (not accessed)", await cache2.get("bg_key") == "bg_val")
|
||||
|
||||
# Start background evictor
|
||||
cache2.start_evictor(interval_seconds=0.2)
|
||||
await asyncio.sleep(0.3) # let background sweep run
|
||||
|
||||
_check("Background eviction: bg_key purged by sweep", await cache2.get("bg_key") is None)
|
||||
cache2.stop_evictor()
|
||||
|
||||
# ======================================================================
|
||||
# (c) Transaction commit visibility vs rollback
|
||||
# ======================================================================
|
||||
print("\n=== (c) Atomic Transactions ===")
|
||||
|
||||
cache3 = LFUCache(capacity=10)
|
||||
await cache3.put("x", 10, ttl_seconds=60.0)
|
||||
await cache3.put("y", 20, ttl_seconds=60.0)
|
||||
|
||||
# --- Commit test ---
|
||||
tx1 = cache3.begin_transaction()
|
||||
tx1.put("x", 99, ttl_seconds=60.0) # local write
|
||||
tx1.put("z", 30, ttl_seconds=60.0) # new key
|
||||
|
||||
_check("TX: read-your-own-write (x)", tx1.get("x") == 99)
|
||||
_check("TX: read-your-own-write (z)", tx1.get("z") == 30)
|
||||
_check("TX: global still sees old x", await cache3.get("x") == 10)
|
||||
|
||||
await tx1.commit()
|
||||
_check("TX: after commit, global sees x=99", await cache3.get("x") == 99)
|
||||
_check("TX: after commit, global sees z=30", await cache3.get("z") == 30)
|
||||
|
||||
# --- Rollback test ---
|
||||
tx2 = cache3.begin_transaction()
|
||||
tx2.put("x", -1, ttl_seconds=60.0)
|
||||
tx2.delete("y")
|
||||
_check("TX rollback: local sees x=-1", tx2.get("x") == -1)
|
||||
_check("TX rollback: local sees y deleted", tx2.get("y") is None)
|
||||
_check("TX rollback: global still sees x=99", await cache3.get("x") == 99)
|
||||
_check("TX rollback: global still sees y=20", await cache3.get("y") == 20)
|
||||
|
||||
tx2.rollback()
|
||||
_check("TX rollback: global unchanged after rollback", await cache3.get("x") == 99)
|
||||
_check("TX rollback: y still present after rollback", await cache3.get("y") == 20)
|
||||
|
||||
# --- Double commit / rollback raises ---
|
||||
tx3 = cache3.begin_transaction()
|
||||
await tx3.commit()
|
||||
try:
|
||||
tx3.put("x", 1)
|
||||
_check("TX: double commit raises", False)
|
||||
except RuntimeError:
|
||||
_check("TX: double commit raises", True)
|
||||
|
||||
tx4 = cache3.begin_transaction()
|
||||
tx4.rollback()
|
||||
try:
|
||||
tx4.put("x", 1)
|
||||
_check("TX: op after rollback raises", False)
|
||||
except RuntimeError:
|
||||
_check("TX: op after rollback raises", True)
|
||||
|
||||
# ======================================================================
|
||||
# (d) Stress test: 50 concurrent async tasks
|
||||
# ======================================================================
|
||||
print("\n=== (d) Stress Test — 50 Concurrent Tasks ===")
|
||||
|
||||
cache4 = LFUCache(capacity=200)
|
||||
errors: list[str] = []
|
||||
|
||||
async def worker(task_id: int, base_key: int) -> None:
|
||||
try:
|
||||
for i in range(50):
|
||||
key = f"t{task_id}_k{i}"
|
||||
val = task_id * 1000 + i
|
||||
await cache4.put(key, val, ttl_seconds=5.0)
|
||||
result = await cache4.get(key)
|
||||
if result != val:
|
||||
errors.append(f"task={task_id} key={key} expected={val} got={result}")
|
||||
# Occasional transaction
|
||||
if i % 10 == 0:
|
||||
tx = cache4.begin_transaction()
|
||||
tx.put(f"tx_{task_id}_{i}", val * 2, ttl_seconds=5.0)
|
||||
r = tx.get(f"tx_{task_id}_{i}")
|
||||
if r != val * 2:
|
||||
errors.append(f"task={task_id} tx key mismatch")
|
||||
await tx.commit()
|
||||
except Exception as e:
|
||||
errors.append(f"task={task_id} exception: {e}")
|
||||
|
||||
tasks = [asyncio.create_task(worker(tid, tid)) for tid in range(50)]
|
||||
await asyncio.gather(*tasks)
|
||||
|
||||
_check("Stress: no errors", len(errors) == 0)
|
||||
_check("Stress: cache size within capacity", cache4.size <= 200)
|
||||
if errors:
|
||||
for e in errors[:5]:
|
||||
print(f" error: {e}")
|
||||
|
||||
# ======================================================================
|
||||
# Summary
|
||||
# ======================================================================
|
||||
total = passed + failed
|
||||
print(f"\n{'='*50}")
|
||||
print(f"Results: {passed}/{total} passed, {failed} failed")
|
||||
if failed == 0:
|
||||
print("All tests passed! ✓")
|
||||
else:
|
||||
print(f"{failed} test(s) FAILED ✗")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
@@ -0,0 +1,365 @@
|
||||
"""
|
||||
In-Memory Concurrent LFU Cache with Async TTL Eviction and Atomic Transactions.
|
||||
Pure Python 3.11+ implementation using only standard library modules.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
from asyncio import Lock
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal Data Structures (O(1) LFU Core)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _Node:
|
||||
"""Doubly linked list node tracking key, value, frequency, and TTL."""
|
||||
__slots__ = ('key', 'value', 'freq', 'expires_at', 'prev', 'next')
|
||||
def __init__(self, key: Any, value: Any, freq: int = 1, expires_at: float = 0.0):
|
||||
self.key = key
|
||||
self.value = value
|
||||
self.freq = freq
|
||||
self.expires_at = expires_at
|
||||
self.prev = None
|
||||
self.next = None
|
||||
|
||||
class _FreqList:
|
||||
"""Doubly linked list maintaining nodes of a specific frequency tier."""
|
||||
__slots__ = ('head', 'tail', 'size')
|
||||
def __init__(self):
|
||||
self.head = _Node(None, None) # Dummy head
|
||||
self.tail = _Node(None, None) # Dummy tail
|
||||
self.head.next = self.tail
|
||||
self.tail.prev = self.head
|
||||
self.size = 0
|
||||
|
||||
def add(self, node: _Node):
|
||||
"""Add node to tail (most recently used in this frequency)."""
|
||||
last = self.tail.prev
|
||||
last.next = node
|
||||
node.prev = last
|
||||
node.next = self.tail
|
||||
self.tail.prev = node
|
||||
self.size += 1
|
||||
|
||||
def remove(self, node: _Node):
|
||||
"""Remove node from the list in O(1)."""
|
||||
node.prev.next = node.next
|
||||
node.next.prev = node.prev
|
||||
node.prev = None
|
||||
node.next = None
|
||||
self.size -= 1
|
||||
|
||||
def pop(self) -> _Node:
|
||||
"""Remove and return node from head (least recently used)."""
|
||||
node = self.head.next
|
||||
self.remove(node)
|
||||
return node
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main Cache Implementation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class LFUCache:
|
||||
"""
|
||||
O(1) In-Memory Concurrent LFU Cache with Async TTL Eviction.
|
||||
Uses a hash map for key lookup and a hash map of frequency-buckets
|
||||
with doubly linked lists for O(1) frequency updates and eviction.
|
||||
"""
|
||||
def __init__(self, capacity: int, evict_interval: float = 1.0):
|
||||
self.capacity = capacity
|
||||
self.evict_interval = evict_interval
|
||||
self.key_to_node: Dict[Any, _Node] = {}
|
||||
self.freq_to_list: Dict[int, _FreqList] = {}
|
||||
self.min_freq: int = 1
|
||||
self.lock: Lock = asyncio.Lock()
|
||||
self._evict_task: Optional[asyncio.Task] = None
|
||||
self._running = False
|
||||
|
||||
async def start_evictor(self):
|
||||
"""Start the non-blocking background TTL eviction loop."""
|
||||
self._running = True
|
||||
self._evict_task = asyncio.create_task(self._eviction_loop())
|
||||
|
||||
async def stop_evictor(self):
|
||||
"""Gracefully stop the background eviction task."""
|
||||
self._running = False
|
||||
if self._evict_task:
|
||||
self._evict_task.cancel()
|
||||
try: await self._evict_task
|
||||
except asyncio.CancelledError: pass
|
||||
|
||||
async def get(self, key: Any) -> Optional[Any]:
|
||||
"""Retrieve value in O(1). Performs lazy TTL eviction and frequency update."""
|
||||
async with self.lock:
|
||||
node = self.key_to_node.get(key)
|
||||
if not node:
|
||||
return None
|
||||
|
||||
# Lazy TTL Eviction
|
||||
if node.expires_at > 0 and node.expires_at <= time.time():
|
||||
self._remove_node(node)
|
||||
return None
|
||||
|
||||
# O(1) Frequency Update
|
||||
self._update_freq(node)
|
||||
return node.value
|
||||
|
||||
async def put(self, key: Any, value: Any, ttl_seconds: float = 0.0):
|
||||
"""Insert/update value in O(1). Handles capacity eviction and TTL."""
|
||||
async with self.lock:
|
||||
node = self.key_to_node.get(key)
|
||||
if node:
|
||||
# Update existing node
|
||||
node.value = value
|
||||
if ttl_seconds > 0:
|
||||
node.expires_at = time.time() + ttl_seconds
|
||||
self._update_freq(node)
|
||||
else:
|
||||
# Evict if capacity reached
|
||||
if len(self.key_to_node) >= self.capacity:
|
||||
self._evict()
|
||||
# Insert new node with freq=1
|
||||
node = _Node(
|
||||
key, value, freq=1,
|
||||
expires_at=float('inf') if ttl_seconds <= 0 else time.time() + ttl_seconds
|
||||
)
|
||||
self.key_to_node[key] = node
|
||||
self._add_to_freq(node)
|
||||
self.min_freq = 1 # New keys always start at freq 1
|
||||
|
||||
async def begin_transaction(self) -> 'Transaction':
|
||||
"""Create an isolated transaction session."""
|
||||
return Transaction(self)
|
||||
|
||||
# --- Internal Cache Helpers (O(1)) ---
|
||||
|
||||
def _remove_node(self, node: _Node):
|
||||
"""Remove node from global structures."""
|
||||
del self.key_to_node[node.key]
|
||||
self.freq_to_list[node.freq].remove(node)
|
||||
if self.freq_to_list[node.freq].size == 0:
|
||||
del self.freq_to_list[node.freq]
|
||||
if self.min_freq == node.freq:
|
||||
self.min_freq += 1
|
||||
|
||||
def _update_freq(self, node: _Node):
|
||||
"""Move node to next frequency bucket in O(1)."""
|
||||
old_freq = node.freq
|
||||
self.freq_to_list[old_freq].remove(node)
|
||||
if self.freq_to_list[old_freq].size == 0:
|
||||
del self.freq_to_list[old_freq]
|
||||
if self.min_freq == old_freq:
|
||||
self.min_freq += 1
|
||||
|
||||
node.freq += 1
|
||||
self._add_to_freq(node)
|
||||
|
||||
def _add_to_freq(self, node: _Node):
|
||||
"""Add node to its frequency bucket."""
|
||||
if node.freq not in self.freq_to_list:
|
||||
self.freq_to_list[node.freq] = _FreqList()
|
||||
self.freq_to_list[node.freq].add(node)
|
||||
|
||||
def _evict(self):
|
||||
"""Evict least frequently used (then LRU) node in O(1)."""
|
||||
if not self.freq_to_list:
|
||||
return
|
||||
evict_list = self.freq_to_list[self.min_freq]
|
||||
node = evict_list.pop()
|
||||
self._remove_node(node)
|
||||
|
||||
async def _eviction_loop(self):
|
||||
"""Background async task that purges expired keys in small batches."""
|
||||
while self._running:
|
||||
await asyncio.sleep(self.evict_interval)
|
||||
async with self.lock:
|
||||
# Batch scan limited to avoid blocking reads
|
||||
keys_to_check = list(self.key_to_node.keys())[:50]
|
||||
for k in keys_to_check:
|
||||
node = self.key_to_node.get(k)
|
||||
if node and node.expires_at > 0 and node.expires_at <= time.time():
|
||||
self._remove_node(node)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Transaction Isolation Layer
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Transaction:
|
||||
"""
|
||||
Atomic transaction session providing Read-Your-Own-Writes isolation.
|
||||
Global readers remain blind to uncommitted changes until commit().
|
||||
"""
|
||||
def __init__(self, cache: LFUCache):
|
||||
self.cache = cache
|
||||
self.buffer: List[Dict] = []
|
||||
self.committed = False
|
||||
|
||||
async def get(self, key: Any) -> Optional[Any]:
|
||||
"""Read with local isolation. Does not update global frequencies."""
|
||||
# Read-Your-Own-Writes: check buffer first
|
||||
for op in reversed(self.buffer):
|
||||
if op['key'] == key:
|
||||
return None if op['op'] == 'delete' else op['value']
|
||||
|
||||
# Global read (no freq update to preserve isolation)
|
||||
async with self.cache.lock:
|
||||
node = self.cache.key_to_node.get(key)
|
||||
if not node:
|
||||
return None
|
||||
# Lazy TTL check without mutation
|
||||
if node.expires_at > 0 and node.expires_at <= time.time():
|
||||
return None
|
||||
return node.value
|
||||
|
||||
async def put(self, key: Any, value: Any, ttl_seconds: float = 0.0):
|
||||
"""Buffer a put operation without mutating global state."""
|
||||
original = self.cache.key_to_node.get(key)
|
||||
self.buffer.append({'op': 'put', 'key': key, 'value': value, 'ttl': ttl_seconds, 'original': original})
|
||||
|
||||
async def delete(self, key: Any):
|
||||
"""Buffer a delete operation without mutating global state."""
|
||||
original = self.cache.key_to_node.get(key)
|
||||
self.buffer.append({'op': 'delete', 'key': key, 'original': original})
|
||||
|
||||
async def commit(self):
|
||||
"""Apply buffered changes to global cache atomically."""
|
||||
async with self.cache.lock:
|
||||
# Merge buffer: keep only the latest operation per key
|
||||
merged = {}
|
||||
for op in self.buffer:
|
||||
merged[op['key']] = op
|
||||
|
||||
for op in merged.values():
|
||||
if op['op'] == 'put':
|
||||
await self._apply_put(op['key'], op['value'], op['ttl'], self.cache.key_to_node.get(op['key']))
|
||||
elif op['op'] == 'delete':
|
||||
await self._apply_delete(op['key'], self.cache.key_to_node.get(op['key']))
|
||||
|
||||
self.buffer.clear()
|
||||
self.committed = True
|
||||
|
||||
async def rollback(self):
|
||||
"""Discard all pending changes without affecting global frequencies or TTL."""
|
||||
self.buffer.clear()
|
||||
self.committed = False
|
||||
|
||||
async def _apply_put(self, key: Any, value: Any, ttl: float, original_node: Optional[_Node]):
|
||||
"""Apply buffered put to global cache."""
|
||||
if original_node:
|
||||
original_node.value = value
|
||||
if ttl > 0:
|
||||
original_node.expires_at = time.time() + ttl
|
||||
self.cache._update_freq(original_node)
|
||||
else:
|
||||
if len(self.cache.key_to_node) >= self.cache.capacity:
|
||||
self.cache._evict()
|
||||
node = _Node(
|
||||
key, value, freq=1,
|
||||
expires_at=float('inf') if ttl <= 0 else time.time() + ttl
|
||||
)
|
||||
self.cache.key_to_node[key] = node
|
||||
self.cache._add_to_freq(node)
|
||||
self.cache.min_freq = 1
|
||||
|
||||
async def _apply_delete(self, key: Any, original_node: Optional[_Node]):
|
||||
"""Apply buffered delete to global cache."""
|
||||
if original_node:
|
||||
self.cache._remove_node(original_node)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Executable Unit Tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def main():
|
||||
print("🧪 Running LFU Cache Test Suite...\n")
|
||||
await test_lfu_eviction()
|
||||
await test_ttl_eviction()
|
||||
await test_transactions()
|
||||
await test_stress()
|
||||
print("✅ All tests passed successfully!")
|
||||
|
||||
async def test_lfu_eviction():
|
||||
"""Verifies strict O(1) LFU eviction order when capacity is reached."""
|
||||
cache = LFUCache(3)
|
||||
await cache.put(1, 1)
|
||||
await cache.put(2, 2)
|
||||
await cache.put(3, 3)
|
||||
|
||||
# Access 1 -> freq becomes 2
|
||||
assert await cache.get(1) == 1
|
||||
|
||||
# Insert 4 -> evicts key 2 (freq=1, LRU among freq=1)
|
||||
await cache.put(4, 4)
|
||||
|
||||
assert await cache.get(2) is None, "Key 2 should be evicted"
|
||||
assert await cache.get(1) == 1
|
||||
assert await cache.get(3) == 3
|
||||
assert await cache.get(4) == 4
|
||||
print(" ✓ O(1) LFU eviction order verified")
|
||||
|
||||
async def test_ttl_eviction():
|
||||
"""Verifies Lazy vs Background Async Sweep eviction."""
|
||||
cache = LFUCache(10, evict_interval=0.1)
|
||||
await cache.start_evictor()
|
||||
|
||||
# Lazy eviction on access
|
||||
await cache.put('a', 1, ttl_seconds=0.2)
|
||||
assert await cache.get('a') == 1
|
||||
await asyncio.sleep(0.3)
|
||||
assert await cache.get('a') is None, "Lazy eviction failed"
|
||||
|
||||
# Background async sweep
|
||||
await cache.put('b', 2, ttl_seconds=0.1)
|
||||
await asyncio.sleep(0.2) # Wait for background loop
|
||||
assert await cache.get('b') is None, "Background sweep failed"
|
||||
|
||||
await cache.stop_evictor()
|
||||
print(" ✓ Dual-layer TTL eviction (Lazy + Background) verified")
|
||||
|
||||
async def test_transactions():
|
||||
"""Verifies Transaction commit visibility vs rollback state restoration."""
|
||||
cache = LFUCache(5)
|
||||
await cache.put('x', 10)
|
||||
|
||||
# Test 1: Isolation & Read-Your-Own-Writes
|
||||
tx = await cache.begin_transaction()
|
||||
await tx.put('x', 20)
|
||||
assert await tx.get('x') == 20, "Transaction failed Read-Your-Own-Writes"
|
||||
assert await cache.get('x') == 10, "Global cache leaked uncommitted write"
|
||||
|
||||
await tx.commit()
|
||||
assert await cache.get('x') == 20, "Commit failed to apply changes"
|
||||
|
||||
# Test 2: Rollback restores state
|
||||
tx2 = await cache.begin_transaction()
|
||||
await tx2.put('y', 30)
|
||||
await tx2.rollback()
|
||||
assert await cache.get('y') is None, "Rollback failed to discard changes"
|
||||
print(" ✓ Transaction commit/rollback isolation verified")
|
||||
|
||||
async def test_stress():
|
||||
"""Stress test with 50 concurrent async tasks reading/writing simultaneously."""
|
||||
cache = LFUCache(100)
|
||||
await cache.start_evictor()
|
||||
|
||||
tasks = []
|
||||
for i in range(50):
|
||||
async def worker(idx):
|
||||
tx = await cache.begin_transaction()
|
||||
await tx.put(f'key_{idx}', idx)
|
||||
await asyncio.sleep(0.01)
|
||||
val = await tx.get(f'key_{idx}')
|
||||
assert val == idx, f"Transaction read mismatch for key_{idx}"
|
||||
await tx.commit()
|
||||
# Verify global visibility post-commit
|
||||
assert await cache.get(f'key_{idx}') == idx, f"Global read mismatch for key_{idx}"
|
||||
tasks.append(asyncio.create_task(worker(i)))
|
||||
|
||||
await asyncio.gather(*tasks)
|
||||
await cache.stop_evictor()
|
||||
print(" ✓ Stress test (50 concurrent tasks) passed without race conditions")
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.run(main())
|
||||
|
||||
@@ -0,0 +1,426 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
In-Memory Concurrent LFU Cache with Async TTL Eviction and Atomic Transactions.
|
||||
Pure Python 3.11+ implementation. No external dependencies.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Optional, Dict, Set, List
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Core Data Structures for O(1) Operations
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class Node:
|
||||
key: Any
|
||||
value: Any
|
||||
freq: int
|
||||
expires_at: Optional[float]
|
||||
prev: Optional['Node'] = None
|
||||
next: Optional['Node'] = None
|
||||
|
||||
|
||||
class DoublyLinkedList:
|
||||
"""Sentinel-based doubly linked list supporting O(1) add/remove/pop."""
|
||||
def __init__(self):
|
||||
self.head = Node(None, None, 0, None)
|
||||
self.tail = Node(None, None, 0, None)
|
||||
self.head.next = self.tail
|
||||
self.tail.prev = self.head
|
||||
self.size = 0
|
||||
|
||||
def add_to_tail(self, node: Node):
|
||||
node.prev = self.tail.prev
|
||||
node.next = self.tail
|
||||
self.tail.prev.next = node
|
||||
self.tail.prev = node
|
||||
self.size += 1
|
||||
|
||||
def remove_node(self, node: Node):
|
||||
if node.prev is None or node.next is None:
|
||||
return
|
||||
node.prev.next = node.next
|
||||
node.next.prev = node.prev
|
||||
node.prev = node.next = None
|
||||
self.size -= 1
|
||||
|
||||
def pop_head(self) -> Optional[Node]:
|
||||
if self.size == 0:
|
||||
return None
|
||||
node = self.head.next
|
||||
self.remove_node(node)
|
||||
return node
|
||||
|
||||
def is_empty(self) -> bool:
|
||||
return self.size == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Transaction Isolation Layer
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Transaction:
|
||||
"""
|
||||
Provides ACID-like sub-sessions with Read-Your-Own-Writes isolation.
|
||||
Changes remain local until commit(), where they are applied atomically.
|
||||
"""
|
||||
def __init__(self, cache: 'LFUCache'):
|
||||
self._cache = cache
|
||||
self._local_writes: Dict[Any, tuple] = {} # key -> (value, ttl)
|
||||
self._local_deletes: Set[Any] = set()
|
||||
self._local_freq_bumps: Set[Any] = set() # Track accesses for LFU bump
|
||||
self._committed = False
|
||||
self._rolled_back = False
|
||||
|
||||
async def put(self, key: Any, value: Any, ttl_seconds: Optional[float] = None):
|
||||
if self._committed or self._rolled_back:
|
||||
raise RuntimeError("Transaction already finished")
|
||||
self._local_writes[key] = (value, ttl_seconds)
|
||||
self._local_deletes.discard(key)
|
||||
self._local_freq_bumps.add(key)
|
||||
|
||||
async def get(self, key: Any) -> Optional[Any]:
|
||||
if self._committed or self._rolled_back:
|
||||
raise RuntimeError("Transaction already finished")
|
||||
# Read Your Own Writes
|
||||
if key in self._local_writes:
|
||||
val, _ = self._local_writes[key]
|
||||
self._local_freq_bumps.add(key)
|
||||
return val
|
||||
if key in self._local_deletes:
|
||||
return None
|
||||
# Read Global State
|
||||
val = await self._cache.get(key)
|
||||
if val is not None:
|
||||
self._local_freq_bumps.add(key)
|
||||
return val
|
||||
|
||||
async def delete(self, key: Any) -> bool:
|
||||
if self._committed or self._rolled_back:
|
||||
raise RuntimeError("Transaction already finished")
|
||||
self._local_deletes.add(key)
|
||||
self._local_writes.pop(key, None)
|
||||
return True
|
||||
|
||||
async def commit(self) -> bool:
|
||||
if self._committed or self._rolled_back:
|
||||
raise RuntimeError("Transaction already finished")
|
||||
self._committed = True
|
||||
|
||||
async with self._cache._lock:
|
||||
# 1. Apply Writes
|
||||
for key, (value, ttl) in self._local_writes.items():
|
||||
if key in self._cache.store:
|
||||
node = self._cache.store[key]
|
||||
if self._cache._is_expired(node):
|
||||
self._cache._remove_node(key)
|
||||
else:
|
||||
node.value = value
|
||||
node.expires_at = time.time() + ttl if ttl else None
|
||||
self._cache._update_freq(node)
|
||||
else:
|
||||
self._cache._add_node(key, value, ttl)
|
||||
|
||||
# 2. Apply Deletes
|
||||
for key in self._local_deletes:
|
||||
self._cache._remove_node(key)
|
||||
|
||||
# 3. Apply Frequency Bumps (for reads during transaction)
|
||||
for key in self._local_freq_bumps:
|
||||
if key in self._cache.store and key not in self._local_deletes:
|
||||
node = self._cache.store[key]
|
||||
self._cache._update_freq(node)
|
||||
|
||||
self._cache._cleanup_freq_lists()
|
||||
return True
|
||||
|
||||
async def rollback(self):
|
||||
if self._committed or self._rolled_back:
|
||||
raise RuntimeError("Transaction already finished")
|
||||
self._rolled_back = True
|
||||
# Completely discard pending changes without mutating global state
|
||||
self._local_writes.clear()
|
||||
self._local_deletes.clear()
|
||||
self._local_freq_bumps.clear()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main Cache Implementation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class LFUCache:
|
||||
"""
|
||||
Concurrent LFU Cache with O(1) get/put, dual-layer TTL eviction,
|
||||
and atomic transaction support.
|
||||
"""
|
||||
def __init__(self, capacity: int, eviction_interval: float = 0.5):
|
||||
if capacity <= 0:
|
||||
raise ValueError("Cache capacity must be positive")
|
||||
self.capacity = capacity
|
||||
self.store: Dict[Any, Node] = {}
|
||||
self.freq_map: Dict[int, DoublyLinkedList] = {}
|
||||
self.min_freq: int = 0
|
||||
self.eviction_interval = eviction_interval
|
||||
self._lock = asyncio.Lock()
|
||||
self._evictor_task: Optional[asyncio.Task] = None
|
||||
self._evictor_running = False
|
||||
self._ttl_keys: Set[Any] = set()
|
||||
|
||||
def begin_transaction(self) -> Transaction:
|
||||
return Transaction(self)
|
||||
|
||||
async def start_evictor(self):
|
||||
"""Starts the background async TTL eviction loop."""
|
||||
if self._evictor_running:
|
||||
return
|
||||
self._evictor_running = True
|
||||
self._evictor_task = asyncio.create_task(self._eviction_loop())
|
||||
|
||||
async def stop_evictor(self):
|
||||
"""Stops the background async TTL eviction loop."""
|
||||
self._evictor_running = False
|
||||
if self._evictor_task:
|
||||
self._evictor_task.cancel()
|
||||
try:
|
||||
await self._evictor_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._evictor_task = None
|
||||
|
||||
async def _eviction_loop(self):
|
||||
while self._evictor_running:
|
||||
await asyncio.sleep(self.eviction_interval)
|
||||
if not self._evictor_running:
|
||||
break
|
||||
await self._evict_batch()
|
||||
|
||||
async def _evict_batch(self):
|
||||
"""Background sweep: acquires lock briefly, removes expired keys in batches."""
|
||||
async with self._lock:
|
||||
expired_keys = [
|
||||
k for k in list(self._ttl_keys)
|
||||
if k in self.store and self.store[k].expires_at and time.time() > self.store[k].expires_at
|
||||
]
|
||||
batch = expired_keys[:100] # Small batch to avoid lock contention
|
||||
for k in batch:
|
||||
self._remove_node(k)
|
||||
self._ttl_keys.discard(k)
|
||||
self._cleanup_freq_lists()
|
||||
|
||||
def _cleanup_freq_lists(self):
|
||||
"""Remove empty frequency buckets and update min_freq."""
|
||||
empty_freqs = [f for f, dll in self.freq_map.items() if dll.is_empty()]
|
||||
for f in empty_freqs:
|
||||
del self.freq_map[f]
|
||||
if self.freq_map:
|
||||
self.min_freq = min(self.freq_map.keys())
|
||||
else:
|
||||
self.min_freq = 0
|
||||
|
||||
def _is_expired(self, node: Node) -> bool:
|
||||
return node.expires_at is not None and time.time() > node.expires_at
|
||||
|
||||
def _remove_node(self, key: Any):
|
||||
"""Removes a key from store and its frequency linked list."""
|
||||
if key not in self.store:
|
||||
return
|
||||
node = self.store.pop(key)
|
||||
self._ttl_keys.discard(key)
|
||||
freq = node.freq
|
||||
if freq in self.freq_map:
|
||||
self.freq_map[freq].remove_node(node)
|
||||
|
||||
def _update_freq(self, node: Node):
|
||||
"""Moves node to next frequency bucket in O(1)."""
|
||||
freq = node.freq
|
||||
self.freq_map[freq].remove_node(node)
|
||||
if self.freq_map[freq].is_empty():
|
||||
del self.freq_map[freq]
|
||||
if self.min_freq == freq:
|
||||
self.min_freq += 1
|
||||
node.freq += 1
|
||||
new_freq = node.freq
|
||||
if new_freq not in self.freq_map:
|
||||
self.freq_map[new_freq] = DoublyLinkedList()
|
||||
self.freq_map[new_freq].add_to_tail(node)
|
||||
|
||||
async def get(self, key: Any) -> Optional[Any]:
|
||||
"""O(1) get with lazy TTL eviction and frequency bump."""
|
||||
async with self._lock:
|
||||
if key not in self.store:
|
||||
return None
|
||||
node = self.store[key]
|
||||
if self._is_expired(node):
|
||||
self._remove_node(key)
|
||||
return None
|
||||
self._update_freq(node)
|
||||
return node.value
|
||||
|
||||
async def put(self, key: Any, value: Any, ttl_seconds: Optional[float] = None):
|
||||
"""O(1) put with capacity eviction and TTL handling."""
|
||||
async with self._lock:
|
||||
if key in self.store:
|
||||
node = self.store[key]
|
||||
if self._is_expired(node):
|
||||
self._remove_node(key)
|
||||
else:
|
||||
node.value = value
|
||||
node.expires_at = time.time() + ttl_seconds if ttl_seconds else None
|
||||
self._update_freq(node)
|
||||
return
|
||||
if len(self.store) >= self.capacity:
|
||||
self._evict()
|
||||
self._add_node(key, value, ttl_seconds)
|
||||
|
||||
def _evict(self):
|
||||
"""Evicts the least frequently used item (O(1) via min_freq bucket)."""
|
||||
if self.min_freq in self.freq_map:
|
||||
dll = self.freq_map[self.min_freq]
|
||||
node = dll.pop_head()
|
||||
if node:
|
||||
self.store.pop(node.key, None)
|
||||
self._ttl_keys.discard(node.key)
|
||||
self._cleanup_freq_lists()
|
||||
|
||||
def _add_node(self, key: Any, value: Any, ttl_seconds: Optional[float]):
|
||||
"""Inserts new node into freq 1 bucket."""
|
||||
node = Node(key, value, 1, time.time() + ttl_seconds if ttl_seconds else None)
|
||||
self.store[key] = node
|
||||
if ttl_seconds:
|
||||
self._ttl_keys.add(key)
|
||||
if 1 not in self.freq_map:
|
||||
self.freq_map[1] = DoublyLinkedList()
|
||||
self.min_freq = 1
|
||||
self.freq_map[1].add_to_tail(node)
|
||||
|
||||
async def delete(self, key: Any) -> bool:
|
||||
async with self._lock:
|
||||
if key in self.store:
|
||||
self._remove_node(key)
|
||||
return True
|
||||
return False
|
||||
|
||||
async def size(self) -> int:
|
||||
async with self._lock:
|
||||
return len(self.store)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Executable Unit Tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def test_lfu_eviction_order():
|
||||
"""a) O(1) LFU eviction order when capacity is reached."""
|
||||
cache = LFUCache(3)
|
||||
await cache.put(1, 'a')
|
||||
await cache.put(2, 'b')
|
||||
await cache.put(3, 'c')
|
||||
|
||||
# Access 1 twice to make it most frequent
|
||||
await cache.get(1)
|
||||
await cache.get(1)
|
||||
|
||||
# Insert 4 -> should evict 2 (freq 1, least used)
|
||||
await cache.put(4, 'd')
|
||||
|
||||
assert await cache.get(2) is None, "Failed: Least frequent key should be evicted"
|
||||
assert await cache.get(1) == 'a', "Failed: Key 1 should still exist"
|
||||
assert await cache.get(3) == 'c', "Failed: Key 3 should still exist"
|
||||
assert await cache.get(4) == 'd', "Failed: Key 4 should exist"
|
||||
print("✅ Test a) LFU eviction order: PASSED")
|
||||
|
||||
|
||||
async def test_ttl_eviction():
|
||||
"""b) Lazy TTL vs. Background Async Sweep eviction."""
|
||||
cache = LFUCache(10)
|
||||
await cache.start_evictor()
|
||||
|
||||
# Lazy eviction
|
||||
await cache.put('lazy', 'val', ttl_seconds=0.1)
|
||||
assert await cache.get('lazy') == 'val', "Failed: Should return value before TTL"
|
||||
await asyncio.sleep(0.15)
|
||||
assert await cache.get('lazy') is None, "Failed: Lazy eviction should trigger on next get"
|
||||
|
||||
# Background sweep eviction
|
||||
await cache.put('bg', 'val2', ttl_seconds=0.1)
|
||||
await asyncio.sleep(0.25) # Wait for background task cycle
|
||||
assert await cache.get('bg') is None, "Failed: Background sweep should evict expired key"
|
||||
|
||||
await cache.stop_evictor()
|
||||
print("✅ Test b) TTL eviction (Lazy & Background): PASSED")
|
||||
|
||||
|
||||
async def test_transaction_isolation():
|
||||
"""c) Transaction commit visibility vs. rollback state restoration."""
|
||||
cache = LFUCache(10)
|
||||
await cache.put('a', 1)
|
||||
|
||||
# Commit test
|
||||
tx = cache.begin_transaction()
|
||||
await tx.put('a', 2)
|
||||
await tx.put('b', 3)
|
||||
|
||||
# Read your own writes
|
||||
assert await tx.get('a') == 2, "Failed: Should see local write"
|
||||
assert await tx.get('b') == 3, "Failed: Should see local write"
|
||||
|
||||
# Global shouldn't see uncommitted changes
|
||||
assert await cache.get('a') == 1, "Failed: Global should not see uncommitted write"
|
||||
|
||||
await tx.commit()
|
||||
assert await cache.get('a') == 2, "Failed: Global should see committed write"
|
||||
assert await cache.get('b') == 3, "Failed: Global should see committed write"
|
||||
|
||||
# Rollback test
|
||||
tx2 = cache.begin_transaction()
|
||||
await tx2.put('c', 4)
|
||||
assert await tx2.get('c') == 4, "Failed: Should see local write"
|
||||
await tx2.rollback()
|
||||
assert await cache.get('c') is None, "Failed: Rollback should discard changes"
|
||||
print("✅ Test c) Transaction isolation (Commit & Rollback): PASSED")
|
||||
|
||||
|
||||
async def test_stress_concurrency():
|
||||
"""d) Stress test with 50 concurrent async tasks reading/writing simultaneously."""
|
||||
cache = LFUCache(100)
|
||||
await cache.start_evictor()
|
||||
|
||||
tasks = []
|
||||
for i in range(50):
|
||||
async def worker(idx: int):
|
||||
for j in range(10):
|
||||
key = f"k{idx}"
|
||||
await cache.put(key, f"v{idx}_{j}")
|
||||
val = await cache.get(key)
|
||||
assert val == f"v{idx}_{j}", f"Failed: Read mismatch for {key}"
|
||||
tasks.append(asyncio.create_task(worker(i)))
|
||||
|
||||
await asyncio.gather(*tasks)
|
||||
|
||||
# All 50 keys should exist
|
||||
assert await cache.size() == 50, f"Failed: Expected 50 keys, got {await cache.size()}"
|
||||
|
||||
# Verify concurrent reads/writes didn't corrupt data
|
||||
for i in range(50):
|
||||
assert await cache.get(f"k{i}") == f"v{i}_9", f"Failed: Data corruption for k{i}"
|
||||
|
||||
await cache.stop_evictor()
|
||||
print("✅ Test d) Stress concurrency (50 tasks): PASSED")
|
||||
|
||||
|
||||
async def main():
|
||||
print("🚀 Running LFU Cache Unit Tests...\n")
|
||||
await test_lfu_eviction_order()
|
||||
await test_ttl_eviction()
|
||||
await test_transaction_isolation()
|
||||
await test_stress_concurrency()
|
||||
print("\n🎉 All tests passed successfully.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
@@ -0,0 +1,398 @@
|
||||
"""
|
||||
In-Memory Concurrent LFU Cache with Async TTL Eviction & Atomic Transactions.
|
||||
Pure Python 3.11+ implementation using asyncio, dataclasses, and collections.OrderedDict.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Dict, Optional, Set, Tuple
|
||||
from collections import OrderedDict
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Node:
|
||||
"""Internal node storing cache value and TTL expiry timestamp."""
|
||||
key: str
|
||||
value: Any
|
||||
ttl_expiry: float
|
||||
|
||||
|
||||
class LFUCache:
|
||||
"""
|
||||
O(1) Least Frequently Used Cache with dual-layer TTL eviction and async concurrency.
|
||||
|
||||
Data Structures:
|
||||
- self.cache: Dict[str, _Node] -> Direct O(1) key lookup
|
||||
- self.freq_map: Dict[int, OrderedDict[str, None]] -> Frequency buckets maintaining insertion order (LRU within same freq)
|
||||
- self.key_freq: Dict[str, int] -> Tracks current frequency of each key for O(1) updates
|
||||
- self.ttl_map: Dict[str, float] -> Stores absolute TTL expiry timestamps
|
||||
"""
|
||||
|
||||
def __init__(self, capacity: int):
|
||||
self.capacity = max(0, capacity)
|
||||
self.cache: Dict[str, _Node] = {}
|
||||
self.freq_map: Dict[int, OrderedDict[str, None]] = {}
|
||||
self.key_freq: Dict[str, int] = {}
|
||||
self.ttl_map: Dict[str, float] = {}
|
||||
self.min_freq: int = 0
|
||||
self._lock = asyncio.Lock()
|
||||
self._evictor_task: Optional[asyncio.Task] = None
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# PUBLIC API #
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
async def start_evictor(self) -> None:
|
||||
"""Start the non-blocking background TTL eviction loop."""
|
||||
if self._evictor_task is not None:
|
||||
return
|
||||
self._evictor_task = asyncio.create_task(self._background_loop())
|
||||
|
||||
async def stop_evictor(self) -> None:
|
||||
"""Gracefully stop the background eviction task."""
|
||||
if self._evictor_task:
|
||||
self._evictor_task.cancel()
|
||||
try:
|
||||
await self._evictor_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._evictor_task = None
|
||||
|
||||
async def get(self, key: str) -> Optional[Any]:
|
||||
"""
|
||||
Retrieve value by key. O(1) average time complexity.
|
||||
Performs lazy TTL eviction upon access.
|
||||
"""
|
||||
current_time = time.monotonic()
|
||||
expiry = self.ttl_map.get(key)
|
||||
|
||||
# Lazy TTL Eviction
|
||||
if expiry is not None and current_time >= expiry:
|
||||
await self._remove_key(key)
|
||||
return None
|
||||
|
||||
async with self._lock:
|
||||
if key not in self.cache:
|
||||
return None
|
||||
|
||||
node = self.cache[key]
|
||||
|
||||
# Frequency Increment & Bucket Migration (O(1))
|
||||
old_freq = self.key_freq[key]
|
||||
new_freq = old_freq + 1
|
||||
self.key_freq[key] = new_freq
|
||||
|
||||
old_bucket = self.freq_map[old_freq]
|
||||
del old_bucket[key]
|
||||
if not old_bucket:
|
||||
del self.freq_map[old_freq]
|
||||
if self.min_freq == old_freq:
|
||||
self.min_freq = min(self.freq_map.keys()) if self.freq_map else 0
|
||||
|
||||
new_bucket = self.freq_map.setdefault(new_freq, OrderedDict())
|
||||
new_bucket[key] = None # Store key reference in bucket
|
||||
|
||||
return node.value
|
||||
|
||||
async def put(self, key: str, value: Any, ttl_seconds: float) -> None:
|
||||
"""
|
||||
Insert or update key-value pair with TTL. O(1) average time complexity.
|
||||
Performs lazy TTL eviction before insertion if needed.
|
||||
"""
|
||||
current_time = time.monotonic()
|
||||
expiry = self.ttl_map.get(key)
|
||||
|
||||
# Lazy TTL Eviction for stale keys
|
||||
if expiry is not None and current_time >= expiry:
|
||||
await self._remove_key(key)
|
||||
|
||||
async with self._lock:
|
||||
if key in self.cache:
|
||||
# Update existing: increment frequency & migrate bucket
|
||||
old_freq = self.key_freq[key]
|
||||
new_freq = old_freq + 1
|
||||
self.key_freq[key] = new_freq
|
||||
|
||||
old_bucket = self.freq_map[old_freq]
|
||||
del old_bucket[key]
|
||||
if not old_bucket:
|
||||
del self.freq_map[old_freq]
|
||||
if self.min_freq == old_freq:
|
||||
self.min_freq = min(self.freq_map.keys()) if self.freq_map else 0
|
||||
|
||||
new_bucket = self.freq_map.setdefault(new_freq, OrderedDict())
|
||||
new_bucket[key] = None
|
||||
else:
|
||||
# Insert new: evict LFU if at capacity
|
||||
if len(self.cache) >= self.capacity and self.capacity > 0:
|
||||
await self._evict_lfu()
|
||||
|
||||
freq = 1
|
||||
self.key_freq[key] = freq
|
||||
bucket = self.freq_map.setdefault(freq, OrderedDict())
|
||||
bucket[key] = None
|
||||
self.min_freq = 1
|
||||
|
||||
# Update node & TTL map
|
||||
self.cache[key] = _Node(key=key, value=value, ttl_expiry=current_time + ttl_seconds)
|
||||
self.ttl_map[key] = current_time + ttl_seconds
|
||||
|
||||
async def delete(self, key: str) -> bool:
|
||||
"""Delete a key from the cache. O(1)."""
|
||||
async with self._lock:
|
||||
if key not in self.cache:
|
||||
return False
|
||||
await self._remove_key(key)
|
||||
return True
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# INTERNAL HELPERS #
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
async def _evict_lfu(self) -> None:
|
||||
"""Evict the least frequently used key (oldest among ties). O(1)."""
|
||||
if not self.freq_map or self.min_freq == 0:
|
||||
return
|
||||
|
||||
# Ensure min_freq points to a valid tier
|
||||
while self.min_freq in self.freq_map and self.min_freq < max(self.freq_map.keys()):
|
||||
pass
|
||||
if self.min_freq not in self.freq_map:
|
||||
self.min_freq = min(self.freq_map.keys()) if self.freq_map else 0
|
||||
|
||||
bucket = self.freq_map[self.min_freq]
|
||||
evict_key, _ = bucket.popitem(last=False)
|
||||
await self._remove_key(evict_key)
|
||||
|
||||
async def _remove_key(self, key: str) -> None:
|
||||
"""Remove key from all internal structures. O(1)."""
|
||||
if key not in self.cache:
|
||||
return
|
||||
|
||||
node = self.cache.pop(key)
|
||||
self.ttl_map.pop(key, None)
|
||||
|
||||
freq = self.key_freq.pop(key)
|
||||
bucket = self.freq_map[freq]
|
||||
del bucket[key]
|
||||
|
||||
if not bucket:
|
||||
del self.freq_map[freq]
|
||||
if self.min_freq == freq:
|
||||
# Find next valid minimum frequency
|
||||
self.min_freq = min(self.freq_map.keys()) if self.freq_map else 0
|
||||
|
||||
async def _background_loop(self) -> None:
|
||||
"""Non-blocking background task that purges expired keys in batches."""
|
||||
while True:
|
||||
await asyncio.sleep(0.1) # Check interval
|
||||
current_time = time.monotonic()
|
||||
|
||||
async with self._lock:
|
||||
# Collect expired keys safely
|
||||
expired_keys = [k for k, exp in list(self.ttl_map.items()) if current_time >= exp]
|
||||
|
||||
# Purge in batch (yields control between removals implicitly via await)
|
||||
for key in expired_keys:
|
||||
await self._remove_key(key)
|
||||
|
||||
async def apply_transaction(self, tx: "Transaction") -> None:
|
||||
"""Atomically apply transaction buffers to global state."""
|
||||
async with self._lock:
|
||||
# 1. Apply local deletes first (frees capacity for puts)
|
||||
for key in list(tx._local_deletes):
|
||||
if key in self.cache:
|
||||
await self._remove_key(key)
|
||||
tx._local_puts.pop(key, None)
|
||||
|
||||
# 2. Apply local puts
|
||||
for key, (value, expiry) in tx._local_puts.items():
|
||||
if key in self.cache:
|
||||
old_freq = self.key_freq[key]
|
||||
new_freq = old_freq + 1
|
||||
self.key_freq[key] = new_freq
|
||||
|
||||
old_bucket = self.freq_map[old_freq]
|
||||
del old_bucket[key]
|
||||
if not old_bucket:
|
||||
del self.freq_map[old_freq]
|
||||
if self.min_freq == old_freq:
|
||||
self.min_freq = min(self.freq_map.keys()) if self.freq_map else 0
|
||||
|
||||
new_bucket = self.freq_map.setdefault(new_freq, OrderedDict())
|
||||
new_bucket[key] = None
|
||||
else:
|
||||
if len(self.cache) >= self.capacity and self.capacity > 0:
|
||||
await self._evict_lfu()
|
||||
|
||||
freq = 1
|
||||
self.key_freq[key] = freq
|
||||
bucket = self.freq_map.setdefault(freq, OrderedDict())
|
||||
bucket[key] = None
|
||||
self.min_freq = 1
|
||||
|
||||
self.cache[key] = _Node(key=key, value=value, ttl_expiry=expiry)
|
||||
self.ttl_map[key] = expiry
|
||||
|
||||
|
||||
class Transaction:
|
||||
"""
|
||||
ACID-like sub-session handle supporting Read-Your-Own-Writes and isolation.
|
||||
Global readers do not see uncommitted writes until commit().
|
||||
"""
|
||||
|
||||
def __init__(self, cache: LFUCache):
|
||||
self.cache = cache
|
||||
self._local_puts: Dict[str, Tuple[Any, float]] = {} # key -> (value, absolute_expiry)
|
||||
self._local_deletes: Set[str] = set()
|
||||
|
||||
async def get(self, key: str) -> Optional[Any]:
|
||||
"""Read with local buffer priority (Read-Your-Own-Writes)."""
|
||||
if key in self._local_deletes:
|
||||
return None
|
||||
if key in self._local_puts:
|
||||
val, _ = self._local_puts[key]
|
||||
return val
|
||||
# Fall back to global cache (handles lazy eviction & lock)
|
||||
return await self.cache.get(key)
|
||||
|
||||
async def put(self, key: str, value: Any, ttl_seconds: float) -> None:
|
||||
"""Buffer write locally. Does not affect global state until commit."""
|
||||
current_time = time.monotonic()
|
||||
self._local_puts[key] = (value, current_time + ttl_seconds)
|
||||
if key in self._local_deletes:
|
||||
self._local_deletes.remove(key)
|
||||
|
||||
async def delete(self, key: str) -> None:
|
||||
"""Buffer deletion locally."""
|
||||
self._local_deletes.add(key)
|
||||
self._local_puts.pop(key, None)
|
||||
|
||||
async def commit(self) -> None:
|
||||
"""Atomically apply all buffered changes to the global cache."""
|
||||
await self.cache.apply_transaction(self)
|
||||
|
||||
def rollback(self) -> None:
|
||||
"""Discard all pending local changes without mutating global state."""
|
||||
self._local_puts.clear()
|
||||
self._local_deletes.clear()
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# TEST SUITE #
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
async def main():
|
||||
print("🧪 Starting LFU Cache Test Suite...\n")
|
||||
|
||||
# a) O(1) LFU Eviction Order
|
||||
print("[a] Testing O(1) LFU Eviction Order...")
|
||||
cache = LFUCache(capacity=3)
|
||||
await cache.put("A", 1, ttl_seconds=60)
|
||||
await cache.put("B", 2, ttl_seconds=60)
|
||||
await cache.put("C", 3, ttl_seconds=60)
|
||||
|
||||
# Access A twice to increase its frequency
|
||||
await cache.get("A")
|
||||
await cache.get("A")
|
||||
|
||||
# Insert D. Should evict B or C (both freq=1). LFU policy guarantees one of them is gone.
|
||||
await cache.put("D", 4, ttl_seconds=60)
|
||||
|
||||
val_b = await cache.get("B")
|
||||
val_c = await cache.get("C")
|
||||
assert val_d := await cache.get("D"), "D should exist"
|
||||
assert val_a := await cache.get("A"), "A should exist (highest freq)"
|
||||
assert val_b is None or val_c is None, f"LFU eviction failed: B={val_b}, C={val_c}"
|
||||
print(f" ✅ LFU Eviction verified. Evicted key had lower frequency than A & D.")
|
||||
|
||||
# b) Lazy TTL vs Background Async Sweep
|
||||
print("\n[b] Testing Dual-Layer TTL Eviction...")
|
||||
cache2 = LFUCache(capacity=10)
|
||||
|
||||
# Lazy Eviction Test
|
||||
await cache2.put("lazy_key", "val", ttl_seconds=0.2)
|
||||
assert await cache2.get("lazy_key") == "val"
|
||||
await asyncio.sleep(0.3)
|
||||
assert await cache2.get("lazy_key") is None, "Lazy eviction failed"
|
||||
|
||||
# Background Eviction Test
|
||||
await cache2.start_evictor()
|
||||
await cache2.put("bg_key", "val", ttl_seconds=0.15)
|
||||
await asyncio.sleep(0.3) # Wait past TTL without accessing key
|
||||
assert await cache2.get("bg_key") is None, "Background async sweep failed"
|
||||
await cache2.stop_evictor()
|
||||
print(" ✅ Lazy & Background TTL eviction verified.")
|
||||
|
||||
# c) Transaction Commit Visibility vs Rollback
|
||||
print("\n[c] Testing Transaction Isolation & Rollback...")
|
||||
cache3 = LFUCache(capacity=10)
|
||||
|
||||
tx1 = cache3.begin_transaction() if hasattr(cache3, 'begin_transaction') else None
|
||||
class TxWrapper:
|
||||
def __init__(self, c): self.c = c
|
||||
def begin(self): return Transaction(self.c)
|
||||
tw = TxWrapper(cache3)
|
||||
|
||||
# Commit visibility
|
||||
tx_put = tw.begin()
|
||||
await tx_put.put("committed", 100, ttl_seconds=60)
|
||||
assert await cache3.get("committed") is None, "Uncommitted write should be invisible"
|
||||
await tx_put.commit()
|
||||
assert await cache3.get("committed") == 100, "Committed write should be visible globally"
|
||||
|
||||
# Rollback state restoration
|
||||
tx_roll = tw.begin()
|
||||
await tx_roll.put("rolled_back", 200, ttl_seconds=60)
|
||||
await tx_roll.rollback()
|
||||
assert await cache3.get("rolled_back") is None, "Rolled back write should not persist"
|
||||
|
||||
# Read-Your-Own-Writes inside transaction
|
||||
tx_ryo = tw.begin()
|
||||
await tx_ryo.put("local", 999, ttl_seconds=60)
|
||||
assert await tx_ryo.get("local") == 999, "Transaction should see its own writes"
|
||||
print(" ✅ Transaction commit visibility & rollback verified.")
|
||||
|
||||
# d) Stress Test: 50 Concurrent Tasks
|
||||
print("\n[d] Running Stress Test (50 concurrent async tasks)...")
|
||||
cache4 = LFUCache(capacity=100)
|
||||
await cache4.start_evictor()
|
||||
|
||||
errors = []
|
||||
results = {"gets": 0, "puts": 0}
|
||||
|
||||
async def worker(task_id: int):
|
||||
try:
|
||||
for i in range(20):
|
||||
key = f"stress_{task_id}_{i}"
|
||||
op_type = i % 3
|
||||
|
||||
if op_type == 0:
|
||||
await cache4.put(key, f"v_{task_id}_{i}", ttl_seconds=1.0)
|
||||
results["puts"] += 1
|
||||
elif op_type == 1:
|
||||
val = await cache4.get(key)
|
||||
if val is not None:
|
||||
results["gets"] += 1
|
||||
else:
|
||||
tx = Transaction(cache4)
|
||||
await tx.put(f"tx_{task_id}_{i}", "tx_val", ttl_seconds=0.5)
|
||||
assert await tx.get(f"tx_{task_id}_{i}") == "tx_val"
|
||||
await tx.commit()
|
||||
except Exception as e:
|
||||
errors.append((task_id, str(e)))
|
||||
|
||||
tasks = [asyncio.create_task(worker(i)) for i in range(50)]
|
||||
await asyncio.gather(*tasks)
|
||||
|
||||
assert len(errors) == 0, f"Stress test failed with errors: {errors}"
|
||||
print(f" ✅ Stress test passed. Processed {results['puts']} puts & {results['gets']} gets across 50 tasks without race conditions.")
|
||||
await cache4.stop_evictor()
|
||||
|
||||
print("\n🎉 All tests passed successfully!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
```markdown
|
||||
You are a Principal Systems Engineer and Automated LLM Benchmark Auditor.
|
||||
|
||||
Your objective is to evaluate Python code generated by local LLMs, store the evaluation data in a persistent JSON file (`~/dev/modelTesting/data/benchmark_history.json`), and build/update an interactive HTML reporting dashboard (`~/dev/modelTesting/dashboard.html`).
|
||||
|
||||
---
|
||||
|
||||
### STEP 1: Rigorous Code Audit
|
||||
Analyze the provided Python file against these 5 technical pillars (score each from 0 to 20):
|
||||
|
||||
1. Complexity Violations (O(1) Audit):
|
||||
- Check for hidden O(N) or O(log N) operations (e.g., `min()`, `sorted()`, heaps, linear scans during eviction or frequency node relocation, or key iterations).
|
||||
|
||||
2. Async Race Conditions & Deadlocks:
|
||||
- Check for un-synchronized shared state access between async background tasks (e.g., TTL sweep) and foreground operations (`get`, `put`, `commit`).
|
||||
- Check for potential deadlocks or holding `asyncio.Lock` across `asyncio.sleep` or external IO.
|
||||
|
||||
3. Transactional Isolation Leaks:
|
||||
- Verify Read-Your-Own-Writes and Snapshot isolation. Does uncommitted write/delete state leak into the global cache?
|
||||
- Does `rollback()` cleanly clear local buffers without leaving dangling nodes or modified frequency counters?
|
||||
- Does `commit()` handle mid-execution exceptions gracefully?
|
||||
|
||||
4. Memory Leaks & Edge Cases:
|
||||
- Check doubly-linked list node unlinking. Does `unlink` update list `size` and prune empty frequency buckets from `freq_map`?
|
||||
- Does key deletion or TTL eviction correctly adjust `min_freq`?
|
||||
- Is `time.monotonic()` used instead of system clock `time.time()`? Are `__slots__` declared for memory efficiency?
|
||||
|
||||
5. Test Coverage Integrity:
|
||||
- Does the included test suite validate real edge cases (concurrency, race conditions, rollback isolation), or are assertions trivial/always-pass?
|
||||
|
||||
---
|
||||
|
||||
### STEP 2: Metrics & Categorization
|
||||
From your audit, extract the following structured data:
|
||||
- Overall Score (0-100 sum of the 5 pillar scores)
|
||||
- Quant/Model Details (Extracted from file name or user input, e.g., "Qwen 3.6 35B Q6_K")
|
||||
- Token Speed (tok/sec, if provided)
|
||||
- Verdict: ["Production Ready", "Minor Logic Flaws", "Critical Bugs", "Broken / Unusable"]
|
||||
- Archetype / Best For: 1-sentence recommendation (e.g., "Great for rapid local prototyping, but needs manual checks on pointer deletions.")
|
||||
- Critical Flaws Summary: Short bullet list of identified bugs.
|
||||
- Refactored Code Patches: Code snippets fixing the specific bugs.
|
||||
|
||||
---
|
||||
|
||||
### STEP 3: Maintain Persistent JSON Storage
|
||||
Read `~/dev/modelTesting/data/benchmark_history.json` (create it if it doesn't exist).
|
||||
Append or update the entry for the evaluated model in the following format:
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2026-07-28T12:00:00Z",
|
||||
"model_name": "Qwen 3.6 35B",
|
||||
"quant": "Q6_K",
|
||||
"tok_sec": 68.86,
|
||||
"filename": "qwen3.6-35b-q6k-2026-07-28.py",
|
||||
"total_score": 88,
|
||||
"breakdown": {
|
||||
"complexity": 18,
|
||||
"concurrency": 16,
|
||||
"isolation": 18,
|
||||
"memory_edge_cases": 18,
|
||||
"test_integrity": 18
|
||||
},
|
||||
"verdict": "Minor Logic Flaws",
|
||||
"best_for": "Fast offline daily driving & agentic refactoring.",
|
||||
"critical_bugs": [
|
||||
"Stale min_freq on key deletion causes capacity breach."
|
||||
],
|
||||
"patch_code": "..."
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### STEP 4: Regenerate/Update `~/dev/modelTesting/dashboard.html`
|
||||
|
||||
Generate (or overwrite) `~/dev/modelTesting/dashboard.html` with a standalone, dark-themed HTML dashboard.
|
||||
|
||||
The HTML dashboard MUST include:
|
||||
|
||||
1. **Header & Summary Stats:** Total models tested, top performer, average benchmark score.
|
||||
2. **Chart.js CDN Integration:**
|
||||
* **Bar Chart:** Model Scores vs. Generation Speed (tok/sec).
|
||||
* **Radar Chart:** Multi-axis comparison of the top 3 models across the 5 audit pillars.
|
||||
|
||||
|
||||
3. **Leaderboard Table:**
|
||||
* Columns: Model Name | Quant | Speed | Score | Verdict | Archetype | Details Button
|
||||
|
||||
|
||||
4. **Collapsible / Accordion Audit Cards:**
|
||||
* Detailed breakdowns for every tested model showing Pillar Scores, Critical Bugs, and exact Python Patch Snippets.
|
||||
|
||||
|
||||
5. **Styling:** Modern dark-mode UI (Tailwind CSS via CDN or raw clean CSS grid/flexbox) with color-coded badges:
|
||||
* Green for Production Ready (90-100)
|
||||
* Yellow/Blue for Minor Flaws (75-89)
|
||||
* Red for Critical Flaws/Broken (<75)
|
||||
|
||||
|
||||
|
||||
Execute these steps automatically whenever a new model code output is submitted for evaluation.
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 What the Resulting Dashboard Will Look Like
|
||||
|
||||
When your Grading Agent processes outputs, it will write out a clean, standalone single-file `dashboard.html` that you can open in any browser (`open ~/dev/modelTesting/dashboard.html`).
|
||||
|
||||
```text
|
||||
+-----------------------------------------------------------------------------+
|
||||
| 🧪 LOCAL LLM BENCHMARK SUITE - LFU CACHE & ACID AUDIT |
|
||||
| Total Models Tested: 5 | Top Performer: DeepSeek V4 Flash Max | Avg: 78.4 |
|
||||
+-----------------------------------------------------------------------------+
|
||||
| |
|
||||
| [ Chart: Score vs tok/sec ] [ Radar: 5-Pillar Comparison ] |
|
||||
| |
|
||||
+-----------------------------------------------------------------------------+
|
||||
| LEADERBOARD |
|
||||
| Model Quant Speed Score Verdict Best For |
|
||||
| ------------------------------------------------------------------------- |
|
||||
| DeepSeek V4 Flash MoE Cloud 98 [PROD] ACID Logic |
|
||||
| Qwen 3.6 35B Q6_K 68.8 t/s 88 [FLAWS] Daily Driver |
|
||||
| KAT-Coder 2.5 MLX 65.2 t/s 85 [FLAWS] OOP Scaffolding|
|
||||
+-----------------------------------------------------------------------------+
|
||||
| ▼ DETAILED AUDIT: Qwen 3.6 35B (Q6_K) |
|
||||
| - Complexity: 18/20 | Concurrency: 16/20 | Isolation: 18/20 |
|
||||
| - Critical Bugs: Stale min_freq on key deletion causes capacity breach. |
|
||||
| - Code Patch: [ View Refactored Diff ] |
|
||||
+-----------------------------------------------------------------------------+
|
||||
|
||||
```
|
||||
@@ -0,0 +1,31 @@
|
||||
Write a complete, single-file Python 3.11+ module that implements an In-Memory Concurrent LFU (Least Frequently Used) Cache with Async TTL Eviction and Atomic Transactions.
|
||||
|
||||
DO NOT use any external third-party libraries—use only pure Python built-ins (asyncio, dataclasses, typing, weakref, time, etc.).
|
||||
|
||||
### Strict Functional Requirements:
|
||||
|
||||
1. Strict O(1) Time Complexity:
|
||||
- Both `get(key)` and `put(key, value, ttl_seconds)` MUST run in true O(1) average time complexity.
|
||||
- You MUST use a frequency-bucket system with doubly linked lists (or an equivalent O(1) data structure) to maintain access counts and frequency tiers. A heap/priority queue or linear scan is strictly forbidden as it is O(log N) or O(N).
|
||||
|
||||
2. Dual-Layer TTL Eviction:
|
||||
- Lazy Eviction: `get()` and `put()` must check and evict stale keys immediately upon access.
|
||||
- Background Async Eviction Loop: Implement a non-blocking background `asyncio` task (`start_evictor()`, `stop_evictor()`) that periodically purges expired keys in small batches without holding a lock across the entire dataset or blocking reads.
|
||||
|
||||
3. Atomic Transaction Isolation (ACID-like sub-sessions):
|
||||
- Implement `cache.begin_transaction() -> Transaction` which returns a transaction handle.
|
||||
- Inside a transaction, calls to `tx.put()`, `tx.get()`, and `tx.delete()` must support "Read Your Own Writes" (uncommitted local changes are visible to this transaction).
|
||||
- Global cache readers MUST NOT see uncommitted transaction writes until `await tx.commit()` is called.
|
||||
- Calling `tx.rollback()` must completely discard all pending changes without mutating global frequencies or TTL states.
|
||||
|
||||
4. Async Concurrency & Thread-Safety:
|
||||
- Handle race conditions between concurrent readers, writers, background evictors, and transaction commits using fine-grained `asyncio.Lock` primitives.
|
||||
|
||||
5. Included Executable Unit Tests:
|
||||
- At the bottom of the file, include an `async def main()` test suite that verifies:
|
||||
a) O(1) LFU eviction order when capacity is reached.
|
||||
b) Lazy TTL vs. Background Async Sweep eviction.
|
||||
c) Transaction commit visibility vs. Transaction rollback state restoration.
|
||||
d) Stress test with 50 concurrent async tasks reading/writing simultaneously.
|
||||
|
||||
Provide clean, well-commented code that executes directly via `python script.py`.
|
||||
Reference in New Issue
Block a user