Initial benchmark suite: 8 graded models + cyberpunk dashboard generator

- prompts/: LFU cache exam + 5-pillar grading rubric
- outputs/: 8 model .py outputs (local + cloud baseline)
- data/benchmark_history.json: graded results (scores, metrics, bugs, patches)
- generate_dashboard.py: builds dashboard.html + pages/*.html from JSON
- Dockerfile + DEPLOY.md: Gitea→Coolify deploy (build-step, nginx static)
- .gitignore: generated HTML excluded (built on deploy)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-07-28 14:00:40 -07:00
co-authored by Claude
commit f0281f2878
17 changed files with 4363 additions and 0 deletions
+31
View File
@@ -0,0 +1,31 @@
Write a complete, single-file Python 3.11+ module that implements an In-Memory Concurrent LFU (Least Frequently Used) Cache with Async TTL Eviction and Atomic Transactions.
DO NOT use any external third-party libraries—use only pure Python built-ins (asyncio, dataclasses, typing, weakref, time, etc.).
### Strict Functional Requirements:
1. Strict O(1) Time Complexity:
- Both `get(key)` and `put(key, value, ttl_seconds)` MUST run in true O(1) average time complexity.
- You MUST use a frequency-bucket system with doubly linked lists (or an equivalent O(1) data structure) to maintain access counts and frequency tiers. A heap/priority queue or linear scan is strictly forbidden as it is O(log N) or O(N).
2. Dual-Layer TTL Eviction:
- Lazy Eviction: `get()` and `put()` must check and evict stale keys immediately upon access.
- Background Async Eviction Loop: Implement a non-blocking background `asyncio` task (`start_evictor()`, `stop_evictor()`) that periodically purges expired keys in small batches without holding a lock across the entire dataset or blocking reads.
3. Atomic Transaction Isolation (ACID-like sub-sessions):
- Implement `cache.begin_transaction() -> Transaction` which returns a transaction handle.
- Inside a transaction, calls to `tx.put()`, `tx.get()`, and `tx.delete()` must support "Read Your Own Writes" (uncommitted local changes are visible to this transaction).
- Global cache readers MUST NOT see uncommitted transaction writes until `await tx.commit()` is called.
- Calling `tx.rollback()` must completely discard all pending changes without mutating global frequencies or TTL states.
4. Async Concurrency & Thread-Safety:
- Handle race conditions between concurrent readers, writers, background evictors, and transaction commits using fine-grained `asyncio.Lock` primitives.
5. Included Executable Unit Tests:
- At the bottom of the file, include an `async def main()` test suite that verifies:
a) O(1) LFU eviction order when capacity is reached.
b) Lazy TTL vs. Background Async Sweep eviction.
c) Transaction commit visibility vs. Transaction rollback state restoration.
d) Stress test with 50 concurrent async tasks reading/writing simultaneously.
Provide clean, well-commented code that executes directly via `python script.py`.