adminandClaude
ec6fd7157a
Surveyed ~/dev (mewtwo) + jirachi. Battery mirrors actual workload:
mcp_server -> 9 MCP servers (joplin/obsidian/vault/project-rag/...)
tts_pipeline -> TTS/audio pipelines (Chatterbox, aygea-tts, vr-to-tts)
webhook_bridge -> Twitch/Discord bridges (multistream, notifier, overlay)
data_service -> data/API (Supabase MCP, PostgresHA, dashboard)
automation_glue -> batch/cron glue (fix-tokens, notesCleanup)
rust_service -> big Rust services (NineSentry, aystreamer): tokio
channels + Arc/Mutex + error enums + graceful shutdown
Each prompt is ~2-3KB (fits 128k context with output room), single-file,
runnable, graded on the same 5-pillar rubric. aygea_test_battery.md is the
index + scoring notes + the prompt_id schema the dashboard will need for
multi-prompt support.
Co-Authored-By: Claude <noreply@anthropic.com>
23 lines
2.3 KiB
Plaintext
23 lines
2.3 KiB
Plaintext
Write a complete, single-file async batch processor (Python 3.11+ asyncio, stdlib only) that processes a list of items through a flaky external call with checkpointing, idempotency, and clean interruption — the kind of "automation glue" that runs unattended and must be re-runnable.
|
||
|
||
### Behavior
|
||
- Input: a list of items (e.g. `["job-a", "job-b", ...]`, 50–200 of them).
|
||
- For each item, call `async def process(item: str) -> str` (provided) that succeeds ~80% of the time and raises `ProcessingError` otherwise, with random latency 10–80ms.
|
||
- Persist progress to a checkpoint file (`checkpoint.json`) after each item: the set of completed item ids + a running summary.
|
||
|
||
### Requirements
|
||
1. **Idempotency / resumability:** on start, load the checkpoint; skip any item already marked completed. Re-running with the same input + checkpoint must NEVER reprocess a completed item and must converge to all-done.
|
||
2. **Retries with backoff:** each item retries up to 3 times on `ProcessingError` with exponential backoff (e.g. 0.1s, 0.2s, 0.4s) before being recorded as `failed`. (After retries, a failed item is terminal — it does not block the rest.)
|
||
3. **Bounded concurrency:** process up to 8 items at once.
|
||
4. **Checkpoint integrity:** the checkpoint file must never be left half-written / corrupt if the process dies mid-write (write to a temp file then atomically rename). A crash at any point must leave a valid checkpoint.
|
||
5. **Graceful SIGINT:** on Ctrl-C / SIGINT, stop accepting new items, let in-flight ones finish (or cancel cleanly), flush the checkpoint, then exit 0. No partial item is ever recorded as completed.
|
||
6. **Structured summary:** at the end, print a single JSON line: `{"succeeded": n, "failed": n, "skipped": n, "total": n, "elapsed_ms": ...}`.
|
||
|
||
### Included test
|
||
Include a runnable test that:
|
||
- a) Runs a batch, kills mid-run (simulate via a small in-process cancellation), restarts, and asserts: no completed item was reprocessed, and the final state is all items either succeeded-or-failed.
|
||
- b) Asserts the checkpoint file is valid JSON at every observed moment (write a watcher that reads it repeatedly during a run and confirms it always parses).
|
||
- c) Asserts the concurrency cap of 8 holds.
|
||
|
||
Provide clean, well-commented code that runs directly via `python file.py`.
|