Add Aygea Test prompt battery: 6 prompts from real project shapes
Surveyed ~/dev (mewtwo) + jirachi. Battery mirrors actual workload:
mcp_server -> 9 MCP servers (joplin/obsidian/vault/project-rag/...)
tts_pipeline -> TTS/audio pipelines (Chatterbox, aygea-tts, vr-to-tts)
webhook_bridge -> Twitch/Discord bridges (multistream, notifier, overlay)
data_service -> data/API (Supabase MCP, PostgresHA, dashboard)
automation_glue -> batch/cron glue (fix-tokens, notesCleanup)
rust_service -> big Rust services (NineSentry, aystreamer): tokio
channels + Arc/Mutex + error enums + graceful shutdown
Each prompt is ~2-3KB (fits 128k context with output room), single-file,
runnable, graded on the same 5-pillar rubric. aygea_test_battery.md is the
index + scoring notes + the prompt_id schema the dashboard will need for
multi-prompt support.
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,25 @@
|
||||
Write a complete, single-file async TTS (text-to-speech) job pipeline in Python 3.11+ (asyncio, stdlib only) OR Node.js (no external deps beyond stdlib). It accepts text jobs and processes them through a mock synthesizer with bounded concurrency.
|
||||
|
||||
### API
|
||||
- `async def submit(text: str, voice: str) -> str` — enqueue a job, return a job id immediately (non-blocking).
|
||||
- `async def drain()` — wait until all queued AND in-flight jobs are finished.
|
||||
- `async def cancel(job_id)` — cancel a queued job; if in-flight, mark for cancellation when safe.
|
||||
- A way to register a callback that receives job lifecycle events: `queued`, `started`, `completed`, `failed`, `cancelled`.
|
||||
|
||||
### Mock synthesizer
|
||||
Provide `async def mock_synthesize(text: str) -> bytes` that sleeps a random 50–300ms and returns `b"\x00" * len(text)` (or similar). Make it fail ~10% of the time (raise a synthetic error) so retry logic is exercised.
|
||||
|
||||
### Requirements
|
||||
1. **Bounded concurrency:** at most 4 jobs synthesize at once.
|
||||
2. **Backpressure:** when the queue length exceeds 100, `submit` must reject immediately with a clear error (not block, not grow unbounded).
|
||||
3. **Retry on failure:** a job that fails must retry up to 3 times with exponential backoff (e.g. 0.1s, 0.2s, 0.4s). After exhausting retries it emits `failed`.
|
||||
4. **Clean cancellation:** cancelling must not leak tasks or leave the worker pool in a bad state; `drain()` must always return.
|
||||
5. **Thread/async safety:** no shared mutable state races between submit, workers, and callbacks.
|
||||
|
||||
### Included test
|
||||
Include an `async def main()` test that:
|
||||
- a) Submits 50 jobs, drains, and asserts exactly 4-or-fewer ran concurrently at any time (record max concurrency).
|
||||
- b) Proves the 100-job backpressure cap rejects when exceeded.
|
||||
- c) Submits a batch, cancels one mid-flight, drains, and asserts no jobs were dropped/duplicated and the pool is still usable afterward.
|
||||
|
||||
Provide clean, well-commented code that runs directly via `python file.py`.
|
||||
Reference in New Issue
Block a user