Unify ready pipeline jobs under one worker pool and remove the global LLM scheduler
This commit is contained in:
@@ -1133,6 +1133,31 @@ only purpose was the retired generic retry/asset API.
|
||||
|
||||
## Stage 25: Replace Worker Pools And Remove The Global LLM Scheduler
|
||||
|
||||
**Status:** Complete. One lazy coordinator now dispatches ready extraction and
|
||||
merge/normalize continuations through the same bounded pool. Ready continuations
|
||||
can run while unrelated extraction remains; prerequisite waits occupy no worker.
|
||||
Cancellation disables dispatch, drains every dispatched result, and joins workers.
|
||||
Stable output/error ordering and sequential step barriers remain. Replaced
|
||||
extraction-only worker inputs with shared workers, using the selected default of
|
||||
64 consistently with configuration. Removed the global scheduler/scheduled client
|
||||
and their obsolete tests and CLI wrapping; native engine admission is independent.
|
||||
Migrated retained concurrency fixtures and replaced legacy scheduler integration
|
||||
coverage with real native-helper/backend tests. New tests cover mixed-job limits,
|
||||
serial work, independent backend limits, nested same-backend validation, strict/
|
||||
permissive capacity failure without another queue, and cancellation/admission reuse.
|
||||
The mixed-job regression demonstrably fails with the old pools (peak 3 under
|
||||
limit 2) and passes with the replacement. Focused pipeline tests/vet and repeated
|
||||
race checks pass, as do generic/Seriatim/contracts/reconciliation tests and worker
|
||||
configuration tests. Production pipeline/config builds and `git diff --check`
|
||||
pass. Full CLI/module integration still stops at the legacy YAML `plugin/limit`
|
||||
import. The full pipeline suite now reaches remaining retired fixtures such as
|
||||
`NormalizeRetry`/`runProducerAttempts`; cleanup remains required. After migrating
|
||||
the configuration extractor fixture, full configuration tests compile but expose
|
||||
three stale assertions: two use literal backslash-x00 rather than NUL paths, and
|
||||
one reads the unresolved binding turn pointer rather than resolved controls.
|
||||
Those full-suite results are not claimed passing; scheduled fixture cleanup and
|
||||
final verification must close them.
|
||||
|
||||
**Goal:** one ready-job limit and native backend admission govern concurrent work.
|
||||
|
||||
**Inspect/change:** `runner_concurrent.go`, synchronized collaborators,
|
||||
|
||||
Reference in New Issue
Block a user