Resolve shared worker pool and native admission policy

This commit is contained in:
2026-11-20 17:32:50 +00:00
parent 1292ed8c4b
commit 254401e5f6

View File

@@ -196,8 +196,9 @@ a singleton flag and use the last successfully parsed value.
| Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. |
| Capture destination | Application `--debug-dir PATH`, with the semantics below. |
Keep worker counts and semantic budgets in configuration, adjustable through
application `--set`. U5 selects `max_additional_turns`; worker defaults remain U7.
Keep the worker limit and semantic budgets in configuration, adjustable through
application `--set`. U5 selects `max_additional_turns`; U7 selects
`concurrency.workers` with a default of `64`.
Use ckprompt's reusable profile, session, and execution flags, including model,
temperature, timeout, reasoning effort, and transient retries, for run-wide
overrides across producers, validators, and continuations. Omission preserves
@@ -280,7 +281,7 @@ Retire `promptkit.profile_file`, `profile_dir`, and `local_backend` in favor of
native companion sources; do not introduce a separate `promptkit.config_file`.
Retire `concurrency.total_llm` and revise retry/worker settings as described below.
U5 selects `max_additional_turns` and removes application structural-repair
controls; worker defaults and retired global-limit migration remain U7.
controls; U7 selects the shared worker pool and retired global-limit migration.
Use one source assembly for runtime, preflight, and inspection:
@@ -321,8 +322,8 @@ and its reference to the bundled unqualified `openrouter` backend. Inherited
application profiles use sibling local IDs in the same collection. The bundled
inventory differs from the old external catalogs and selects no model for an
application request; document intentional replacements for removed catalog IDs.
The bundled backend supplies no concurrency limit. Application/operator limits
need explicit native configuration; application-supplied defaults remain U7.
The bundled backend supplies no concurrency limit. Under U7, Notarius injects no
backend concurrency or queue defaults; operators use native backend configuration.
Preserve prompt prose, message ordering, roles, and cache boundaries where their
meaning carries forward. Review and deliberately revise initial/repair prompts
@@ -667,12 +668,39 @@ conversation state in process, outside ordinary artifact/cache payloads;
explicitly requested detailed debug is separate. U6 is resolved; implementation
and module-specific construction details remain in Phases 3–4.
### Admission And Pipeline Workers
### Admission And Pipeline Workers (U7 Resolved)
Remove `Scheduler`, `scheduledClient`, and custom global queue/permit plumbing.
Retire the separate cross-backend ceiling and the default/maximum extraction
worker coupling to `concurrency.total_llm`. Keep bounded extraction workers and
bounded lane continuations with independent defaults to be selected.
worker coupling to `concurrency.total_llm`. Use one shared pool for ready pipeline
jobs, configured by the positive integer `concurrency.workers`, default `64`.
Omission uses the default; explicit `1` provides serial job execution. Keep this
in application configuration, adjustable through `--set`, without a dedicated
CLI flag or separate extraction/lane worker settings.
The pool serves both kinds of work:
| Job | Work covered by one worker slot |
| --- | --- |
| Extraction | One chunk-and-lane extraction, including its validation and semantic correction. |
| Lane completion | A ready lane's merge and normalization, including their validation and correction. |
The limit counts total active jobs across both kinds, with no separate reserved
capacity. The existing pipeline coordinator tracks dependencies and makes lane
completion ready after that lane's extraction finishes. A worker never holds a
slot while waiting for prerequisite jobs. Validators and correction turns execute
within the current job instead of acquiring another worker slot. Preserve stable
public result ordering regardless of completion order. Input preparation, chunking,
and final publication retain their sequential placement around lane execution.
This remains execution of the fixed pipeline, not a general workflow scheduler.
Count application jobs, not OS threads or all process goroutines. Do not derive
the default from CPU count or `GOMAXPROCS`; model-backed jobs often wait for I/O
or native admission. The pool bounds active application state and preparation as
well as model-backed work. `64` supplies practical headroom for backend utilization
while remaining well below the reviewed native default queue capacity; it is not
a benchmark-derived optimum or a guarantee of backend saturation. Operators may
tune it for workload and host resources independently of backend settings.
Use one shared native engine across all roles. Native capacity belongs to the
engine and backend ID: profiles selecting the same ID share its pool; different
@@ -680,6 +708,15 @@ backend IDs or separate engines do not automatically share limits, even for the
same endpoint. Endpoint-only profiles and backends without a positive limit
remain unrestricted by native backend admission.
Inject no Notarius-owned backend concurrency or queue defaults. Preserve native
operator configuration and inherit upstream defaults rather than deriving backend
settings from the worker limit. In the reviewed PromptKit baseline, omitted/zero
backend concurrency is unlimited and has no queue; with positive concurrency,
omitted queue capacity defaults to `1024`. Explicit queue capacity, including zero,
uses native semantics. These are upstream defaults, not values pinned by Notarius.
With an unlimited backend, active jobs can generate concurrent calls up to the
application's workload; the worker default does not imply a separate LLM ceiling.
For limited backends, the reviewed baseline bounds active model attempts by
`concurrency_limit` and admitted operations by that limit plus `queue_capacity`.
Ordinary execution admission spans preparation, retries, and repairs; split
@@ -687,12 +724,36 @@ Ordinary execution admission spans preparation, retries, and repairs; split
`RunPrepared`. Active permits are released during retry waits.
A completed continuation turn releases admission before assessment, allowing a
nested validator to use the same backend. Configure native limits at the intended
capacity boundary and handle `ErrCapacityExceeded` through application policy.
capacity boundary. Worker counts need not be less than backend concurrency;
native admission can queue waiting operations.
Handle `ErrCapacityExceeded` through U6's existing strict/permissive policy:
- A producer admission failure fails in strict mode or uses the module's
supported fallback in permissive mode.
- A validator admission failure means assessment is unavailable; apply the
producing module's resolved mode. Do not turn it into a semantic rejection
or a correction instruction.
- Cancellation stops work and does not initiate fallback.
Preserve the error and backend identity in diagnostics. Add no application retry,
backoff, or second admission queue. A full native queue rejects admission without
calling the model; the large default queue makes this unusual with modest worker
counts, but explicit operator limits remain authoritative. Use ordinary fallback
reporting and cache exclusions when permissive handling continues.
Aggregate concurrency across independent backends may exceed the former global
ceiling. That change is intentional. Bounded pipeline workers are application
work controls, not a substitute cross-backend LLM ceiling. Defaults, retired-field
migration, and role-specific admission-failure dispositions remain open.
work controls, not a substitute cross-backend LLM ceiling.
Retire `concurrency.stage_workers.extract` along with `concurrency.total_llm`.
Migration guidance selects `concurrency.workers` independently and configures
each backend natively. Existing explicit extraction-worker values can inform the
new shared limit, but its scope now includes lane completion. Do not automatically
translate a cross-backend ceiling into per-backend limits or preserve worker
inheritance/caps from that field. Provide a native limited-backend example with
usable queue capacity, without requiring operators to override the upstream
queue default. U7 is resolved; implementation remains in Phase 4.
### Accepted-Artifact Cache
@@ -991,7 +1052,6 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
| ID | Remaining choice or dependency | Resolve for |
| --- | --- | --- |
| U1 | Review the message-content hash API/release documented upstream in v0.24.0, including its ConfigKit dependency; verify the final compatible dependency set against the selected baseline. | Track in Phase 1; required to finish cache integration in Phase 5. |
| U7 | Independent worker defaults; application-supplied native backend/queue limits, if any; retired `total_llm` migration and admission-failure handling by role. | Phase 4. |
| U8 | Cache directory configuration and reuse/bypass/refresh controls (names, defaults, scope); old-checkpoint treatment and unusable-entry diagnostics. Hash inputs, lookup timing, accepted value, and ownership are settled. | Design with Phase 3; finish in Phase 5. |
| U9 | Observer/group enrollment and application scope; concrete conditional report fields and debug filenames/serialization within the agreed contracts; publication-layout/consumer migration details preserving intrinsic artifact schemas. | Implementation details in Phases 3–5; no new receipt/manifest inventory or capture-policy decision required. |
@@ -1034,6 +1094,14 @@ Chunking falls back to one whole-transcript chunk; spell extraction uses its emp
artifact, and normalization uses safe final-response groups or merged input.
Implement module-specific construction details in Phases 3–4.
**U7 resolved:** [One pipeline worker pool and native admission](#admission-and-pipeline-workers-u7-resolved)
replace the separate worker pools and application LLM scheduler.
`concurrency.workers` defaults to `64` and bounds ready extraction/lane completion
jobs together, independently of CPU count and backend settings. PromptKit owns
backend capacity with no injected application defaults; admission failures use
U6's mode without extra retries or queues. Retire `total_llm` and the old extraction
worker field with explicit migration guidance. Implement in Phase 4.
**U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight,
output, debug policy, and coordinated cutover sequencing are selected above.
The former later CommandKit phase is retired. CLI compatibility is a non-goal;
@@ -1054,9 +1122,9 @@ major boundary changes in successor ADRs without rewriting accepted decisions.
Keep the cache dependency explicit without blocking independent work.
- Map affected ADRs, current callers, contracts, and removal inventory. Carry
forward Audita evidence without adopting its application wrappers.
- Preserve U7–U9 with their dependent phases; do not reopen resolved module
- Preserve U8–U9 with their dependent phases; do not reopen resolved module
execution, prompt/guidance ownership, retry controls/defaults, assets/schemas,
CommandKit, debug, cache, or output ownership.
worker/admission policy, CommandKit, debug, cache, or output ownership.
**Completion evidence:** reviewed dependency baseline, upstream dependency, scope,
and outstanding choices are explicit. Upstream availability alone does not
@@ -1108,7 +1176,8 @@ without the private completion/correction API.
- Migrate every LLM-backed chunker, extractor, normalizer, and validator; preserve
deterministic paths and ordered validation. Apply U4's prompt/guidance design
and U5's retry policy across roles; implement U6's module fallbacks and settle U7.
and U5's retry policy across roles; implement U6's module fallbacks and U7's
shared worker pool and native admission handling.
- Remove the adapter, copied completion contracts, manual replay/capability checks,
generic outer attempt loop, all producer/validator `retries` fields, global
scheduler/ceiling, and coupled worker settings.
@@ -1120,7 +1189,9 @@ without the private completion/correction API.
composition root and producer collaboration to the pipeline.
**Completion evidence:** every role uses the shared native engine and selected
admission policy. No production path depends on the private completion API,
admission policy. One worker limit bounds extraction and lane completion together,
including serial operation without dependency or nested-validator deadlock.
No production path depends on the private completion API,
latest-response-only correction, generic stage retries, global ceiling, or legacy
CLI infrastructure. Obsolete tests retire with the machinery; surviving domain
requirements remain.
@@ -1170,7 +1241,7 @@ Release preparation/publication is governed separately by release policy.
| Generic `runProducerAttempts` outer loop, repeated validator executions in `executeValidationChain`, all producer/validator `retries` fields, module rerun directives, hard-coded `CorrectionRequest` message builder | Ordinary deterministic execution, native LLM execution/continuation, ordered validators, validator/module-owned guidance, native domain correction templates, and domain acceptance/rejection/fallback. |
| Application `structured_output_repair_attempts` fields/resolution and explicit bundled `output.repair_attempts` | Native upstream structural-repair default, with operator customization through native prompt configuration. |
| Failure-category `validation_policy` fields and separate `fail_run`/`reject_output`/`warn_continue` actions | Strict/permissive selection, module-owned fallback construction, and explicit fallback outcomes. |
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, worker coupling | Bounded pipeline work and native backend admission. |
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, `concurrency.stage_workers.extract`, separate extraction/lane worker pools | One ready-job pool controlled by `concurrency.workers`, default `64`, and native backend admission. |
| Checkpoint stage/dependency/configuration fingerprints, invocation-wide barriers, manifests, acceptance/provenance envelopes | Small accepted-artifact cache using the upstream message hash, ordinary load checks, explicit controls. |
| Inspection wrappers, copied effective parameters, usage summaries/conversions/accumulators, debug completion wrapper, synthetic IDs, profile recorder | Native facts with application scope for diagnostics. |
| Custom dispatch/help/version/parsing, command-specific loading, copied inspection, legacy aliases/flags, parallel encoders and verbosity handling | Native declarations/loaders, pipeline handlers, domain input checks. |
@@ -1197,7 +1268,7 @@ retry tests or freeze complete help text and native debug snapshots.
| Non-LLM failure handling | Deterministic errors propagate without generic retries; rejected deterministic candidates reach terminal policy without regeneration; cancellation stops work. Producer/validator `retries` fields and the generic outer attempt loop are retired. |
| Continuation/failures | Accumulated history, full revalidation and final-turn assessment; no blanket retries beyond native exhaustion; validator failure differs from rejection; no earlier-candidate recovery or cross-turn proposal union; cancellation stops without fallback work. |
| Strict/permissive outcomes | `mode` resolves module binding, pipeline, then strict; omission inherits and either explicit value overrides. Strict is the default for every role, including normalization, without module-specific defaults. Strict failure when no acceptable result is obtained; permissive handling produces module fallback with required artifact/downstream invariants, without rerunning the ordinary full validation chain or claiming normal acceptance. Chunking fallback is one whole-transcript chunk preserving source order/references. Fallback construction failure fails execution; successful fallback use is disclosed in human/machine completion reports. |
| Admission/workers | Shared backend pools across roles, independent backends, bounded pipeline work, nested validators, queue exhaustion, and cancellation without leaked capacity or a global ceiling. |
| Admission/workers | One shared extraction/lane completion pool obeys `concurrency.workers`, default `64`; explicit `1` runs serially without dependency or nested-validator deadlock. Only ready jobs occupy slots; result ordering remains stable. Backend limits remain independent with no application-injected defaults. Producer/validator admission failures use U6's mode, preserve useful diagnostics, and add no retries; cancellation releases capacity. No separate global LLM ceiling. |
| Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss. Failed/rejected/incomplete work is not stored; fallbacks and affected downstream work skip lookup/storage while unaffected accepted predecessors remain reusable; no required history. |
| Cache migration | Selected reuse/bypass/refresh and old-record treatment work; invocation configuration no longer blocks lookup; canonical chunk caching retains its distinct policy. |
| Output/outcomes | Artifact files independent of report format; native text/JSON/YAML summary and qualified outcomes; no mandatory metadata bundle; quiet suppression, verbose stderr, debug path; native nonzero failures and useful explanations. Publication survives later capture/report failure without model reruns. |
@@ -1230,8 +1301,12 @@ Provide practical release guidance for operators and subprocess callers:
- Version 6 and native resources, embedded-only versus companion examples in
appropriate directories, imports, qualified IDs, replacement/inheritance,
native local backends, removed catalogs/versions, and separate source versions.
- Native admission and worker defaults, removed global limit, `max_additional_turns`
default/precedence/zero behavior, and intent-based migration from LLM `retries`.
- Shared `concurrency.workers` default `64`, ready-job scope, explicit serial
execution, native backend concurrency/queue settings, and removal of the global
limit and old extraction-worker field without automatic backend translation.
Explain strict/permissive handling of producer and validator admission failures.
- `max_additional_turns` default/precedence/zero behavior, and intent-based
migration from LLM `retries`.
Explain retirement of all producer/validator `retries` fields and generic non-LLM
retries, upstream-owned structural defaults, removal of application repair
controls and bundled counts, native operator customization, accumulated-history