Resolve shared worker pool and native admission policy
This commit is contained in:
@@ -196,8 +196,9 @@ a singleton flag and use the last successfully parsed value.
|
||||
| Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. |
|
||||
| Capture destination | Application `--debug-dir PATH`, with the semantics below. |
|
||||
|
||||
Keep worker counts and semantic budgets in configuration, adjustable through
|
||||
application `--set`. U5 selects `max_additional_turns`; worker defaults remain U7.
|
||||
Keep the worker limit and semantic budgets in configuration, adjustable through
|
||||
application `--set`. U5 selects `max_additional_turns`; U7 selects
|
||||
`concurrency.workers` with a default of `64`.
|
||||
Use ckprompt's reusable profile, session, and execution flags, including model,
|
||||
temperature, timeout, reasoning effort, and transient retries, for run-wide
|
||||
overrides across producers, validators, and continuations. Omission preserves
|
||||
@@ -280,7 +281,7 @@ Retire `promptkit.profile_file`, `profile_dir`, and `local_backend` in favor of
|
||||
native companion sources; do not introduce a separate `promptkit.config_file`.
|
||||
Retire `concurrency.total_llm` and revise retry/worker settings as described below.
|
||||
U5 selects `max_additional_turns` and removes application structural-repair
|
||||
controls; worker defaults and retired global-limit migration remain U7.
|
||||
controls; U7 selects the shared worker pool and retired global-limit migration.
|
||||
|
||||
Use one source assembly for runtime, preflight, and inspection:
|
||||
|
||||
@@ -321,8 +322,8 @@ and its reference to the bundled unqualified `openrouter` backend. Inherited
|
||||
application profiles use sibling local IDs in the same collection. The bundled
|
||||
inventory differs from the old external catalogs and selects no model for an
|
||||
application request; document intentional replacements for removed catalog IDs.
|
||||
The bundled backend supplies no concurrency limit. Application/operator limits
|
||||
need explicit native configuration; application-supplied defaults remain U7.
|
||||
The bundled backend supplies no concurrency limit. Under U7, Notarius injects no
|
||||
backend concurrency or queue defaults; operators use native backend configuration.
|
||||
|
||||
Preserve prompt prose, message ordering, roles, and cache boundaries where their
|
||||
meaning carries forward. Review and deliberately revise initial/repair prompts
|
||||
@@ -667,12 +668,39 @@ conversation state in process, outside ordinary artifact/cache payloads;
|
||||
explicitly requested detailed debug is separate. U6 is resolved; implementation
|
||||
and module-specific construction details remain in Phases 3–4.
|
||||
|
||||
### Admission And Pipeline Workers
|
||||
### Admission And Pipeline Workers (U7 Resolved)
|
||||
|
||||
Remove `Scheduler`, `scheduledClient`, and custom global queue/permit plumbing.
|
||||
Retire the separate cross-backend ceiling and the default/maximum extraction
|
||||
worker coupling to `concurrency.total_llm`. Keep bounded extraction workers and
|
||||
bounded lane continuations with independent defaults to be selected.
|
||||
worker coupling to `concurrency.total_llm`. Use one shared pool for ready pipeline
|
||||
jobs, configured by the positive integer `concurrency.workers`, default `64`.
|
||||
Omission uses the default; explicit `1` provides serial job execution. Keep this
|
||||
in application configuration, adjustable through `--set`, without a dedicated
|
||||
CLI flag or separate extraction/lane worker settings.
|
||||
|
||||
The pool serves both kinds of work:
|
||||
|
||||
| Job | Work covered by one worker slot |
|
||||
| --- | --- |
|
||||
| Extraction | One chunk-and-lane extraction, including its validation and semantic correction. |
|
||||
| Lane completion | A ready lane's merge and normalization, including their validation and correction. |
|
||||
|
||||
The limit counts total active jobs across both kinds, with no separate reserved
|
||||
capacity. The existing pipeline coordinator tracks dependencies and makes lane
|
||||
completion ready after that lane's extraction finishes. A worker never holds a
|
||||
slot while waiting for prerequisite jobs. Validators and correction turns execute
|
||||
within the current job instead of acquiring another worker slot. Preserve stable
|
||||
public result ordering regardless of completion order. Input preparation, chunking,
|
||||
and final publication retain their sequential placement around lane execution.
|
||||
This remains execution of the fixed pipeline, not a general workflow scheduler.
|
||||
|
||||
Count application jobs, not OS threads or all process goroutines. Do not derive
|
||||
the default from CPU count or `GOMAXPROCS`; model-backed jobs often wait for I/O
|
||||
or native admission. The pool bounds active application state and preparation as
|
||||
well as model-backed work. `64` supplies practical headroom for backend utilization
|
||||
while remaining well below the reviewed native default queue capacity; it is not
|
||||
a benchmark-derived optimum or a guarantee of backend saturation. Operators may
|
||||
tune it for workload and host resources independently of backend settings.
|
||||
|
||||
Use one shared native engine across all roles. Native capacity belongs to the
|
||||
engine and backend ID: profiles selecting the same ID share its pool; different
|
||||
@@ -680,6 +708,15 @@ backend IDs or separate engines do not automatically share limits, even for the
|
||||
same endpoint. Endpoint-only profiles and backends without a positive limit
|
||||
remain unrestricted by native backend admission.
|
||||
|
||||
Inject no Notarius-owned backend concurrency or queue defaults. Preserve native
|
||||
operator configuration and inherit upstream defaults rather than deriving backend
|
||||
settings from the worker limit. In the reviewed PromptKit baseline, omitted/zero
|
||||
backend concurrency is unlimited and has no queue; with positive concurrency,
|
||||
omitted queue capacity defaults to `1024`. Explicit queue capacity, including zero,
|
||||
uses native semantics. These are upstream defaults, not values pinned by Notarius.
|
||||
With an unlimited backend, active jobs can generate concurrent calls up to the
|
||||
application's workload; the worker default does not imply a separate LLM ceiling.
|
||||
|
||||
For limited backends, the reviewed baseline bounds active model attempts by
|
||||
`concurrency_limit` and admitted operations by that limit plus `queue_capacity`.
|
||||
Ordinary execution admission spans preparation, retries, and repairs; split
|
||||
@@ -687,12 +724,36 @@ Ordinary execution admission spans preparation, retries, and repairs; split
|
||||
`RunPrepared`. Active permits are released during retry waits.
|
||||
A completed continuation turn releases admission before assessment, allowing a
|
||||
nested validator to use the same backend. Configure native limits at the intended
|
||||
capacity boundary and handle `ErrCapacityExceeded` through application policy.
|
||||
capacity boundary. Worker counts need not be less than backend concurrency;
|
||||
native admission can queue waiting operations.
|
||||
|
||||
Handle `ErrCapacityExceeded` through U6's existing strict/permissive policy:
|
||||
|
||||
- A producer admission failure fails in strict mode or uses the module's
|
||||
supported fallback in permissive mode.
|
||||
- A validator admission failure means assessment is unavailable; apply the
|
||||
producing module's resolved mode. Do not turn it into a semantic rejection
|
||||
or a correction instruction.
|
||||
- Cancellation stops work and does not initiate fallback.
|
||||
|
||||
Preserve the error and backend identity in diagnostics. Add no application retry,
|
||||
backoff, or second admission queue. A full native queue rejects admission without
|
||||
calling the model; the large default queue makes this unusual with modest worker
|
||||
counts, but explicit operator limits remain authoritative. Use ordinary fallback
|
||||
reporting and cache exclusions when permissive handling continues.
|
||||
|
||||
Aggregate concurrency across independent backends may exceed the former global
|
||||
ceiling. That change is intentional. Bounded pipeline workers are application
|
||||
work controls, not a substitute cross-backend LLM ceiling. Defaults, retired-field
|
||||
migration, and role-specific admission-failure dispositions remain open.
|
||||
work controls, not a substitute cross-backend LLM ceiling.
|
||||
|
||||
Retire `concurrency.stage_workers.extract` along with `concurrency.total_llm`.
|
||||
Migration guidance selects `concurrency.workers` independently and configures
|
||||
each backend natively. Existing explicit extraction-worker values can inform the
|
||||
new shared limit, but its scope now includes lane completion. Do not automatically
|
||||
translate a cross-backend ceiling into per-backend limits or preserve worker
|
||||
inheritance/caps from that field. Provide a native limited-backend example with
|
||||
usable queue capacity, without requiring operators to override the upstream
|
||||
queue default. U7 is resolved; implementation remains in Phase 4.
|
||||
|
||||
### Accepted-Artifact Cache
|
||||
|
||||
@@ -991,7 +1052,6 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
|
||||
| ID | Remaining choice or dependency | Resolve for |
|
||||
| --- | --- | --- |
|
||||
| U1 | Review the message-content hash API/release documented upstream in v0.24.0, including its ConfigKit dependency; verify the final compatible dependency set against the selected baseline. | Track in Phase 1; required to finish cache integration in Phase 5. |
|
||||
| U7 | Independent worker defaults; application-supplied native backend/queue limits, if any; retired `total_llm` migration and admission-failure handling by role. | Phase 4. |
|
||||
| U8 | Cache directory configuration and reuse/bypass/refresh controls (names, defaults, scope); old-checkpoint treatment and unusable-entry diagnostics. Hash inputs, lookup timing, accepted value, and ownership are settled. | Design with Phase 3; finish in Phase 5. |
|
||||
| U9 | Observer/group enrollment and application scope; concrete conditional report fields and debug filenames/serialization within the agreed contracts; publication-layout/consumer migration details preserving intrinsic artifact schemas. | Implementation details in Phases 3–5; no new receipt/manifest inventory or capture-policy decision required. |
|
||||
|
||||
@@ -1034,6 +1094,14 @@ Chunking falls back to one whole-transcript chunk; spell extraction uses its emp
|
||||
artifact, and normalization uses safe final-response groups or merged input.
|
||||
Implement module-specific construction details in Phases 3–4.
|
||||
|
||||
**U7 resolved:** [One pipeline worker pool and native admission](#admission-and-pipeline-workers-u7-resolved)
|
||||
replace the separate worker pools and application LLM scheduler.
|
||||
`concurrency.workers` defaults to `64` and bounds ready extraction/lane completion
|
||||
jobs together, independently of CPU count and backend settings. PromptKit owns
|
||||
backend capacity with no injected application defaults; admission failures use
|
||||
U6's mode without extra retries or queues. Retire `total_llm` and the old extraction
|
||||
worker field with explicit migration guidance. Implement in Phase 4.
|
||||
|
||||
**U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight,
|
||||
output, debug policy, and coordinated cutover sequencing are selected above.
|
||||
The former later CommandKit phase is retired. CLI compatibility is a non-goal;
|
||||
@@ -1054,9 +1122,9 @@ major boundary changes in successor ADRs without rewriting accepted decisions.
|
||||
Keep the cache dependency explicit without blocking independent work.
|
||||
- Map affected ADRs, current callers, contracts, and removal inventory. Carry
|
||||
forward Audita evidence without adopting its application wrappers.
|
||||
- Preserve U7–U9 with their dependent phases; do not reopen resolved module
|
||||
- Preserve U8–U9 with their dependent phases; do not reopen resolved module
|
||||
execution, prompt/guidance ownership, retry controls/defaults, assets/schemas,
|
||||
CommandKit, debug, cache, or output ownership.
|
||||
worker/admission policy, CommandKit, debug, cache, or output ownership.
|
||||
|
||||
**Completion evidence:** reviewed dependency baseline, upstream dependency, scope,
|
||||
and outstanding choices are explicit. Upstream availability alone does not
|
||||
@@ -1108,7 +1176,8 @@ without the private completion/correction API.
|
||||
|
||||
- Migrate every LLM-backed chunker, extractor, normalizer, and validator; preserve
|
||||
deterministic paths and ordered validation. Apply U4's prompt/guidance design
|
||||
and U5's retry policy across roles; implement U6's module fallbacks and settle U7.
|
||||
and U5's retry policy across roles; implement U6's module fallbacks and U7's
|
||||
shared worker pool and native admission handling.
|
||||
- Remove the adapter, copied completion contracts, manual replay/capability checks,
|
||||
generic outer attempt loop, all producer/validator `retries` fields, global
|
||||
scheduler/ceiling, and coupled worker settings.
|
||||
@@ -1120,7 +1189,9 @@ without the private completion/correction API.
|
||||
composition root and producer collaboration to the pipeline.
|
||||
|
||||
**Completion evidence:** every role uses the shared native engine and selected
|
||||
admission policy. No production path depends on the private completion API,
|
||||
admission policy. One worker limit bounds extraction and lane completion together,
|
||||
including serial operation without dependency or nested-validator deadlock.
|
||||
No production path depends on the private completion API,
|
||||
latest-response-only correction, generic stage retries, global ceiling, or legacy
|
||||
CLI infrastructure. Obsolete tests retire with the machinery; surviving domain
|
||||
requirements remain.
|
||||
@@ -1170,7 +1241,7 @@ Release preparation/publication is governed separately by release policy.
|
||||
| Generic `runProducerAttempts` outer loop, repeated validator executions in `executeValidationChain`, all producer/validator `retries` fields, module rerun directives, hard-coded `CorrectionRequest` message builder | Ordinary deterministic execution, native LLM execution/continuation, ordered validators, validator/module-owned guidance, native domain correction templates, and domain acceptance/rejection/fallback. |
|
||||
| Application `structured_output_repair_attempts` fields/resolution and explicit bundled `output.repair_attempts` | Native upstream structural-repair default, with operator customization through native prompt configuration. |
|
||||
| Failure-category `validation_policy` fields and separate `fail_run`/`reject_output`/`warn_continue` actions | Strict/permissive selection, module-owned fallback construction, and explicit fallback outcomes. |
|
||||
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, worker coupling | Bounded pipeline work and native backend admission. |
|
||||
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, `concurrency.stage_workers.extract`, separate extraction/lane worker pools | One ready-job pool controlled by `concurrency.workers`, default `64`, and native backend admission. |
|
||||
| Checkpoint stage/dependency/configuration fingerprints, invocation-wide barriers, manifests, acceptance/provenance envelopes | Small accepted-artifact cache using the upstream message hash, ordinary load checks, explicit controls. |
|
||||
| Inspection wrappers, copied effective parameters, usage summaries/conversions/accumulators, debug completion wrapper, synthetic IDs, profile recorder | Native facts with application scope for diagnostics. |
|
||||
| Custom dispatch/help/version/parsing, command-specific loading, copied inspection, legacy aliases/flags, parallel encoders and verbosity handling | Native declarations/loaders, pipeline handlers, domain input checks. |
|
||||
@@ -1197,7 +1268,7 @@ retry tests or freeze complete help text and native debug snapshots.
|
||||
| Non-LLM failure handling | Deterministic errors propagate without generic retries; rejected deterministic candidates reach terminal policy without regeneration; cancellation stops work. Producer/validator `retries` fields and the generic outer attempt loop are retired. |
|
||||
| Continuation/failures | Accumulated history, full revalidation and final-turn assessment; no blanket retries beyond native exhaustion; validator failure differs from rejection; no earlier-candidate recovery or cross-turn proposal union; cancellation stops without fallback work. |
|
||||
| Strict/permissive outcomes | `mode` resolves module binding, pipeline, then strict; omission inherits and either explicit value overrides. Strict is the default for every role, including normalization, without module-specific defaults. Strict failure when no acceptable result is obtained; permissive handling produces module fallback with required artifact/downstream invariants, without rerunning the ordinary full validation chain or claiming normal acceptance. Chunking fallback is one whole-transcript chunk preserving source order/references. Fallback construction failure fails execution; successful fallback use is disclosed in human/machine completion reports. |
|
||||
| Admission/workers | Shared backend pools across roles, independent backends, bounded pipeline work, nested validators, queue exhaustion, and cancellation without leaked capacity or a global ceiling. |
|
||||
| Admission/workers | One shared extraction/lane completion pool obeys `concurrency.workers`, default `64`; explicit `1` runs serially without dependency or nested-validator deadlock. Only ready jobs occupy slots; result ordering remains stable. Backend limits remain independent with no application-injected defaults. Producer/validator admission failures use U6's mode, preserve useful diagnostics, and add no retries; cancellation releases capacity. No separate global LLM ceiling. |
|
||||
| Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss. Failed/rejected/incomplete work is not stored; fallbacks and affected downstream work skip lookup/storage while unaffected accepted predecessors remain reusable; no required history. |
|
||||
| Cache migration | Selected reuse/bypass/refresh and old-record treatment work; invocation configuration no longer blocks lookup; canonical chunk caching retains its distinct policy. |
|
||||
| Output/outcomes | Artifact files independent of report format; native text/JSON/YAML summary and qualified outcomes; no mandatory metadata bundle; quiet suppression, verbose stderr, debug path; native nonzero failures and useful explanations. Publication survives later capture/report failure without model reruns. |
|
||||
@@ -1230,8 +1301,12 @@ Provide practical release guidance for operators and subprocess callers:
|
||||
- Version 6 and native resources, embedded-only versus companion examples in
|
||||
appropriate directories, imports, qualified IDs, replacement/inheritance,
|
||||
native local backends, removed catalogs/versions, and separate source versions.
|
||||
- Native admission and worker defaults, removed global limit, `max_additional_turns`
|
||||
default/precedence/zero behavior, and intent-based migration from LLM `retries`.
|
||||
- Shared `concurrency.workers` default `64`, ready-job scope, explicit serial
|
||||
execution, native backend concurrency/queue settings, and removal of the global
|
||||
limit and old extraction-worker field without automatic backend translation.
|
||||
Explain strict/permissive handling of producer and validator admission failures.
|
||||
- `max_additional_turns` default/precedence/zero behavior, and intent-based
|
||||
migration from LLM `retries`.
|
||||
Explain retirement of all producer/validator `retries` fields and generic non-LLM
|
||||
retries, upstream-owned structural defaults, removal of application repair
|
||||
controls and bundled counts, native operator customization, accumulated-history
|
||||
|
||||
Reference in New Issue
Block a user