Record agreement to retire the global LLM scheduler

This commit is contained in:
2026-11-18 20:04:48 +00:00
parent ae8e3ec698
commit 40cd4fe97d

View File

@@ -5,7 +5,7 @@
Working notepad for reviewing the ownership boundaries in the
[PromptKit v0.23.2 adoption roadmap](promptkit.md). Work through the topics
below in order, recording questions, evidence, decisions, and follow-up work.
Items 1–3 record selected architectural directions. Detailed design,
Items 1–4 record selected architectural directions. Detailed design,
read-only filesystem and schema decisions, and the remaining topics are still
under discussion.
Item 7 records the completed Audita comparison and its implications for the
@@ -19,8 +19,9 @@ features where PromptKit does not yet meet an application need.
No application changes have been implemented through this notepad. The adoption
roadmap still needs reconciliation with the selected direction, including its
adapter-preservation and latest-response-only language. Significant changes to
accepted architecture need successor ADRs under the
adapter-preservation, latest-response-only, and cross-backend scheduler
preservation language. Significant changes to accepted architecture need
successor ADRs under the
[documentation policy](../policy/documentation.md#architecture-decision-records).
Current boundaries are described in [Architecture](../policy/architecture.md)
and [LLM Runtime Internals](../internal/llm.md).
@@ -273,21 +274,75 @@ the retired message-reconstruction protocol rather than preserving its mechanics
## 4. Global Scheduling And Admission
Consider whether backend capacity limits and pipeline worker bounds meet the
operational need, or whether a separate cross-backend ceiling remains useful.
If it remains useful, consider implementing that reusable mechanism upstream.
**Discussion / decision:** Direction agreed: remove Notarius's LLM scheduler and
retire the separate cross-backend ceiling. Rely on PromptKit's native backend
admission and capacity limits while keeping bounded Notarius pipeline workers.
This deliberately changes the existing global-limit contract; reproducing that
ceiling in PromptKit is not a migration prerequisite. No application changes
have been implemented.
Questions:
### Agreed Ownership
- What resource or operational budget does each limit protect?
- Should a global permit remain occupied during backend queueing and repair?
- Can the Notarius scheduler be removed, or should PromptKit gain engine-wide
admission support?
| Concern | Owner |
| --- | --- |
| Bounded extraction jobs and lane continuations | Notarius pipeline workers |
| Backend admission, active model attempts, queues, cancellation, and permit release | PromptKit |
| Application disposition after admission or execution failure | Notarius application policy |
**Discussion / decision:** Pending.
For limited backends, PromptKit v0.23.2 bounds active model attempts by
`concurrency_limit` and admitted operations by that limit plus `queue_capacity`.
An operation holds admission
across preparation, retries, and structural repair, while active permits cover
individual model attempts and are released during retry waits. A completed
continuation turn releases admission before application assessment, so its
callback can invoke an LLM validator using the same backend.
**Follow-up:** Account for the existing global-limit contract. PromptKit
v0.23.2's per-backend limits do not directly replace it.
The current Notarius wrapper holds a global permit around the complete
structured call, including backend waiting and repairs. Removing it eliminates
that additional queue and avoids making independent backends share those
permits. Independent native backend limits may allow aggregate concurrency above
the former global ceiling; that is an accepted behavior change. Pipeline worker
bounds remain application work controls, not a replacement global LLM ceiling.
### Removal And Configuration Consequences
- Remove `Scheduler`, `scheduledClient`, their production wiring, and custom
queue/cancellation/permit bookkeeping.
- Retire `concurrency.total_llm`. Give pipeline worker settings an independent
default and remove the extraction-worker maximum derived from `total_llm`.
- Use one shared execution engine for producers, validators, and repair turns.
Native capacity belongs to an engine and selected backend ID; separate engine
instances have independent pools.
- Configure native limits at the intended backend capacity boundary. Profiles
selecting the same backend ID share its pool; separate IDs pointing to the
same endpoint do not automatically share capacity. Endpoint-only profiles and
backends without a positive limit remain unrestricted by backend admission.
- Handle native `ErrCapacityExceeded` as an admission failure through application
policy, consistently with item 3's removal of blanket execution retries.
- Retire scheduler implementation tests with the removed code. Retain focused
integration checks for shared native admission across roles, independent
backends, bounded pipeline work, queue exhaustion, and cancellation.
### Remaining Details
Select independent worker defaults, native backend/queue defaults where the
application supplies them, and the configuration migration treatment of the
retired field. Specify how admission failures are reported and handled for each
application role. These details do not reopen the selected ownership boundary.
A future demonstrated need for capacity shared across several backend IDs could
justify an upstream PromptKit feature request. No such feature is required for
this agreed design, and it does not justify retaining the Notarius scheduler.
Evidence: [scheduled client](../../internal/framework/llm/scheduled_client.go),
[scheduler](../../internal/framework/llm/scheduler.go),
[pipeline workers](../../internal/framework/pipeline/runner_concurrent.go), and
[native capacity behavior](../../../promptkit/docs/internal/capacity.md).
**Follow-up:** Reconcile the adoption roadmap's scheduler-preservation language
and record the changed ownership in an ADR. Update architecture, configuration,
operations, and implementation documentation when the migration lands. Continue
with item 5's execution identity and checkpoint reuse discussion.
## 5. Execution Identity And Checkpoint Reuse
@@ -452,8 +507,8 @@ and [prompt guidance](../../../audita/docs/internal/prompts.md).
adapter-removal direction remains in force.
- Proposal, validator, and repair roles share one engine and its per-backend
admission. The runner separately bounds section work. This demonstrates useful
separation of application work scheduling and provider capacity, but does not
supply Notarius's independent cross-backend ceiling.
separation of application work scheduling and provider capacity. Item 4 now
selects that ownership and retires Notarius's independent cross-backend ceiling.
- Ordinary calls use `Run`, `OperationTimeout`, and `OnPrepared`. Observations
feed a native `ExecutionGroup`; application records add logical-call and
stage/section/proposal attribution. Nested validator executions enroll
@@ -485,8 +540,9 @@ conversation assembly or the turn loop local. Item 3 owns the agreed Notarius
flow and the remaining repair-prompt, response-contract, and policy details;
the comparison alone did not settle those details.
Items 4 and 6 can draw on Audita's native admission, observations, and CommandKit
integration. Item 5 remains independent upstream/application identity work.
Item 4 now records agreement to use native admission with bounded application
workers. Item 6 can draw on Audita's observations and CommandKit integration.
Item 5 remains independent upstream/application identity work.
The two requested capabilities already work with v0.23.2; no new PromptKit
feature is needed merely to provide replaceable defaults or domain-directed
continuation. Any remaining upstream request should identify the specific unmet