Resolve native prompt asset and schema design

This commit is contained in:
2026-11-20 12:45:00 +00:00
parent 2eaea2b1fd
commit 0833450c71
3 changed files with 115 additions and 93 deletions

View File

@@ -21,10 +21,11 @@ planning work. Implementation remains pending.
The roadmap resolves U10's CommandKit scope, tracks the pending upstream
message-content hash in U1, replaces U8's checkpoint design, and narrows U9 to
implementation details within the agreed output/debug contracts. Schema handling,
role-specific correction/fallback policy, worker defaults, and concrete cache
controls/transition remain visible there. Item-level directions below record the
accepted design; the roadmap is the authority for sequencing and current status.
implementation details within the agreed output/debug contracts. U2 now resolves
asset and schema assembly. Role-specific correction/fallback policy, worker
defaults, and concrete cache controls/transition remain visible there. Item-level
directions below record the accepted design; the roadmap is the authority for
sequencing and current status.
## Agreed Proposal
@@ -1213,8 +1214,9 @@ separate Phase 7 is retired.
Its dependency baseline includes the selected versions and pending upstream hash
request. U10 is resolved; U8 now concerns cache operation/transition, and U9
concerns implementation details within the selected output/debug contracts.
Earlier receipt/manifest inventories are retired. Remaining schema, correction,
fallback, worker, and cache-transition choices are explicit in its
Earlier receipt/manifest inventories are retired, and U2 resolves asset/schema
assembly. Remaining correction, fallback, worker, and cache-transition choices
are explicit in its
[unresolved-choice table](promptkit.md#unresolved-choices).
The master roadmap owns sequencing and implementation status. This notepad and

View File

@@ -144,7 +144,7 @@ not require preserving a private completion interface.
construction, interpretation, configured validation, and terminal disposition.
- Choose the location of engine assembly and focused application helpers without
introducing a replacement LLM service layer.
- Resolve item 2's deferred filesystem and schema choices and the remaining
- Implement item 2's resolved asset/schema design and settle the remaining
migration details recorded under items 3–6, including durable compatibility.
- Evaluate any remaining upstream gap against an actual application requirement,
rather than preserving historical machinery by default.
@@ -158,70 +158,40 @@ outcomes as the meaningful Notarius-owned boundaries.
## 2. Prompt Assets And Schema Assembly
**Discussion / decision:** Direction agreed: use native PromptKit declarations
as the sole prompt assembly mechanism and remove the duplicate application
assembly plumbing. Resolution of the custom read-only filesystem and schemas
is explicitly deferred; the next topic may clarify those choices.
**Discussion / decision:** Agreed and incorporated as resolved U2 in the
[master roadmap](promptkit.md#prompt-assets-and-response-schemas-u2-resolved).
The filesystem and schema questions initially deferred here are now settled.
Implementation is pending; the master roadmap owns the detailed design and checks.
**Selected direction:** Keep the embedded content tree and load its prompt and
profile definitions as native PromptKit configuration in the application
collection. Use native filesystem contexts and imports to reference shared
content directly. Application composition assembles the embedded and operator
sources once; modules select prompt IDs, supply domain inputs, and interpret
typed results.
Use ordinary embedded assets, native prompt declarations, and native schema
loading. Keep the content tree and module ownership of prompt meaning and domain
interpretation. Shared files use real native filesystem paths. Remove duplicate
asset registration, per-module manifests, synthetic mounts, flattened schema
namespaces, and the custom read-only filesystem that served that assembly.
The spell extractor illustrates the current duplication: its prompt YAML
declares ordered messages and a schema, while a separate Go asset manifest
lists files, mounts shared fragments, registers filesystems, and hashes them.
The selected direction makes native declarations the sole prompt assembly source.
Prompt definitions select output contracts by default. Initial and correction
prompts can share schema files. Use native per-request replacement only for a
demonstrated runtime-dependent contract, without an application wrapper or schema
registry. Audita demonstrates ordinary embedded schema loading and per-call
contracts; its per-call selection is not required for every Notarius operation.
Candidates for removal:
The default schema source is the embedded asset tree. Operator-selected external
schema directories use native whole-root replacement, not an application overlay.
Prompt underlays do not implicitly overlay schema files. PromptKit owns private
response-schema loading, structural checks, and repair; Notarius keeps typed
interpretation, domain validation, and distinct durable artifact schemas/codecs.
- Prompt and fallback-profile registration plumbing in `AssetRegistry` and
individual modules; schema responsibilities remain under review.
- `ModulePromptFS`, `PromptAssetManifest`, shared-name lookup tables, synthetic
`sharedassets` mounts.
- Manual prompt-file enumeration for checkpoint fingerprints, retired under
item 5's simpler reuse policy. Native prompt hashes remain available for
inspection and diagnostics.
Retire response-schema hashes and copied metadata used only by the adapter,
checkpoints, or reporting. Preserve actual artifact contracts and native diagnostic
snapshots. Neither asset nor schema metadata adds another cache fingerprint.
Keep prompt prose, ordered messages, cache controls, workload profiles, private
response schema documents, and domain interpretation. Durable artifact schemas
remain separate. Content ownership does not require runtime asset registration.
The spell extractor illustrated the duplication: prompt YAML declared ordered
messages and a schema while a Go manifest separately listed, mounted, registered,
and hashed assets. Native declarations now supply the selected assembly model.
Native schema paths resolve against the configured schema root, separately from
message-file source contexts. PromptKit v0.23.2 loads schemas during preparation
and requires self-contained schema documents. A captured prompt hash alone does
not establish schema equality. Existing schema metadata in manifests and
checkpoint inputs must be deliberately replaced or retired; its presence does
not require preserving the complete wrapper system.
Deferred decisions:
- Whether the custom read-only filesystem can be removed after reviewing its
remaining responsibilities.
- Schema assembly, root/path selection, wrapper removal, and schema metadata.
Supplying the embedded filesystem as a native schema root remains a candidate,
not a selected design.
- Schema metadata for artifact compatibility and inspection. Item 5 settles that
cache matching does not require exact schema identity; the CommandKit output
decision removes mandatory manifests.
Implementation questions:
- Can native sources represent the existing shared fragments and module assets?
- Which registry, remounting, and hashing helpers could disappear?
- What schema metadata remains useful for inspection and artifact loading under
item 5's simpler reuse policy?
- How do we preserve message ordering and cache controls while simplifying
assembly?
**Follow-up:** Continue with item 3, then revisit the deferred filesystem and
schema decisions with any new clarity. Confirm native source contexts for shared
fragments and operator replacements. Apply item 5's reuse policy and resolve
remaining metadata with item 6. Verify the resulting definitions through native
preparation and fake-provider execution; filesystem remounting tests can retire
with the removed implementation. The agreed direction has not yet been implemented.
**Follow-up:** Implement the master roadmap's Phase 2 and retire tests of removed
mounting/registry machinery. Verify shared paths, operator replacements, native
schema sources, and any necessary complete request-contract replacement.
## 3. Retry And Semantic Correction Ownership
@@ -335,8 +305,8 @@ Notarius's serialized schema validator currently receives the codec-produced
artifact and its artifact schema, which can differ from the model response.
Do not remove that boundary merely because both use JSON Schema. Audit actual
duplicate response checks individually; artifact validation and Go/domain
interpretation are not automatically redundant. Item 2's filesystem and schema
assembly decisions remain open.
interpretation are not automatically redundant. Item 2's resolved native asset
and schema design preserves this distinction.
Evidence: [producer attempt loop](../../internal/framework/pipeline/producer_attempts.go),
[validation executor](../../internal/framework/pipeline/validation_executor.go),
@@ -355,9 +325,8 @@ and the native continuation evidence in item 7.
candidate may survive a later failed turn.
- Failure handling for non-LLM operations and any justified recovery beyond
native execution budgets.
- Item 2's deferred filesystem and schema assembly choices, item 5's remaining
cache implementation details, and migration to the selected CommandKit output
contract.
- Item 5's remaining cache implementation details and migration to the selected
CommandKit output contract.
**Follow-up:** Resolve the remaining details above, then reconcile the adoption
roadmap. Record the changed architecture in a successor to
@@ -825,7 +794,8 @@ Evidence: [LLM runtime](../../../audita/docs/internal/llm-runtime.md),
Item 2's native underlay direction has a working application precedent. Audita
also offers evidence for ordinary embedded filesystems and per-request schema
selection, while our filesystem and schema decisions remain open.
selection. U2 now selects ordinary embedded assets and prompt-declared contracts
by default, reserving per-request replacement for demonstrated runtime needs.
The subsequent item 3 discussion selects native continuation and accumulated
conversation history. Keeping domain validation local does not require keeping

View File

@@ -240,8 +240,8 @@ content tree and module ownership of prompt meaning, response requirements, and
domain interpretation. Remove prompt/fallback-profile registration in
`AssetRegistry`, per-module `PromptAssetManifest` plumbing, `ModulePromptFS`,
shared-name maps, synthetic `sharedassets` mounts, and manual prompt enumeration
for checkpoint fingerprints. Schema-related registry responsibilities remain
subject to the explicit schema decision below.
for checkpoint fingerprints. The resolved schema design below also removes
schema registration and filesystem-flattening machinery.
Convert maintained resources, examples, and fixtures to native identified lists,
`config:` source controls, and sibling inheritance (`extends: {id: parent}`).
@@ -268,19 +268,61 @@ for accumulated conversation history; preserving every old byte or retry message
is not a migration requirement. Shared content should use native filesystem
contexts and imports rather than reconstructed virtual trees.
PromptKit owns private model-response validation and structural repair. Native
schemas resolve against the configured schema root, separately from message-file
contexts. The reviewed baseline requires self-contained schemas loaded during
preparation.
A request override replaces the complete output contract; it must retain the
intended schema, validation mode, and repair policy. A prompt hash alone does not
establish schema equality.
#### Prompt Assets And Response Schemas (U2 Resolved)
The custom read-only filesystem, schema root/path selection, module-supplied
request contracts versus prompt-declared contracts, wrapper removal, and useful
schema metadata remain open. An ordinary embedded schema filesystem, as used by
Audita, is a candidate rather than a selected requirement. Durable artifact
schemas and their validators remain distinct from private response schemas.
Use the existing root `assets` package's ordinary `embed.FS` directly for native
prompt sources and response schemas, with `fs.Sub` only where useful. Shared
message files use their actual paths within the native filesystem context.
Remove synthetic mounts, file copying, flattened schema namespaces, and module
asset manifests. Retire `readonlyfs` when its prompt-mounting and asset-flattening
consumers disappear; no replacement filesystem abstraction is needed.
Declare response format, validation mode, schema path, and structural-repair
budget in native prompt definitions by default. Modules select prompt IDs, supply
inputs, and interpret responses. Initial and correction prompts may reference
the same schema file. Execution, inspection, and prepared-request preview then
use the same declared contract without repeating schema selection in Go.
Role-specific prompt contents remain U4, and retry defaults remain U5.
Use a request-level native `OutputContract` only for a demonstrated runtime-
dependent contract. It replaces the whole configured contract, so supply the
intended format, validation mode, schema, and repair policy together. Do not
introduce a wrapper or per-call registry solely to reproduce Audita's approach.
Operator prompt replacements must still produce responses the selected module
can interpret; that module boundary does not require duplicating each prompt's
schema selection in Go.
Default to native `WithSchemaFS(assets.FS(), ".")`. Schema paths are relative to
that selected root, for example `dnd/spells/schemas/dnd_spells_llm.v1.json`.
They are separate from message-file source contexts: prompt underlays do not
implicitly overlay schema files. When an operator configures an external schema
directory, select PromptKit's native directory source instead of the embedded
root. It replaces the whole schema source and must supply the schemas used by
the selected prompts. Do not build an application filesystem overlay to mix
embedded and external schemas. Transparent layering would be a separate upstream
capability, not a requirement for this migration. Concrete configuration binding
is an implementation detail within this selected source policy.
Keep private response schemas as self-contained JSON documents. PromptKit owns
loading, compilation, structural validation, provider structured-output settings,
and structural repair. Remove Notarius's response-schema byte registry, copied
schema wrappers, defensive-copy layer, digest calculation, and duplicate
structural-validation machinery. Native prepared snapshots provide diagnostic
schema details; no schema fingerprint is added to the message-content cache key.
Keep typed decoding, source/reference checks, reconciliation safety, semantic
validators, artifact codecs, and durable artifact schemas in Notarius. Response
schemas describe model output; artifact schemas describe published and consumed
domain artifacts. Preserve identifiers and versions belonging to actual artifact
contracts. Retire private schema keys, registration records, hashes, and copied
identity/version/name metadata whose only consumers were adapters, checkpoints,
or reporting. No independent schema metadata system replaces them.
Implementation is pending. Verify native preparation of embedded and operator
prompt definitions, shared-file paths, schema selection from embedded and external
roots, and complete request-contract replacement where actually used. Missing or
invalid schemas should follow native preparation behavior. No new schema check
requiring synthetic requests is added to offline pipeline preflight.
### Native Production, Continuation, And Assessment
@@ -669,7 +711,6 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
| ID | Remaining choice or dependency | Resolve for |
| --- | --- | --- |
| U1 | Supplying PromptKit API/release for the requested message-content hash; verify the final compatible dependency set against the selected baseline. The upstream request is pending. | Track in Phase 1; required to finish cache integration in Phase 5. |
| U2 | Custom read-only filesystem responsibilities; schema root/path and assembly; prompt-declared versus per-request contracts; removable wrappers and useful artifact-compatibility/inspection metadata. Mandatory manifest metadata is retired. | Phase 2. |
| U3 | Concrete module/pipeline signatures and helper placement for request construction, interpretation, assessment, and native engine assembly. | Implementation detail in Phase 3, refined in Phase 4. |
| U4 | Role-specific initial/repair definitions, response contracts, and profile selection within complete replacement and fixed operation inputs. | Phases 3–4. |
| U5 | Producer/validator retry-setting migration to native controls and semantic budgets: field names/defaults, non-LLM failure handling, and any justified recovery beyond native exhaustion. | Phases 3–4 and configuration migration. |
@@ -678,6 +719,12 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
| U8 | Cache directory configuration and reuse/bypass/refresh controls (names, defaults, scope); old-checkpoint treatment and unusable-entry diagnostics. Hash inputs, lookup timing, accepted value, and ownership are settled. | Design with Phase 3; finish in Phase 5. |
| U9 | Observer/group enrollment and application scope; concrete conditional report fields and debug filenames/serialization within the agreed contracts; publication-layout/consumer migration details preserving intrinsic artifact schemas. | Implementation details in Phases 3–5; no new receipt/manifest inventory or capture-policy decision required. |
**U2 resolved:** Ordinary embedded assets, prompt-declared output contracts,
native schema-source selection, and removal of duplicate schema/filesystem
machinery are selected in [the schema design](#prompt-assets-and-response-schemas-u2-resolved).
Implementation remains in Phase 2; role-specific prompts and retry budgets remain
U4/U5.
**U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight,
output, debug policy, and coordinated cutover sequencing are selected above.
The former later CommandKit phase is retired. CLI compatibility is a non-goal;
@@ -698,8 +745,8 @@ major boundary changes in successor ADRs without rewriting accepted decisions.
Keep the cache dependency explicit without blocking independent work.
- Map affected ADRs, current callers, contracts, and removal inventory. Carry
forward Audita evidence without adopting its application wrappers.
- Preserve U2–U9 with their dependent phases; do not treat resolved CommandKit,
debug, cache, or output choices as requirements to redesign.
- Preserve U3–U9 with their dependent phases; do not treat resolved assets/schemas,
CommandKit, debug, cache, or output choices as requirements to redesign.
**Completion evidence:** reviewed dependency baseline, upstream dependency, scope,
and outstanding choices are explicit. This phase does not claim that the hash
@@ -707,15 +754,17 @@ already exists or that the migration is implemented.
### Phase 2: Build Native Composition, Sources, And Preflight
- Resolve U2, update available dependencies, introduce configuration version 6,
convert native embedded/companion resources, and establish schema loading.
- Implement the resolved U2 design, update available dependencies, introduce
configuration version 6, convert native embedded/companion resources, and use
native schema loading with embedded/default or operator-selected roots.
- Build the small CommandKit application, shared ConfigKit/engine loaders,
standard suites, pipeline declarations/native alias, and `internal/preflight`.
- Add maintained embedded-only and operator-override examples. Remove duplicated
loading and asset assembly as callers migrate.
**Completion evidence:** runtime/inspection/preflight share intended definitions;
source contexts, strict peer conflicts, and replacement work offline. Help,
source contexts, shared message paths, prompt-declared output contracts, selected
schema roots, strict peer conflicts, and replacement work offline. Help,
version, and completion need no configuration. Preflight checks known dependencies
without credentials or synthetic requests. `config diff` handles two selections.
@@ -784,7 +833,8 @@ Release preparation/publication is governed separately by release policy.
| Retire or replace | Retained application responsibility |
| --- | --- |
| `StructuredLLMClient`, private request/response types, adapter conversions | Native request construction and typed interpretation. |
| Prompt/profile `AssetRegistry`, `PromptAssetManifest`, `ModulePromptFS`, synthetic mounts, manual fingerprint enumeration | Embedded content and module meaning; schema choices under U2. |
| `AssetRegistry`, `PromptAssetManifest`, `ModulePromptFS`, synthetic mounts, flattened schema namespaces, `readonlyfs`, manual fingerprint enumeration | Ordinary embedded content, native source assembly, and module meaning. |
| Private response-schema registry/wrappers, byte copies, digest and reporting-only identity metadata, duplicate structural validation | Native prompt output contracts and schema loading; application interpretation and durable artifact contracts. |
| `single_response_v1`, raw `ModelCandidate`/`SemanticCorrection`, manual replay, correction capability checks | Typed candidates, actionable findings, native exchanges, assessment. |
| LLM retry loops in `runProducerAttempts`/`executeValidationChain`, module rerun directives | Ordered validators, domain acceptance/rejection/fallback. |
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, worker coupling | Bounded pipeline work and native backend admission. |
@@ -805,7 +855,7 @@ retry tests or freeze complete help text and native debug snapshots.
| Area | Required evidence |
| --- | --- |
| Configuration/assets | Version 6/new resources work; old formats intentionally rejected; companions/native imports retain main-source context; peer conflicts fail; same-ID replacement and selected schemas work. |
| Configuration/assets | Version 6/new resources work; old formats intentionally rejected; companions/native imports retain main-source context; peer conflicts fail; same-ID replacement, real shared-file paths, prompt-declared contracts, and embedded/external schema roots work; necessary request overrides replace the complete contract. |
| Commands/loaders | Required operands, native alias, supplied-value distinctions, references, and run-wide overrides reach intended handlers/roles; help/version/completion need no config; diff selections remain distinct. |
| Preflight/inspection | Same definitions across execution and inspection; missing statically selected resources fail with context; partial profiles alone pass; no provider calls, transcript, or credential requirement for offline preflight. |
| Domain workflow | Representative extraction and normalization preserve source/reference identity, fixed handles, safe plan application, ordered validators, artifact schemas, and stable output ordering. |