Files
notarius/docs/roadmap/promptkit.md

244 lines
14 KiB
Markdown

# PromptKit v0.23.2 Adoption
## Purpose
Replace Notarius's PromptKit v0.12 integration with a native v0.23.2
integration. Delegate configuration composition, resource inspection, request
preparation, and model execution to PromptKit and ConfigKit. Keep application
machinery where it implements a distinct Notarius decision, such as typed
artifact handling, stage acceptance, cross-backend scheduling, or checkpoint
reuse.
This is a hard cutover for both the Notarius application configuration and
PromptKit resource definitions. Provide migration guidance and maintained
examples; support only the new formats in the resulting application. This
roadmap defines the PromptKit cutover. Operator interface standardization
belongs to the subsequent CommandKit adoption.
## Reviewed Baseline
Notarius at `65b8e8f0a2d60041c18a29a3c33afc5c689468df` uses application
configuration version 5, Go 1.27, new-path ConfigKit v1.2.0, and old-path
PromptKit v0.12.0. The latter brings in a separate old-path ConfigKit v0.5.0
and two external provider catalogs. Notarius and PromptKit currently share
the YAML v4 release candidate.
The target is `go.maximumdirect.net/promptkit@v0.23.2`, published at
`382ccfb14b9630fd8f48c8364dbe2b6159637d71`. The tagged [release
notes][release], [migration guide][migration], [format reference][formats],
and [Go consumer guide][consumer] are the primary upstream references.
The main baseline seams are:
- `internal/core/config` resolves `promptkit.profile_file` and
`profile_dir` and interprets the optional `local_backend` object.
`promptkit_profiles.go` builds the operator layers for execution and
inspection; `asset_registry.go` supplies the embedded layer.
- `assets/` and `examples/llm-profiles/` use keyed resource definitions.
Completion requests and diagnostics carry a PromptKit prompt version.
- `internal/framework/llm` does split preparation and execution to retain
debug material; it projects a backend-completed profile report and walks
effective YAML to discover versioned prompts for checkpoint identity.
- The pipeline's producer-attempt loop coordinates domain and stage policy.
The scheduled LLM client enforces the application-wide limit independently
of any PromptKit backend policy.
## Target End State
### Dependency And Application Configuration Cutover
Use the vanity PromptKit import path and shared ConfigKit v1.2.1, stable
go-yaml v4, and the resulting compatible validation dependencies. Remove the
old-path imports and catalog dependencies once unused. Use tagged modules
without a compatibility fork or permanent replacement. The consumer minimum
remains Go 1.27. Update attribution for the selected and removed components
under the [OSS policy](../policy/oss.md).
Introduce application configuration version 6 as the only supported application
version. Version 6 retains unrelated pipeline structure, discovery, and
resolution behavior. ConfigKit continues to own application loading and
composition under [ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md).
Retire `profile_file`, `profile_dir`, and `local_backend` in favor of native
PromptKit companion files. There is no separate `promptkit.config_file` selector.
Preserve the ability to override transport retries through the PromptKit engine
option; leave the upstream default in effect when the operator does not
override it.
### Companion Files And Embedded Defaults
Adopt Audita's companion-file convention. Use the directory of the main
Notarius configuration actually selected by ConfigKit, whether explicit or
discovered. Load the optional `prompts.yml` and recursive `.yml` and `.yaml`
files under `prompts/` beside it as one higher-priority operator underlay
using native ConfigKit imports. Do not add a working-directory fallback
for companion discovery. Configurations in the same directory share those
companions. Missing companions are fine; loading errors follow upstream
semantics rather than silently falling back to embedded assets. Native
imports and source-relative resource paths govern any files reached from the
companions. The two companion locations are strict peers; filenames do not
set precedence between conflicting operator records.
Use one source assembly for runtime, preflight, and inspection: PromptKit's
bundled OpenRouter underlay, Notarius's embedded underlay, then the operator
underlay when present. Embed all Notarius default prompts and profiles under
`collections.notarius` in the application underlay. Operator companion files
can replace individual defaults by defining complete records with the same
local IDs in that same collection. They can also define new resources and
local backends using the native format.
The bundled OpenRouter inventory differs from the former external catalogs
and supplies no backend-local concurrency limit. It does not implicitly
select a model for any Notarius request. Migrate deployments that use removed
profiles or backends to intentionally selected native records. The
application-wide LLM ceiling remains independent of backend-local settings.
### Native Prompt And Profile Resources
Convert maintained prompts, profiles, examples, and fixtures to identified
lists, with `config:` for source controls and native sibling inheritance
(`extends: {id: parent}`). Under native default composition, a later same-ID
record replaces the earlier record in full. Explain complete replacements,
distinct inherited variants, and the new backend fallback semantics for
profile nulls.
The optional PromptKit source specification header is version 2; it is not the
Notarius application version. Do not translate legacy resource files,
synthesize aliases, or reconstruct old recursive same-ID overlays.
Select prompts by ID alone and remove the obsolete PromptKit version selector
and metadata from Notarius's request, debug, and reconciliation identity
plumbing. Review affected diagnostic fields and document deliberate changes.
Keep the current local resource IDs and use their qualified forms in
lookups and references, for example `notarius/dnd.spells` and
`notarius/dnd-extraction`. Update prompt profile references, application
profile selections, and inspection examples to use the qualified IDs.
The `dnd-extraction` profile retains its current model and settings and
continues to reference the unqualified bundled `openrouter` backend. A
profile that extends it belongs in the same collection and uses its local
sibling ID in `extends`. Retain message roles, bytes, order, cache boundaries,
and intended output/repair behavior while changing the declaration format.
Keep module-owned asset selection and embedded filesystem contexts intact.
Verify maintained private response schemas work under upstream's
self-contained-schema rule; this is not a redesign of durable D&D artifacts.
### Inspection And Reuse Identity
Use PromptKit's captured configuration reports for effective configuration
and provenance. Use its resource inventory and inspection APIs for exact
identities, presence-aware resource fields, and captured prompt hashes, rather
than deriving an inventory from rendered configuration YAML. Make only the
changes needed to keep the existing `config effective`, `config explain`, and
`config profile` commands usable after the cutover. Report PromptKit's
resolved profile resource rather than rebuilding the former
`effective_model_params` view or designing a combined profile/backend report.
Retain the existing disclosure boundary, including redaction of
credential-shaped provider parameters. Defer new operator commands and report
surfaces to the CommandKit work.
Retain a private, versioned Notarius checkpoint identity based on the engine's
captured state. Use native resource enumeration and prompt hashes, keeping
the effective execution inputs needed to distinguish reuse safely. A diff of
effective values alone cannot detect edits to a referenced message file.
Version the fingerprint for the cutover so obsolete identity inputs cannot
cause a false hit; do not reread mutable sources or expose the digest inputs.
Keep run manifests based on the profiles actually used, not the inventory.
### Offline Preflight
Keep `config validate` focused on validating the resolved Notarius application
configuration. With `--pipeline`, also resolve the selected stages and lanes
against the module catalog. For an LLM-backed selection, load the same
effective PromptKit definitions used by `run`, even when the pipeline does
not explicitly select a profile. Let PromptKit validate and capture those
definitions, including companion sources and referenced message files.
Check the prompt and profile IDs known from pipeline resolution, including a
prompt's default profile when the prompt is known and its default is the
effective selection. Do not simulate stages to discover runtime selections.
Preflight requires no transcript, credential values, or provider calls. Its
success confirms configuration and known resource selection, not that every
concrete request is runnable. A valid PromptKit profile can be partial. Let
PromptKit check the completed target, required credentials, inputs, and schema
during request preparation. Use `InspectRequirements` to explain recognized
dependencies when useful; use `Prepare` or `ValidateRequest` for an actual
request when no-generation validation is needed. Do not construct synthetic
requests or reimplement target resolution for offline preflight.
### Native Execution And Retained Pipeline Policy
Use ordinary `Engine.Run` for structured completion and use `OnPrepared`
where opt-in debug capture needs the actual prepared request. The callback
runs after admission, before generation, and is not delivered for failed
preparation or admission. Keep the callback lightweight and use native
observations for safe terminal execution facts, including attempts, repairs,
and available token usage. Where a diagnostic needs totals over selected
executions, use `ExecutionGroup` rather than inventing a second accumulator.
Use `OperationTimeout` where a completion calls for that scope of budget;
do not add a new default deadline or replace application deadlines that also
cover work outside the PromptKit call.
PromptKit owns provider admission, transport retries, and structural repair.
Keep the Notarius cross-backend scheduler because it serves a different
budget from backend or pipeline worker limits. Keep the adapter's typed
decode, actual-run provenance, and provider-neutral error and diagnostic
mapping. PromptKit execution types stay within that integration boundary.
In particular, preserve the distinction between invalid structured
output, provider failure, capacity rejection, and cancellation. A configured
repair override still selects a complete upstream output contract; it must not
accidentally drop the selected schema or validation mode.
Notarius owns stage attempts, validation and fallback decisions, and
publication of accepted artifacts. Keep its latest-response-only correction
policy using native appended messages, as defined by
[ADR-0014](../adr/0014-feedback-aware-validation-retries.md) and
[ADR-0016](../adr/0016-feedback-aware-module-requested-retries.md).
`Exchange` and `Coordinate` are for workflows that benefit from retained
conversation history. Adopting them for stage retries would change what the
model sees and which work consumes an attempt. They are not needed to execute
the existing retry policy.
## Migration And Verification
Update the maintained minimal and complete D&D examples to distinguish
embedded-only operation from companion configuration. Use separate example
directories if necessary to demonstrate those deployments independently.
Document how to move existing PromptKit sources into the companion layout
or import them from it, declare a local backend, and replace or extend an
embedded resource. Explain qualified IDs, retired catalog IDs, the changed
profile report, and any new checkpoint cold misses. Update the canonical
configuration, CLI, operations, and integration guides when behavior lands.
Record the changed decisions about source precedence and inspection in a
successor to
[ADR-0019](../adr/0019-adopt-native-promptkit-configuration.md) rather than
rewriting its accepted decision text. Release preparation is separate.
Verify the dependency graph and offline examples after the cutover. Use
focused behavior tests for the new format and intentional rejection of the
old one, main-relative companion discovery, and native relative imports.
Verify that a companion can replace a qualified embedded prompt or profile
without changing unrelated defaults, and that preflight, inspection, and
runtime use the same effective definitions. Verify that pipeline preflight
catches invalid effective definitions and unknown selected IDs without
requiring inputs or credentials, and does not reject a profile merely for
being partial. Exercise preparation, generation, structural repair, and
semantic retries through the maintained fake-provider integration. Check
that real message or target changes affect checkpoint reuse, and that session
routing, cache controls, diagnostics, and durable outputs keep their existing
meaning. Live provider checks are useful for deployment confidence, not a
requirement of the offline suite.
## Follow-Up: CommandKit
The planned follow-up adopts `go.maximumdirect.net/commandkit` with its
`ckconfig` and `ckprompt` integrations and replaces Notarius-owned
configuration and PromptKit commands where applicable. That work owns the
standardized CLI and decisions about exposing resource lists, configuration
diffs, request previews, and explicit provider smoke diagnostics. Leave
those commands and the CommandKit dependencies to the follow-up. This
deferral does not change offline pipeline preflight, internal use of native
inspection and observation APIs, or existing run diagnostics.
[release]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/releases/v0.23.2.md
[migration]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/consumers/configuration-migration.md
[formats]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/formats.md
[consumer]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/consumers/pkg-promptkit.md