234 lines
13 KiB
Markdown
234 lines
13 KiB
Markdown
# PromptKit v0.23.2 Adoption
|
|
|
|
## Purpose
|
|
|
|
Replace Notarius's PromptKit v0.12 integration with a native v0.23.2
|
|
integration. Delegate configuration composition, resource inspection, request
|
|
preparation, and model execution to PromptKit and ConfigKit. Keep application
|
|
machinery where it implements a distinct Notarius decision, such as typed
|
|
artifact handling, stage acceptance, cross-backend scheduling, or checkpoint
|
|
reuse.
|
|
|
|
This is a hard cutover for both the Notarius application configuration and
|
|
PromptKit resource definitions. Provide migration guidance and maintained
|
|
examples; support only the new formats in the resulting application. This
|
|
draft describes the target, assuming the recommendations under
|
|
[Open Decisions](#open-decisions). It does not select a release date or
|
|
implementation sequence.
|
|
|
|
## Reviewed Baseline
|
|
|
|
Notarius at `65b8e8f0a2d60041c18a29a3c33afc5c689468df` uses application
|
|
configuration version 5, Go 1.27, new-path ConfigKit v1.2.0, and old-path
|
|
PromptKit v0.12.0. The latter brings in a separate old-path ConfigKit v0.5.0
|
|
and two external provider catalogs. Notarius and PromptKit currently share
|
|
the YAML v4 release candidate.
|
|
|
|
The target is `go.maximumdirect.net/promptkit@v0.23.2`, published at
|
|
`382ccfb14b9630fd8f48c8364dbe2b6159637d71`. The tagged [release
|
|
notes][release], [migration guide][migration], [format reference][formats],
|
|
and [Go consumer guide][consumer] are the primary upstream references.
|
|
|
|
The main baseline seams are:
|
|
|
|
- `internal/core/config` resolves `promptkit.profile_file` and
|
|
`profile_dir` and interprets the optional `local_backend` object.
|
|
`promptkit_profiles.go` builds the operator layers for execution and
|
|
inspection; `asset_registry.go` supplies the embedded layer.
|
|
- `assets/` and `examples/llm-profiles/` use keyed resource definitions.
|
|
Completion requests and diagnostics carry a PromptKit prompt version.
|
|
- `internal/framework/llm` does split preparation and execution to retain
|
|
debug material; it projects a backend-completed profile report and walks
|
|
effective YAML to discover versioned prompts for checkpoint identity.
|
|
- The pipeline's producer-attempt loop coordinates domain and stage policy.
|
|
The scheduled LLM client enforces the application-wide limit independently
|
|
of any PromptKit backend policy.
|
|
|
|
## Target End State
|
|
|
|
### Dependency And Application Configuration Cutover
|
|
|
|
Use the vanity PromptKit import path and shared ConfigKit v1.2.1, stable
|
|
go-yaml v4, and the resulting compatible validation dependencies. Remove the
|
|
old-path imports and catalog dependencies once unused. Use tagged modules
|
|
without a compatibility fork or permanent replacement. The consumer minimum
|
|
remains Go 1.27. Update attribution for the selected and removed components
|
|
under the [OSS policy](../policy/oss.md).
|
|
|
|
Introduce application configuration version 6 as the only supported application
|
|
version. Version 6 retains unrelated pipeline structure, discovery, and
|
|
resolution behavior. ConfigKit continues to own application loading and
|
|
composition under [ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md).
|
|
The proposed operator seam is an optional `promptkit.config_file` pointing to a
|
|
native PromptKit main source. Resolve this selector relative to the Notarius
|
|
file that declares it; leave imports and resource paths *inside* the selected
|
|
source to the upstream loader. Retire `profile_file`, `profile_dir`, and
|
|
`local_backend`: express related sources through native imports and local
|
|
endpoints through native backend records. Preserve the ability to override
|
|
transport retries through the PromptKit engine option; leave the upstream
|
|
default in effect when the operator does not override it.
|
|
|
|
Pass the selected main source and application asset underlay through one source
|
|
assembly used by runtime, preflight, and inspection. Use the bundled OpenRouter
|
|
underlay as supplied by PromptKit. That bundle is a different inventory from
|
|
the former external catalogs and supplies no backend-local concurrency limit.
|
|
It does not implicitly select a model for any Notarius request. Migrate
|
|
deployments that use removed profiles or backends to intentionally selected
|
|
native records. The application-wide LLM ceiling remains independent of
|
|
backend-local settings.
|
|
|
|
### Native Prompt And Profile Resources
|
|
|
|
Convert maintained prompts, profiles, examples, and fixtures to identified
|
|
lists, with `config:` for source controls and native sibling inheritance
|
|
(`extends: {id: parent}`). Under native default composition, a later same-ID
|
|
record replaces the earlier record in full. Explain complete replacements,
|
|
distinct inherited variants, and the new backend fallback semantics for
|
|
profile nulls.
|
|
The optional PromptKit source specification header is version 2; it is not the
|
|
Notarius application version. Keep source-local message resolution and use
|
|
native collections when deployments need namespaces. Do not translate
|
|
legacy resource files, synthesize aliases, or reconstruct old recursive
|
|
same-ID overlays.
|
|
|
|
Select prompts by ID alone and remove the obsolete PromptKit version selector
|
|
and metadata from Notarius's request, debug, and reconciliation identity
|
|
plumbing. Review affected diagnostic fields and document deliberate changes.
|
|
Retain logical workload IDs, including `dnd-extraction` and its current
|
|
explicit model/settings. Retain message roles, bytes, order, cache boundaries,
|
|
and intended output/repair behavior while changing the declaration format.
|
|
Keep module-owned asset selection and embedded filesystem contexts intact.
|
|
Verify maintained private response schemas work under upstream's
|
|
self-contained-schema rule; this is not a redesign of durable D&D artifacts.
|
|
|
|
### Inspection And Reuse Identity
|
|
|
|
Use PromptKit's captured configuration reports for effective configuration
|
|
and provenance. Use its resource inventory and inspection APIs for exact
|
|
identities, presence-aware resource fields, and captured prompt hashes, rather
|
|
than deriving an inventory from rendered configuration YAML. Change the
|
|
`config profile` report to show the resolved profile and its selected backend,
|
|
when present, as separate resources. Use upstream field semantics and retain
|
|
the existing operator disclosure boundary, including redaction of
|
|
credential-shaped provider parameters. Do not recreate backend default
|
|
resolution to produce the former `effective_model_params` view.
|
|
|
|
Distinguish configuration and resource preflight from request readiness.
|
|
Use `InspectRequirements` to explain recognized dependencies when helpful,
|
|
not as proof that a request is executable. Use `Prepare` or
|
|
`ValidateRequest` when concrete request inputs are available and their
|
|
validation is actually needed. A routine config-only check should remain
|
|
credential-free and offline, without synthetic requests.
|
|
|
|
Retain a private, versioned Notarius checkpoint identity based on the engine's
|
|
captured state. Use native resource enumeration and prompt hashes, keeping
|
|
the effective execution inputs needed to distinguish reuse safely. A diff of
|
|
effective values alone cannot detect edits to a referenced message file.
|
|
Version the fingerprint for the cutover so obsolete identity inputs cannot
|
|
cause a false hit; do not reread mutable sources or expose the digest inputs.
|
|
Keep run manifests based on the profiles actually used, not the inventory.
|
|
|
|
### Native Execution And Retained Pipeline Policy
|
|
|
|
Use ordinary `Engine.Run` for structured completion and use `OnPrepared`
|
|
where opt-in debug capture needs the actual prepared request. The callback
|
|
runs after admission, before generation, and is not delivered for failed
|
|
preparation or admission. Keep the callback lightweight and use native
|
|
observations for safe terminal execution facts, including attempts, repairs,
|
|
and available token usage. Where a diagnostic needs totals over selected
|
|
executions, use `ExecutionGroup` rather than inventing a second accumulator.
|
|
Use `OperationTimeout` where a completion calls for that scope of budget;
|
|
do not add a new default deadline or replace application deadlines that also
|
|
cover work outside the PromptKit call.
|
|
|
|
PromptKit owns provider admission, transport retries, and structural repair.
|
|
Keep the Notarius cross-backend scheduler because it serves a different
|
|
budget from backend or pipeline worker limits. Keep the adapter's typed
|
|
decode, actual-run provenance, and provider-neutral error and diagnostic
|
|
mapping. PromptKit execution types stay within that integration boundary.
|
|
In particular, preserve the distinction between invalid structured
|
|
output, provider failure, capacity rejection, and cancellation. A configured
|
|
repair override still selects a complete upstream output contract; it must not
|
|
accidentally drop the selected schema or validation mode.
|
|
|
|
Notarius owns stage attempts, validation and fallback decisions, and
|
|
publication of accepted artifacts. Keep its latest-response-only correction
|
|
policy using native appended messages, as defined by
|
|
[ADR-0014](../adr/0014-feedback-aware-validation-retries.md) and
|
|
[ADR-0016](../adr/0016-feedback-aware-module-requested-retries.md).
|
|
`Exchange` and `Coordinate` are for workflows that benefit from retained
|
|
conversation history. Adopting them for stage retries would change what the
|
|
model sees and which work consumes an attempt. They are not needed to execute
|
|
the existing retry policy.
|
|
|
|
## Migration And Verification
|
|
|
|
Update the maintained minimal and complete D&D examples, including operator
|
|
profile inheritance, and document how to convert a directory deployment,
|
|
declare a local backend, and replace an embedded prompt or profile. Explain
|
|
retired catalog IDs, the changed profile report, and any new checkpoint cold
|
|
misses. Update the canonical configuration, CLI, operations, and integration
|
|
guides when behavior lands. Record the changed decisions about source
|
|
precedence and inspection in a successor to
|
|
[ADR-0019](../adr/0019-adopt-native-promptkit-configuration.md) rather than
|
|
rewriting its accepted decision text. Release preparation is separate.
|
|
|
|
Verify the dependency graph and offline examples after the cutover. Use
|
|
focused behavior tests for the new format and intentional rejection of the
|
|
old one, native same-ID replacement and relative imports, and shared source
|
|
selection across preflight, inspection, and runtime. Exercise preparation,
|
|
generation, structural repair, and semantic retries through the maintained
|
|
fake-provider integration. Check that real message or target changes affect
|
|
checkpoint reuse, and that session routing, cache controls, diagnostics, and
|
|
durable outputs keep their existing meaning. Live provider checks are useful
|
|
for deployment confidence, not a requirement of the offline suite.
|
|
|
|
[release]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/releases/v0.23.2.md
|
|
[migration]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/consumers/configuration-migration.md
|
|
[formats]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/formats.md
|
|
[consumer]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/consumers/pkg-promptkit.md
|
|
|
|
## Open Decisions
|
|
|
|
### Operator Source Selection
|
|
|
|
**Recommendation:** Expose one optional native main-file selector. It gives
|
|
operators a single entry point for native file and directory imports,
|
|
collection mounting, and relative resource resolution, while keeping source
|
|
composition out of Notarius.
|
|
|
|
**Alternative:** Expose an explicit list of file or directory inputs assembled
|
|
into native PromptKit source groups. This is useful if a caller must select
|
|
independent inputs without owning a common entry file. It also requires
|
|
Notarius to define group ordering, path resolution, and precedence for a
|
|
second composition surface. Prefer the main file unless that workflow is
|
|
needed.
|
|
|
|
### How Far Offline Preflight Goes
|
|
|
|
**Recommendation:** Check that the captured configuration is valid and that
|
|
selected resource IDs resolve. Make clear that a valid profile can be partial
|
|
and that concrete request readiness is checked during preparation. This
|
|
preserves a useful offline preflight without duplicating target resolution or
|
|
requiring credentials or source data.
|
|
|
|
**Alternative:** Preserve offline checking of completed targets by pursuing a
|
|
dedicated upstream API that resolves targets without artifacts or credentials.
|
|
That would catch more missing endpoint/model settings before a run and could
|
|
serve other consumers, but it would add an upstream dependency to this work or
|
|
require a subsequent release. Reimplementing completion rules in Notarius
|
|
would risk a discrepancy with actual execution.
|
|
|
|
### Operator Diagnostics In This Upgrade
|
|
|
|
**Recommendation:** Migrate existing inspection commands and use native
|
|
inventory and observation APIs where they replace local work. Evaluate new
|
|
commands for provider smoke checks or effective-config diffs separately,
|
|
after identifying the operator workflows and output contracts they need.
|
|
|
|
**Alternative:** Include an explicitly invoked smoke command in the cutover.
|
|
This would help operators verify credentials and structured inference after
|
|
migrating a deployment, and it can delegate to `Engine.SmokeTest`. It adds a
|
|
network-capable CLI surface, with cost and output handling to define. Keep it
|
|
separate from ordinary offline validation in either case.
|