Files
audita/docs/roadmap/promptkit.md

181 lines
11 KiB
Markdown

# PromptKit Adoption Outline
**Updated:** 2026-11-07
**Status:** Initial outline. Follow the [master roadmap](library-adoption.md)
for shared adoption principles, dependency identities, and sequencing.
## Baseline and Target
Audita currently uses PromptKit v0.14.0. Its
[LLM runtime](../internal/llm-runtime.md) uses a shared engine, application-owned
backend overrides, curated profile inspection, and prepared execution.
Adopt PromptKit v0.20.0 through its native configuration and execution APIs.
Retain the built-in OpenRouter backend and bundled profiles. They must be usable
through the normal selection and credential mechanisms, not merely visible in
an inventory while Audita rejects them for lacking a duplicate registration.
Expose native resource and configuration inspection through the full standard
ckprompt surface described in the [CommandKit outline](commandkit.md).
PromptKit owns resources, composition, provider settings and credentials,
preparation, transport, structural repair, and backend admission. Audita owns
proposal/validation role selection, transcript processing, domain decisions,
response contracts, and truthful partial-success reporting. Remove the existing
application-only endpoint and credential restrictions as described below.
Use the pinned [migration guide](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/8ff9aa3efdc9f16b9d2729e0a29ea083eb3c3eaf/docs/consumers/configuration-migration.md),
[format reference](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/8ff9aa3efdc9f16b9d2729e0a29ea083eb3c3eaf/docs/formats.md),
and [v0.20 release notes](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/8ff9aa3efdc9f16b9d2729e0a29ea083eb3c3eaf/docs/releases/v0.20.0.md).
The bundle is default-on; older migration prose about catalog removal does not
mean the current release has no bundled resources. It does not select an implicit
execution profile or make provider calls during construction.
## Agreed Direction and Migration Work
### Backend Concurrency Ownership — Resolved
Adopt PromptKit's native per-backend concurrency behavior, including backend
defaults and supported configuration and CLI overrides. The purpose is to keep
requests within the concurrency a backend can accommodate; no separate Audita
domain requirement has been identified for an additional limit.
Remove Audita-owned concurrency settings, default injection, validation, and
enforcement wherever they duplicate or override that native behavior. Do not
retain a second limiter or impose an Audita-specific finite-cap requirement on
otherwise valid PromptKit configuration. Use the shared PromptKit engine so
proposal and validation work for the same backend share its admission control.
Audita still owns cancellation and deterministic processing around model calls.
Verify the exact upstream override APIs while wiring the adopted configuration
and CLI integrations. Update the architecture policy's finite-admission wording,
runtime documentation, examples, and tests with implementation. Remove superseded
behavior directly in the major release, with hard-cutover guidance.
### Backend and Credential Ownership — Resolved
Native PromptKit backend/profile resources are authoritative for endpoints,
provider settings, and credential selection. No domain-specific requirement has
been identified for a separate Audita ownership policy.
- Remove Audita's separate `backends` mapping and generated backend override
layer. Remove duplicate registration requirements and checks that force profile
endpoints to match an Audita-owned backend endpoint. Native selection and
validation govern bundled and operator-defined resources.
- Use PromptKit's native credential sources and precedence. Remove Audita's
LLM-specific credential references, automatic direct-key injection, and
rejection of profile-level environment lookups. Backend/profile documents
contain environment-variable names, not credential values; operators may use
the bundled OpenRouter lookup or configure their own names.
- Follow native credential availability timing. Loading and resource inspection
must not require credentials for every configured backend. Preparation and
execution enforce the selected request's native required/optional semantics:
missing optional credentials permit unauthenticated requests. Do not add an
eager all-backend credential check or silently strengthen those semantics.
Audita retains role/profile selection, operation budgets, domain request
construction, response contracts, and safe handling of its reports and debug artifacts. Native
configuration redaction does not replace redaction at Audita's artifact boundary.
Keep preparing and executing the same captured request while retiring target
checks whose purpose was enforcing the removed application backend registry.
Remove superseded fields, code, and tests in the major-release cutover and update
the affected policies and current contracts with implementation. The ConfigKit
outline still owns decisions about file locations, discovery, and imports; those
choose how native resources are supplied, not who owns their semantics.
### Migration Work
1. Move to `go.maximumdirect.net/promptkit` together with ConfigKit's new types.
Remove the old external catalog dependencies when no longer incorporated;
retain PromptKit's built-in catalog rather than disabling it.
2. Convert root and collection backends, profiles, and prompts to identified lists
with explicit `id` fields. Use sibling `extends: {id: parent}` and complete
same-ID replacement records. Typed source builders use ordered slices.
Preserve provider settings deliberately: partial same-ID replacements no
longer merge fields. Use exact native collection references where collections
are selected; mounting does not rewrite stored references.
3. Convert the seven embedded prompts and maintained operator examples/fixtures.
Remove prompt version selectors and descriptive `PromptVersion` metadata from
requests, registries, inspection, and diagnostics. Response-schema versions
are separate and need not change. Keep domain instructions and per-call schemas;
this adoption is not a prompt-quality rewrite. Shared instruction variables
remain available to templates, while native operator overrides control their
inclusion. Domain validation remains enforced after model responses.
4. Adapt resource inspection to optional profile fields and separate backend
inspection. These describe resources, not completed requests. Use real
`PrepareExecution` details and `RunPrepared` for execution of the same captured
request. Do not duplicate PromptKit's full resolver or construct synthetic
requests merely to reproduce the old profile summary. Inspect bundled and
unselected partial resources without requiring them to be runnable.
5. Implement the resolved backend and credential ownership decision above.
Replace application backend/credential configuration with native resources
and document the hard cutover. Preserve secret-safe diagnostics using the
library's public boundaries and Audita's domain artifact redaction; evaluate
actual gaps with synthetic keys.
6. Review presence semantics field by field. Null endpoint, credential-source,
and whole provider map now allow backend defaults; explicit empty strings
clear endpoint/credential sources. Null does not suppress the entire backend
provider map. Sampling fields have distinct semantics. Avoid blanket null or
zero conversion and document intentional request changes.
7. Update captured-source accounting for list indexes and collection prefixes,
including inherited/moved content-file origins. Preserve output-collision
protection for consumed host documents and effective message files. Array
positions are addresses, not durable resource identities.
8. Keep self-contained embedded response schemas. The new schema rule does not
require a bundler for Audita's current schemas. Check custom artifact-reader
ownership only where such a reader is actually used. Preserve native retries,
structural repair, cancellation, deadlines, and shared engine admission while
retaining Audita's domain reconciliation and partial-success outcomes.
Update [configuration](../config.md), [LLM integration](../integrations/openai-compatible-llm.md),
[runtime internals](../internal/llm-runtime.md), [prompt guidance](../internal/prompts.md),
operator examples, diagnostics metadata documentation, and attribution with the
implementation. Standard inspection replaces curated-only disclosure; prompts
and provider settings may appear where the library's normal inspection exposes
them, with native redaction. Do not accidentally promise old prompt hashes.
## Acceptance and Upstream Requests
Verify native bundle inventory and selection, configuration/resource/prepared
view distinctions, inheritance and replacement, credential precedence, optional
fields, source-relative messages and collisions, and captured configuration.
Use controlled providers to verify actual targets, domain schemas, shared
admission, retry/repair budgets, cancellation, and partial success. Standard
inspection must work without model calls; library presence alone does not prove
successful Audita integration.
Verify backend defaults and explicit concurrency overrides through the assembled
application, including shared admission across proposal and validation roles.
Check that Audita no longer injects an additional limit or rejects native settings
under its former finite-cap policy. Keep integration coverage for Audita's wiring;
remove tests that only preserve retired local concurrency behavior.
Verify native bundled and custom backend selection without duplicate Audita
registration, supported profile endpoint and credential overrides, and inspection
without credential availability. Use controlled requests to cover required-key
failures and optional unauthenticated execution under native semantics. Check
that no application override layer or direct-key injection defeats native
precedence, and that synthetic keys stay out of Audita diagnostics and artifacts.
No upstream request is yet selected. PromptKit is mature but pre-v1: significant
blockers or awkward public boundaries can justify focused requests. Submit the
minimal reproducer, desired outcome, and domain justification before building
substantial local machinery. A review probe demonstrated distinct base/derived
backend records preserving provider fields; that is an available native technique,
not a decision to retain Audita's old backend ownership model.
## Open Decisions
### Existing Curated Profile Summary
**Recommendation:** use standard resource inspection and prepared-request preview
for their distinct meanings; retire the old completed-profile summary unless a
specific Audita role summary remains useful. This avoids a parallel target resolver
and aligns operator expectations.
**Alternative:** retain a small role-to-resource summary alongside the full native
commands. It helps transcript operators identify proposal and validation choices,
but must clearly describe resource values rather than implying completed request
settings. It adds maintenance and must not constrain native inspection.