audita/docs/roadmap/promptkit.md

PromptKit Adoption Outline

Updated: 2026-11-07

Status: Initial outline. Follow the master roadmap for shared adoption principles, dependency identities, and sequencing.

Baseline and Target

Audita currently uses PromptKit v0.14.0. Its LLM runtime uses a shared engine, application-owned backend overrides, curated profile inspection, and prepared execution.

Adopt PromptKit v0.20.0 through its native configuration and execution APIs. Retain the built-in OpenRouter backend and bundled profiles. They must be usable through the normal selection and credential mechanisms, not merely visible in an inventory while Audita rejects them for lacking a duplicate registration. Expose native resource and configuration inspection through the full standard ckprompt surface described in the CommandKit outline.

PromptKit owns resources, composition, provider settings and credentials, preparation, transport, structural repair, and backend admission. Audita owns proposal/validation role selection, transcript processing, domain decisions, response contracts, and truthful partial-success reporting. Remove the existing application-only endpoint and credential restrictions as described below.

Use the pinned migration guide, format reference, and v0.20 release notes. The bundle is default-on; older migration prose about catalog removal does not mean the current release has no bundled resources. It does not select an implicit execution profile or make provider calls during construction.

Agreed Direction and Migration Work

Backend Concurrency Ownership — Resolved

Adopt PromptKit's native per-backend concurrency behavior, including backend defaults and supported configuration and CLI overrides. The purpose is to keep requests within the concurrency a backend can accommodate; no separate Audita domain requirement has been identified for an additional limit.

Remove Audita-owned concurrency settings, default injection, validation, and enforcement wherever they duplicate or override that native behavior. Do not retain a second limiter or impose an Audita-specific finite-cap requirement on otherwise valid PromptKit configuration. Use the shared PromptKit engine so proposal and validation work for the same backend share its admission control. Audita still owns cancellation and deterministic processing around model calls.

Verify the exact upstream override APIs while wiring the adopted configuration and CLI integrations. Update the architecture policy's finite-admission wording, runtime documentation, examples, and tests with implementation. Remove superseded behavior directly in the major release, with hard-cutover guidance.

Backend and Credential Ownership — Resolved

Native PromptKit backend/profile resources are authoritative for endpoints, provider settings, and credential selection. No domain-specific requirement has been identified for a separate Audita ownership policy.

• Remove Audita's separate backends mapping and generated backend override layer. Remove duplicate registration requirements and checks that force profile endpoints to match an Audita-owned backend endpoint. Native selection and validation govern bundled and operator-defined resources.
• Use PromptKit's native credential sources and precedence. Remove Audita's LLM-specific credential references, automatic direct-key injection, and rejection of profile-level environment lookups. Backend/profile documents contain environment-variable names, not credential values; operators may use the bundled OpenRouter lookup or configure their own names.
• Follow native credential availability timing. Loading and resource inspection must not require credentials for every configured backend. Preparation and execution enforce the selected request's native required/optional semantics: missing optional credentials permit unauthenticated requests. Do not add an eager all-backend credential check or silently strengthen those semantics.

Audita retains role/profile selection, operation budgets, domain request construction, response contracts, and safe handling of its reports and debug artifacts. Native configuration redaction does not replace redaction at Audita's artifact boundary. Keep preparing and executing the same captured request while retiring target checks whose purpose was enforcing the removed application backend registry.

Remove superseded fields, code, and tests in the major-release cutover and update the affected policies and current contracts with implementation. The ConfigKit outline still owns decisions about file locations, discovery, and imports; those choose how native resources are supplied, not who owns their semantics.

Bundled Prompts and Operator Replacements — Resolved

Ship Audita's production default prompt definitions in embed.FS, exposed as io/fs.FS. Load them through PromptKit's filesystem inputs into a ConfigKit underlay, with operator sources at higher priority. Using the shipped prompts must not require a separate external prompt installation.

Place all Audita-owned default prompts in the native audita collection namespace. Keep local IDs such as modules.glossary.proposal; callers use the qualified runtime ID audita/modules.glossary.proposal. Update registry entries, call sites, references, examples, and diagnostics accordingly. Mounting a source does not rewrite references, so qualified references must be explicit.

Operators may replace individual prompts or supply a complete replacement set by defining the same collection and local IDs in higher-priority sources. Unreplaced prompts continue to come from the underlay. Follow native whole-record replacement semantics for matching IDs and document the required complete prompt definitions. The namespace identifies Audita's prompts without making them immutable or requiring a separate development mode.

The development workflow is edit external definitions and rerun Audita using the same binary. A newly constructed runtime captures the changed definitions and message files; live reload within an active run is not required. The same mechanism supports production customization. Standard inspection, provenance, and request preview must make effective prompts and their origins understandable. Audita's response schemas and domain validation continue to apply.

External source selection remains part of the ConfigKit discovery/import decision. This workflow is required PromptKit adoption scope and should use native composition rather than a separate prompt loader or override system.

Profile Inspection and Summary — Resolved

Retire config print-effective --view profiles and its Audita-owned summary code. Use application configuration inspection to show proposal/validation profile selections, standard PromptKit resource inspection for profile/backend/ prompt definitions, and request preview for resolved execution settings and rendered messages. These views have distinct meanings and must be presented as such.

Do not add a replacement custom role summary at this stage. Native interfaces cover the selected needs and reduce maintenance without a parallel target resolver. Include the retired view and its standard replacements in the hard-cutover guidance. The CommandKit outline owns the operator surface and request-preview integration work.

Migration Work

1. Move to go.maximumdirect.net/promptkit together with ConfigKit's new types. Remove the old external catalog dependencies when no longer incorporated; retain PromptKit's built-in catalog rather than disabling it.
2. Convert root and collection backends, profiles, and prompts to identified lists with explicit id fields. Use sibling extends: {id: parent} and complete same-ID replacement records. Typed source builders use ordered slices. Preserve provider settings deliberately: partial same-ID replacements no longer merge fields. Use exact native collection references where collections are selected; mounting does not rewrite stored references.
3. Convert the seven embedded prompts into the audita collection and update maintained operator examples/fixtures for the resolved replacement workflow. Remove prompt version selectors and descriptive PromptVersion metadata from requests, registries, inspection, and diagnostics. Response-schema versions are separate and need not change. Keep domain instructions and per-call schemas; this adoption is not a prompt-quality rewrite. Shared instruction variables remain available to templates, while native operator overrides control their inclusion. Domain validation remains enforced after model responses.
4. Adapt resource inspection to optional profile fields and separate backend inspection. These describe resources, not completed requests. Use real PrepareExecution details and RunPrepared for execution of the same captured request. Remove the curated profile view and its supporting projection code; do not duplicate PromptKit's full resolver or construct synthetic requests merely to reproduce that summary. Inspect bundled and unselected partial resources without requiring them to be runnable.
5. Implement the resolved backend and credential ownership decision above. Replace application backend/credential configuration with native resources and document the hard cutover. Preserve secret-safe diagnostics using the library's public boundaries and Audita's domain artifact redaction; evaluate actual gaps with synthetic keys.
6. Review presence semantics field by field. Null endpoint, credential-source, and whole provider map now allow backend defaults; explicit empty strings clear endpoint/credential sources. Null does not suppress the entire backend provider map. Sampling fields have distinct semantics. Avoid blanket null or zero conversion and document intentional request changes.
7. Update captured-source accounting for list indexes and collection prefixes, including inherited/moved content-file origins. Preserve output-collision protection for consumed host documents and effective message files. Array positions are addresses, not durable resource identities.
8. Keep self-contained embedded response schemas. The new schema rule does not require a bundler for Audita's current schemas. Check custom artifact-reader ownership only where such a reader is actually used. Preserve native retries, structural repair, cancellation, deadlines, and shared engine admission while retaining Audita's domain reconciliation and partial-success outcomes.

Update configuration, LLM integration, runtime internals, prompt guidance, operator examples, diagnostics metadata documentation, and attribution with the implementation. Standard inspection replaces curated-only disclosure; prompts and provider settings may appear where the library's normal inspection exposes them, with native redaction. Do not accidentally promise old prompt hashes.

Acceptance and Upstream Requests

Verify native bundle inventory and selection, configuration/resource/prepared view distinctions, inheritance and replacement, credential precedence, optional fields, source-relative messages and collisions, and captured configuration. Use controlled providers to verify actual targets, domain schemas, shared admission, retry/repair budgets, cancellation, and partial success. Standard inspection must work without model calls; library presence alone does not prove successful Audita integration.

Verify backend defaults and explicit concurrency overrides through the assembled application, including shared admission across proposal and validation roles. Check that Audita no longer injects an additional limit or rejects native settings under its former finite-cap policy. Keep integration coverage for Audita's wiring; remove tests that only preserve retired local concurrency behavior.

Verify native bundled and custom backend selection without duplicate Audita registration, supported profile endpoint and credential overrides, and inspection without credential availability. Use controlled requests to cover required-key failures and optional unauthenticated execution under native semantics. Check that no application override layer or direct-key injection defeats native precedence, and that synthetic keys stay out of Audita diagnostics and artifacts.

Verify the embedded audita prompts work without external prompt definitions. Exercise both individual-prompt and complete-set replacements using native collection IDs, checking that unreplaced defaults remain available. Demonstrate that editing external prompt/message files changes a newly constructed runtime's prepared request without rebuilding the binary, while the existing runtime keeps its captured inputs. Check qualified-ID selection, effective prompt inspection, source provenance, and continued response-schema and domain validation with controlled responses. Include a maintained edit-and-rerun example for developers.

Verify that role selections are visible in application configuration inspection, resources through standard inspection, and completed request settings through preview. Remove tests and examples that only preserve the retired curated view; cover its replacement workflows through the assembled CLI.

No upstream request is yet selected. PromptKit is mature but pre-v1: significant blockers or awkward public boundaries can justify focused requests. Submit the minimal reproducer, desired outcome, and domain justification before building substantial local machinery. A review probe demonstrated distinct base/derived backend records preserving provider fields; that is an available native technique, not a decision to retain Audita's old backend ownership model.