audita/docs/roadmap/promptkit.md

PromptKit Adoption Outline

Updated: 2026-11-10

Status: Adoption direction and automatic operator-source selection resolved; implementation is complete. Follow the master roadmap for shared adoption principles and dependency identities, and the implementation plan for sequencing.

Baseline and Target

Before this work, Audita used PromptKit v0.14.0. Its LLM runtime uses a shared engine, application-owned backend overrides, curated profile inspection, and prepared execution.

Adopt PromptKit v0.21.0 through its native configuration and execution APIs. Retain the built-in OpenRouter backend and bundled profiles. They must be usable through the normal selection and credential mechanisms, not merely visible in an inventory while Audita rejects them for lacking a duplicate registration. Expose native resource and configuration inspection through the full standard ckprompt surface described in the CommandKit outline.

PromptKit owns resources, composition, provider settings and credentials, preparation, transport, structural repair, and backend admission. Audita owns proposal/validation role selection, transcript processing, domain decisions, response contracts, and truthful partial-success reporting. Remove the existing application-only endpoint and credential restrictions as described below.

Use the pinned migration guide, format reference, and v0.21 release notes. The bundle is default-on; older migration prose about catalog removal does not mean the current release has no bundled resources. It does not select an implicit execution profile or make provider calls during construction.

Agreed Direction and Migration Work

Backend Concurrency Ownership — Resolved

Adopt PromptKit's native per-backend concurrency behavior, including backend defaults and native configuration. The purpose is to keep requests within the concurrency a backend can accommodate; no separate Audita domain requirement has been identified for an additional limit.

Remove Audita-owned concurrency settings, default injection, validation, and enforcement wherever they duplicate or override that native behavior. Do not retain a second limiter or impose an Audita-specific finite-cap requirement on otherwise valid PromptKit configuration. Use the shared PromptKit engine so proposal and validation work for the same backend share its admission control. Audita still owns cancellation and deterministic processing around model calls.

The selected PromptKit configuration scope does not forward ConfigKit CLI overrides, and request overrides do not set concurrency. Configure limits on native backend resources. Architecture policy, runtime documentation, examples, and tests describe the adopted ownership and hard cutover.

Backend and Credential Ownership — Resolved

Native PromptKit backend/profile resources are authoritative for endpoints, provider settings, and credential selection. No domain-specific requirement has been identified for a separate Audita ownership policy.

• Remove Audita's separate backends mapping and generated backend override layer. Remove duplicate registration requirements and checks that force profile endpoints to match an Audita-owned backend endpoint. Native selection and validation govern bundled and operator-defined resources.
• Use PromptKit's native credential sources and precedence. Remove Audita's LLM-specific credential references, automatic direct-key injection, and rejection of profile-level environment lookups. Backend/profile documents contain environment-variable names, not credential values; operators may use the bundled OpenRouter lookup or configure their own names.
• Follow native credential availability timing. Loading and resource inspection must not require credentials for every configured backend. Preparation and execution enforce the selected request's native required/optional semantics: missing optional credentials permit unauthenticated requests. Do not add an eager all-backend credential check or silently strengthen those semantics.

Audita retains role/profile selection, operation budgets, domain request construction, response contracts, and safe handling of its reports and debug artifacts. Native configuration redaction does not replace redaction at Audita's artifact boundary. Keep preparing and executing the same captured request while retiring target checks whose purpose was enforcing the removed application backend registry.

Remove superseded fields, code, and tests in the major-release cutover and update the affected policies and current contracts with implementation. The ConfigKit outline owns Audita main-file discovery and general import policy; this outline owns the automatic PromptKit source selection below. PromptKit retains resource semantics under both policies.

Bundled Prompts and Operator Replacements — Resolved

Ship Audita's production default prompt definitions in embed.FS, exposed as io/fs.FS. Load them through PromptKit's filesystem inputs into a ConfigKit underlay, with operator sources at higher priority. Using the shipped prompts must not require a separate external prompt installation.

Place all Audita-owned default prompts in the native audita collection namespace. Keep local IDs such as modules.glossary.proposal; callers use the qualified runtime ID audita/modules.glossary.proposal. Update registry entries, call sites, references, examples, and diagnostics accordingly. Mounting a source does not rewrite references, so qualified references must be explicit.

Operators may replace individual prompts or supply a complete replacement set by defining the same collection and local IDs in higher-priority sources. Unreplaced prompts continue to come from the underlay. Follow native whole-record replacement semantics for matching IDs and document the required complete prompt definitions. The namespace identifies Audita's prompts without making them immutable or requiring a separate development mode.

The development workflow is edit external definitions and rerun Audita using the same binary. A newly constructed runtime captures the changed definitions and message files; live reload within an active run is not required. The same mechanism supports production customization. Standard inspection, provenance, and request preview must make effective prompts and their origins understandable. Audita's response schemas and domain validation continue to apply.

External source selection follows the automatic source contract below, using the ConfigKit import policy. This workflow is required PromptKit adoption scope and uses native composition.

Automatic Prompt Sources — Resolved

Automatically compose optional ./prompts.yml and YAML files recursively under ./prompts/, relative to the selected main application configuration file's directory. Follow the shared resource-base and overlay rule. When no main context exists, skip these automatic inputs and retain the embedded Audita prompts and bundled catalog; do not fall back to the working directory.

Supply one operator-source underlay with these native ConfigKit imports:

config:
  imports:
    - path: prompts.yml
      required: false
    - dir: prompts
      required: false
      recursive: true
      pattern: ["*.yml", "*.yaml"]

Construct the engine after application loading. Enable imports and supply the operator declaration's concrete filesystem context from ConfigKit's recorded application main source/context evidence, preserving its resolved lookup base, containment root, and source accounting. Input.MainRelative inside PromptKit refers to the engine's own main input; it does not automatically refer to Audita's separately loaded main. Do not feed the application document into PromptKit's schema or rediscover/reload it to obtain a directory. When no main context exists, omit the automatic operator inputs. Keep embedded prompt contexts separate. ConfigKit owns traversal, composition, inheritance, and provenance; do not add an Audita directory walker, YAML parser, or replacement mechanism.

Compose the engine's layers from lowest to highest priority:

1. PromptKit's default-on bundled OpenRouter catalog.
2. Audita's embedded audita prompt underlay.
3. The operator-source underlay containing both automatic imports.

Load both automatic locations when present. Missing optional sources contribute nothing; present but unreadable, invalid, or conflicting sources retain native errors. The file and directory compose as strict peers within one tier. Neither gains priority from declaration, filename, or discovery order; duplicate resource identities in that tier are errors. At the boundary above embedded defaults, native same-ID replacement replaces the complete matching record while retaining unmatched defaults. Follow the upstream source and profile precedence.

Use native PromptKit documents at the document root, with namespaces and IDs declared in their content. Each file may define one or several resources, including supported profiles and backends. Filenames and subdirectories do not assign identities or namespaces. Replacing an Audita prompt uses the same collections.audita namespace and local prompt ID as its embedded definition, as specified above; no source-selector setting is required.

Operators can use prompts.yml for definitions or as a manifest importing shared libraries elsewhere, and organize local definitions beneath prompts/. Nested imports and prompt content_file references retain native source-relative handling. Avoid importing the automatically discovered directory again from the manifest; repeated inclusion within the tier retains ConfigKit's native error.

Use one shared engine-loading path for processing, configuration/resource inspection, request preview, validation, and smoke commands. All must observe the same layers and captured definitions for the same invocation inputs. --config selects Audita's application configuration and therefore determines the companion lookup base. Selecting an application overlay does not relocate that base to the overlay's directory. All engine consumers share this handoff.

Profile Inspection and Summary — Resolved

Retire config print-effective --view profiles and its Audita-owned summary code. Use application configuration inspection to show proposal/validation profile selections, standard PromptKit resource inspection for profile/backend/ prompt definitions, and request preview for resolved execution settings and rendered messages. These views have distinct meanings and must be presented as such.

Do not add a replacement custom role summary at this stage. Native interfaces cover the selected needs and reduce maintenance without a parallel target resolver. Include the retired view and its standard replacements in the hard-cutover guidance. The CommandKit outline owns the operator surface and request-preview integration work.

Target Request and Source Boundaries

PromptKit resource sources use identified lists, native sibling extends, and complete same-ID replacement records. Qualified references remain explicit. Audita requests, registries, inspection, and diagnostics use prompt IDs without obsolete prompt-version selection or labels. The existing domain instructions, self-contained response schemas, per-call schema and repair controls, and proposal/validator behavior remain distinct from resource format changes.

Resource inspection shows native optional profile fields and backend definitions; prepared execution shows the completed request. Audita executes the captured request it prepared, without a parallel target resolver. Native field-specific null and presence rules govern defaults. Audita retains logical operation budgets, cancellation, truthful partial success, and safe diagnostics.

Output-collision protection accounts for consumed host documents, including imports, and effective source-relative content_file resources. Source metadata tracks inherited and moved values at native list-index and collection paths; array positions are addresses, not durable resource IDs. Native redaction governs standard inspection, while Audita owns its artifact redaction.

Canonical configuration, LLM integration, runtime, and prompt guidance will describe the implemented behavior and cutover, alongside updated examples and attribution. Standard inspection can show prompts and provider settings where the library exposes them; old prompt hashes are not a contract.

Acceptance and Upstream Requests

Verify native bundle inventory and selection, configuration/resource/prepared view distinctions, inheritance and replacement, credential precedence, optional fields, source-relative messages and collisions, and captured configuration. Use controlled providers to verify actual targets, domain schemas, shared admission, retry/repair budgets, cancellation, and partial success. Standard inspection must work without model calls; library presence alone does not prove successful Audita integration.

Verify backend defaults and explicit concurrency overrides through the assembled application, including shared admission across proposal and validation roles. Check that Audita no longer injects an additional limit or rejects native settings under its former finite-cap policy. Keep integration coverage for Audita's wiring; remove tests that only preserve retired local concurrency behavior.

Verify native bundled and custom backend selection without duplicate Audita registration, supported profile endpoint and credential overrides, and inspection without credential availability. Use controlled requests to cover required-key failures and optional unauthenticated execution under native semantics. Check that no application override layer or direct-key injection defeats native precedence, and that synthetic keys stay out of Audita diagnostics and artifacts.

Verify the embedded audita prompts work without external prompt definitions. Cover both automatic locations absent, each present alone, and both present with distinct resources, including nested YAML files. Exercise missing optional sources, unreadable/invalid sources, and same-tier identity conflicts through native loading. Confirm that explicit and discovered application mains elsewhere anchor imports beside that main, independently of the working directory. With no main context, confirm automatic imports are skipped while bundled and embedded resources remain available. Verify the recorded-context handoff to the separate engine, including source accounting and overlay selection without relocation. Check manifest imports of shared libraries and source-relative message files, with provenance and consumed-source collision protection intact. Exercise both individual-prompt and complete-set replacements using native collection IDs, checking that unreplaced defaults remain available. Demonstrate that editing external prompt/message files changes a newly constructed runtime's prepared request without rebuilding the binary, while the existing runtime keeps its captured inputs. Check qualified-ID selection, effective prompt inspection, source provenance, and continued response-schema and domain validation with controlled responses. Verify that processing and standard inspection/preview use the same effective replacements and that the bundled catalog remains available. Include a maintained edit-and-rerun example using the automatic sources, without requiring a source-selector setting or binary rebuild.

Verify that role selections are visible in application configuration inspection, resources through standard inspection, and completed request settings through preview. Remove tests and examples that only preserve the retired curated view; cover its replacement workflows through the assembled CLI.

No upstream request is yet selected. PromptKit is mature but pre-v1: significant blockers or awkward public boundaries can justify focused requests. Submit the minimal reproducer, desired outcome, and domain justification before building substantial local machinery. A review probe demonstrated distinct base/derived backend records preserving provider fields; that is an available native technique, not a decision to retain Audita's old backend ownership model.