diff --git a/docs/roadmap/inspect-backend.md b/docs/roadmap/inspect-backend.md index eac829c..c57bb48 100644 --- a/docs/roadmap/inspect-backend.md +++ b/docs/roadmap/inspect-backend.md @@ -9,8 +9,9 @@ presentation; Promptkit exposes the selected resource's resolved configuration. The direction is accepted for planning; implementation has not begun. Selected work includes fuller prompt inspection, profile inspection of the resolved profile itself, a new `InspectBackend`, `BackendConfig.Extends`, typed prompt -construction through `PromptConfig`/`PromptSource`, and simplification of -Promptkit's post-handoff null handling. Execution-policy reporting +construction through `PromptConfig`/`PromptSource`, generically renderable +inspection results, and simplification of Promptkit's post-handoff null handling. +Execution-policy reporting and prepared-request enhancements have been removed from this roadmap. The decisions below record the accepted scope and representation approach. @@ -76,9 +77,8 @@ values, including zero. Absence is not displayed as zero or `unspecified`. Treat null like omission for nullable resource fields after ConfigKit resolution, under the selected post-handoff semantics below. Literal nulls inside provider data retain their meaning; they are not optional resource-field presence states. -The previous draft's rule to omit suppressed outbound controls belonged to an -execution report and does not define resource inspection. Accepted decision 4 -selects typed results without a third state for nullable resource fields. +Accepted decision 4 selects typed results without a third state for nullable +resource fields. Inspection reports effective resource values rather than original YAML layout, comments, or the history of each inherited field. `InspectConfiguration` remains @@ -90,10 +90,52 @@ Retain existing captured-state, namespace, ownership, and credential-handling contracts unless a change is deliberately selected. Mutable results should not let a consumer change retained engine state; that does not require repeated copies between private helpers or copying immutable strings. Ordinary resource -inspection needs no rendering, provider calls, or credential availability checks. +inspection needs no template rendering, provider calls, or credential +availability checks. Templates and arbitrary provider settings remain application data whose presentation and disclosure are controlled by the consuming application. +## Generic Rendering And Presence + +**Requirement:** Consumers must be able to render `InspectPrompt`, +`InspectProfile`, and `InspectBackend` results generically, without maintaining +parallel resource schemas or field-by-field adapters. Keep typed Go results for +programmatic use and make them directly encodable as JSON and YAML using +ordinary encoders. A consumer calls inspection, handles the error, and passes +the result to its selected encoder; it need not interpret individual settings +or register each nested Promptkit type. + +Promptkit owns serialization field names, nesting, and omission semantics for +these results. Use consistent names aligned with configuration fields where +applicable. CommandKit owns command selection, output format, presentation, and +delivery. Adding an inspection field should not require another CommandKit +field mapping after updating its Promptkit dependency. Consumers may still +choose custom layouts, but generic rendering must be sufficient for a complete +resource report. + +Capture presence before converting configured values to execution values that +collapse omission into zero, false, or an empty string. This applies across all +three resource kinds, including prompt input `required`, output +`repair_attempts`, and backend capacity settings. Use small inspection-specific +values where existing types such as `PromptInputDefinition` or `OutputContract` +cannot preserve that distinction. Do not change shared execution types merely +to make them suitable for inspection. Existing configuration validation rules, +including rejection of null for non-nullable fields, remain intact. + +Omit absent optional settings and nullable fields resolved to null. Retain +explicit zero, false, empty strings, and explicit empty collections where +represented in the resolved resource, along with literal nulls inside provider +data. Generic encoding must preserve these distinctions; a blanket `omitempty` +on scalar or collection fields is not sufficient. Prefer ordinary optional +fields and serialization tags, with focused encoding support only where needed +for the selected semantics. Consumers must not reconstruct presence themselves. + +This is a modest serialization contract for inspection results, not a new +reporting framework, configuration round-trip API, or promise of byte-for-byte +output stability. Formatting choices such as indentation and mapping-key order +belong to the encoder or consumer; semantic sequence order remains meaningful. +Promptkit need not provide CLI layouts or a generic reflection framework. + ## Expanded Prompt Inspection Expose description, ordered message templates with roles and cache-control @@ -158,9 +200,12 @@ Remove the existing special treatment of null in these profile fields: Retain explicit values, including zero, false, and the existing empty-string clearing behavior for `endpoint` and `api_key_env`. Retain the ordinary map overlay behavior: `extra_params: {}` does not clear backend defaults, and -`extra_params: ""` is not a valid mapping. Whole-map suppression of backend -parameters is deliberately removed; do not replace it with another clearing -option without a concrete requirement. +`extra_params: ""` is not a valid mapping. Profile-level suppression of the +whole backend provider-parameter map via null is deliberately removed; do not +replace it with another clearing option without a concrete requirement. +Operators remain free to remove or replace parameters in backend definitions +through ordinary ConfigKit composition. This does not prohibit those workflows +or change ConfigKit's null semantics. Null entries within arbitrary provider maps and arrays remain literal provider data. This is not recursive null removal. Required-field checks and fields that @@ -212,17 +257,22 @@ without combining their settings into the inspected resource. Stored references retain their exact meaning; caller namespace defaults do not rewrite them. A particular connected layout is a consumer choice, not a new Promptkit API. -Coordinate CommandKit adoption of the changed profile result and richer prompt -and backend views. In particular, migrate any formatter that currently expects -`EffectiveModelParams`. The profile semantic change must be documented as such, -not presented as merely adding more fields. Document the Go/serialization -migration for the typed result selected in decision 4 and follow the existing -release policy for compatibility changes. +Coordinate CommandKit adoption of generically rendered resource results, +replacing parallel report structures and per-field conversions where they are +no longer needed. In particular, migrate consumers of `EffectiveModelParams`. +Also account for prompt inspection fields whose types need to change to retain +presence; compatibility work is not limited to `ProfileInspection`. Document +the Go and serialized-result migrations with the final types, and follow the +existing release policy. The profile semantic change must be documented as such, +not presented as merely adding more fields. Document the post-handoff null change as a configuration compatibility change as well. Existing null endpoint/credential-source clearing can use the existing explicit empty-string form when that behavior is desired. There is no new -replacement for whole-map suppression of backend provider parameters. Update +replacement for profile-level whole-map suppression of backend parameters. +Include a migration example showing that a profile's null now permits backend +endpoint, credential-source, or provider-parameter fallback that it previously +suppressed; this changes execution as well as inspection. Update the [format reference](../formats.md#defaults-and-overrides) when implementation lands; it describes the current behavior until then. @@ -294,9 +344,9 @@ source round-tripping are not selected deliverables. ## Decisions -All five decisions below are resolved, including the post-handoff null -simplification in decision 4. Numbering is retained for continuity with the -discussion; implementation is pending. +All five decisions below are resolved, including generic rendering and the +post-handoff null simplification in decision 4. Numbering is retained for +continuity with the discussion; implementation is pending. ### 1. How Should An Unknown Backend Be Classified? @@ -367,6 +417,8 @@ is unnecessary; raw mappings continue to serve those cases. suitable public nested values. Change `ProfileInspection` deliberately to report the resolved profile rather than retaining a second execution-settings view for compatibility. Apply the same resource semantics to prompt/backend additions. +Require generic rendering under the contract above; typed access must not force +consumers to maintain parallel report types or interpret every setting. Represent absent settings and explicit values, including zero, false, and empty strings where valid. Treat nullable resource fields set to null as absent in @@ -381,7 +433,7 @@ represent the uncompleted profile and must not substitute for it. consumers interpret a generic configuration mapping. Endpoint and credential overrides already take precedence without first clearing a value, and explicit empty strings already support clearing those settings. No concrete requirement -justifies retaining whole-map suppression of backend provider parameters. +justifies retaining profile-level whole-map suppression of backend parameters. Removing these exceptions simplifies interpretation and avoids carrying source syntax and inheritance history into every field. @@ -391,10 +443,10 @@ typed discoverability and introduces a broader mapping-value contract. It is not selected as the primary resource-inspection result; `InspectConfiguration` remains available for explicit configuration/provenance inspection. -Document migration for existing `EffectiveModelParams` consumers and any -serialization changes with the final Go types. CommandKit owns report formatting; -new JSON contracts or custom serializers are not automatically required. Do not -leak internal package types or mutable captured state. +Document migration for existing profile and prompt inspection consumers with +the final Go types and their serialization semantics. Generic JSON/YAML encoding +is selected; bespoke consumer reports and custom serializers are not inherently +required. Do not leak internal package types or mutable captured state. ### 5. Backend Diagnostics Show Configured Values, Not Derived Destinations @@ -409,25 +461,34 @@ in resource inspection. ## Implementation And Verification -1. Define the typed results and migration selected in decision 4. Source-input - helper types and inspection results have distinct purposes; select their - shapes accordingly, preserving explicit values without a null presence state. -2. Simplify Promptkit's post-handoff handling of nullable fields and remove - presence machinery made unnecessary by that change. Retain presence needed - to distinguish omission from explicit zero, false, or empty values. Expose - expanded prompt content and resolved profile fields through existing - capture/lookup paths, before execution completion, without another - composition pass or changes to ConfigKit's retained result. -3. Add backend inspection and the accepted public missing-backend error, +1. Define typed results, serialization names/nesting, and consumer migration + selected in decision 4. Account for presence across prompt, profile, and + backend results. Source-input helpers, inspection results, and execution + values have distinct purposes; reuse types only where their semantics fit. +2. Implement and verify the post-handoff null change as a focused unit, separate + from inspection projection. Remove presence machinery made unnecessary by + that change while retaining omission versus explicit-value information. + Verify the changed execution behavior and document its migration example. +3. Expose expanded prompt content and resolved profile fields through existing + capture/lookup paths, before execution completion, without another composition + pass or changes to ConfigKit's retained result. Support generic encoding with + the selected omission rules and preserve explicit values. +4. Add backend inspection and the accepted public missing-backend error, returning configured diagnostic metadata without endpoint derivation. -4. Add typed backend inheritance and the prompt source/helper types, delegating +5. Add typed backend inheritance and the prompt source/helper types, delegating both to the ordinary ConfigKit mapping and loading path. -5. Update GoDoc, formats, consumer/internal guidance, and release notes for the - selected behavior, profile-result migration, and null compatibility change. - Coordinate CommandKit reporting. +6. Update GoDoc, formats, consumer/internal guidance, and release notes for the + selected behavior, inspection-result migrations, and null compatibility + change. Coordinate CommandKit's generic reporting adoption. Use focused offline tests for actual boundary risks: +- Directly encode each inspection result with ordinary JSON and YAML encoders, + without consumer-specific report structures. Verify semantic field names, + nesting, omitted settings, explicit zero/false/empty values, and nested + provider data. Include prompt input/output and backend capacity presence, + rather than testing only profile fields. Assert semantic output, not incidental + whitespace or mapping-key order. - Child resources show the resolved inherited fields and overrides, not an unresolved `extends` directive; explicit zero survives. Omitted/null nullable settings use the same absent representation; provider-data nulls retain their @@ -452,13 +513,15 @@ Use focused offline tests for actual boundary risks: relative content-file case using ordinary source context. Reuse generic loading, validation, and inheritance coverage rather than duplicating it for each helper. -CommandKit owns presentation tests for the migrated resource view. Follow the +Promptkit owns the inspection serialization checks; CommandKit owns presentation +and delivery tests for the migrated resource view. Follow the [testing policy](../policy/testing.md), combine cases where useful, and reuse existing coverage. No new benchmark threshold or release gate is selected. Completion means operators can inspect the resolved selected prompt, profile, -or backend accurately without assembling execution settings or navigating raw -configuration paths. All three resource kinds have typed source helpers, -including inheritance support, and consumers have migration guidance for changed -profile inspection and post-handoff null semantics. +or backend accurately through generic rendering, without per-field consumer +adapters, assembling execution settings, or navigating raw configuration paths. +All three resource kinds have typed source helpers, including inheritance +support, and consumers have migration guidance for changed inspection results +and post-handoff null semantics. No engine-policy or prepared-request additions are part of completion.