diff --git a/docs/roadmap/inspect-backend.md b/docs/roadmap/inspect-backend.md index 89ba8eb..f46ec4f 100644 --- a/docs/roadmap/inspect-backend.md +++ b/docs/roadmap/inspect-backend.md @@ -2,114 +2,132 @@ ## Purpose And Status -Give operators using CommandKit-based applications a complete, accurate view of -Promptkit's effective prompt, profile, and backend configuration. CommandKit is -the primary consumer of these inspection APIs and owns operator-facing commands -and formatting. Promptkit owns resolved typed facts and their semantics. +Help operators using CommandKit-based applications understand Promptkit's +captured prompt, profile, and backend configuration and the settings applicable +to a concrete request. CommandKit owns operator-facing commands and presentation; +Promptkit supplies facts from its existing configuration and execution paths. -The selected direction is accepted for planning; implementation has not begun. -Expand `InspectPrompt`, expose setting presence in profile and prepared results, -and add `InspectBackend` with effective diagnostic destinations. Retain the -planned `BackendConfig.Extends` convenience addition. Remaining decisions are -listed below; typed prompt construction and listing return-type changes are -lower-priority considerations, not prerequisites for operator inspection. +The direction is accepted for planning; implementation has not begun. Selected +work includes expanded prompt inspection, presence-aware profile/prepared +inspection, backend inspection with effective diagnostic destinations, and +`BackendConfig.Extends`. Open decisions below cover unresolved scope and public +representation choices. They are not implicit implementation requirements. -This use case supersedes the earlier recommendation to keep prompt inspection -limited to interface metadata and backend diagnostics limited to declared -settings. Operators need to see captured message content and the destinations -that default diagnostic rules select. The implementation should reuse existing -capture, resolution, and adapter logic rather than build a second resolver. +## Design Balance -## What Fully Resolved Means +Provide useful information already captured or cheaply derived. Prefer exposing +existing values and small shared helpers over introducing new resolution paths, +parallel report models, provenance tracking, or retained snapshots solely for +inspection. This is a design preference, not a prohibition on a small adapter +or additional field when it serves a demonstrated consumer need. -| View | What can be resolved | +“Fully resolved” describes the applicable resolution stage. It is not a promise +that inspection can reconstruct the entire authored source, report every future +configuration field automatically, predict provider defaults, or reproduce a +future invocation independently of its request and external state. + +| View | Information available | | --- | --- | | Resource inspection | Captured values after composition, inheritance, normalization, and applicable Promptkit defaults. | -| Concrete request preparation | Selected profile, request overrides, rendered messages, inputs, session ID, and loaded schema for supplied request data. | +| Concrete request preparation | Selected profile, request overrides, rendered messages, input metadata, session ID, and loaded schema for supplied request data. | | Execution | Provider responses, actual usage, and other observed runtime facts. | -A prompt ID alone cannot determine future request overrides or template values. -Resource inspection therefore shows effective templates and references, while -preparation shows the concrete request. Neither should invent provider defaults, -resolve a moving model alias to a hypothetical future model, or claim execution -success from configuration alone. +The planned fields below define the initial additions. Additional cheap, +useful fields can be considered during implementation without building a +uniform resource framework or promising exhaustive future coverage. If a field +requires substantial new machinery, record the tradeoff before expanding scope. -**Reporting decision:** Omit unset optional settings from operator reports. -Retain explicitly configured values, including zero, and effective Promptkit -configuration defaults. Do not print omitted controls as zero or as -`unspecified`. Suppressed provider controls should likewise be absent from an -effective-settings report. This is a presentation of execution behavior; -`InspectConfiguration` remains available when an operator wants the authored -null, omission, or inheritance history. +### Review Findings And Existing Boundaries -Promptkit must expose sufficient typed presence information for CommandKit to -make those choices correctly. A zero-valued Go field alone is not sufficient. -This does not remove fields from existing public types or silently change their -existing JSON representation; CommandKit owns its report projection. +The current implementation provides a relatively small path to the core work: -## Relationship To Existing APIs +- Captured prompt definitions already retain description, session template, + message text, roles, and cache-control metadata alongside compiled templates. + Inspection can expose that data without reconstructing source files or ASTs. +- Internal prepared results already retain `TargetPresence`; the root API has + an `ExecutionTargetPresence` type. Presence-aware output need not introduce + a second interpretation of configuration or provider-value conversion. +- Smoke selection already has endpoint derivation, including the OpenRouter + key URL helper. Reusing or factoring that logic need not create an adapter + capability registry or provider metadata service. +- Transport timeout and retry policy live in different components and are not + all carried by prepared results. Their reporting scope and cost need a + decision; they are no longer an unconditional completion requirement. -| API | Responsibility | -| --- | --- | -| `ListBackends` | Discover captured backend identities. | -| Proposed `InspectBackend` | Describe one backend's effective settings by ID. | -| Expanded `InspectPrompt` | Describe the captured prompt, including effective template content. | -| Enhanced `InspectProfile` | Report backend-completed settings with the presence information needed for accurate output. | -| `Prepare` / `PreparedExecution.Details` | Describe a concrete prepared request, including setting presence. | -| `InspectConfiguration` | Inspect effective configuration and source provenance. | -| `SmokeTest` | Execute an explicitly requested diagnostic and report evidence. | +These observations are implementation evidence, not promises to retain those +private types or helper arrangements. See the current +[source/capture](../internal/sources.md), [runner](../internal/runner.md), and +[model-client](../internal/llm.md) descriptions. -See the [consumer guide](../consumers/pkg-promptkit.md#discover-available-resources-and-collections) -for the implemented inspection and discovery boundaries, and the -[architecture policy](../policy/architecture.md) for library responsibilities. +Retain established public ownership, credential handling, captured-state, and +namespace behavior unless a change is deliberately selected. Independent +mutable results prevent callers from changing engine state; they do not require +fresh copies of immutable strings or repeated deep copies between private layers. +Existing `Inspect*` APIs avoid provider work and request preparation. Extending +their returned data should not silently change those behaviors. These boundaries +follow the existing API and the selected operator use case, rather than a new +requirement that all future inspection APIs have identical behavior. -## Complete Prompt Inspection +## Reporting Semantics + +**Selected presentation:** Omit unset optional settings from operator reports. +Preserve explicit values, including zero, and applicable Promptkit defaults. +Suppressed provider controls are also absent from an effective-settings report. +Neither zero nor `unspecified` should stand in for an omitted control. + +This concerns the effective setting, not an explanation of every reason it was +omitted. ConfigKit inspection remains useful for authored values and history. +Promptkit supplies sufficient presence information; CommandKit chooses report +layout. Representation and serialization choices remain open in decision 4. + +Do not infer provider defaults from absent controls. Likewise, a configured +model alias is the alias, not a prediction of the model it will select later. +Credential source names and requirements are useful inspection facts; resolved +credential values remain excluded under existing contracts. Templates and +arbitrary provider parameters are application data, not automatically safe +for disclosure. No new general-purpose sanitization guarantee is selected. + +## Expanded Prompt Inspection Expand `PromptInspection` to include description, ordered message templates -with roles and cache-control metadata, and the session-ID template. Retain -identity, opaque prompt hash, default profile reference, declared inputs, and -the complete output contract, including schema reference and repair settings. +with roles and cache-control metadata, and the session-ID template. Retain the +existing identity, opaque hash, default profile reference, input declarations, +and output contract, including schema reference and repair settings. -For messages imported from text or Markdown files, return the effective content -captured at engine construction. Operators should not have to locate or reopen -the original file to see what execution will render. Return templates as text, -not internal parsed-template objects. Inspection neither renders templates nor -rereads content files; edits require a new engine. The schema remains a -reference here; concrete preparation loads the selected schema. +Expose effective content captured from imported text or Markdown files, as well +as inline content. Return the template text available in the captured definition; +source-layout reconstruction, original YAML formatting, and parsed-template +objects are unnecessary for this use case. There is no new byte-for-byte source +round-trip requirement. Message order and text should reflect what rendering +consumes. Inspection continues to use captured state without rendering or +reopening files; the schema stays a reference until ordinary preparation loads it. -Keep prompt inspection independent of profile selection and request readiness. -An absent or incomplete default profile must not hide an otherwise captured -prompt. A prompt without a default profile remains inspectable. Return owned -mutable results, preserving message and input order and request isolation. +Prompt inspection remains useful without a supplied request or a resolvable +default profile. The existing independent lookup permits CommandKit to show a +prompt even when related execution settings need attention. ## Presence-Aware Profile And Prepared Inspection -`ProfileInspection` currently exposes effective model settings but not the -`ExecutionTargetPresence` information that distinguishes omitted controls from -explicit zero values. `PreparedRun` has the same reporting gap. Expose the -existing presence semantics alongside the effective target in both results, -including the `PreparedRun` returned through `PreparedExecution.Details`. +`ProfileInspection` exposes effective settings without the presence information +that execution uses for explicit-zero controls. `PreparedRun` has the same +public reporting gap, including values returned by `PreparedExecution.Details`. +Expose enough existing resolved presence information to distinguish omission +from explicit settings in each of those results. -Use the resolved target and presence captured by the execution path; do not -infer presence from numeric values or re-run ConfigKit composition. Preserve -backend completion, profile inheritance, explicit suppression, and request -precedence. Keep credential requirement metadata distinct from the credential -source name; no credential value is returned or checked by resource inspection. +For example, an unset temperature produces no temperature entry in the report; +an explicit zero produces `temperature: 0`. A concrete request's override appears +in its prepared view. Backend completion, inheritance, and request precedence +remain the execution path's responsibility. Inspectors need not reconstruct +that history or distinguish all historical reasons for an absent setting. -For example, the report for a profile that leaves temperature unset should have -no temperature entry. A profile explicitly setting temperature to zero should -show `temperature: 0`. A concrete request that overrides that profile should -show its effective setting instead. Presence metadata is for accurate report -construction; CommandKit need not expose an implementation-oriented bit map to -operators. +Reuse the captured target and presence semantics, including existing handling +of nonzero values. Decision 4 weighs exposing the current presence type against +another public representation; it does not reopen the omission-reporting decision. +See [defaults and overrides](../formats.md#defaults-and-overrides). -Keep the meaning of effective defaults explicit. A configured zero generation -timeout disables that particular deadline; it does not disable the transport -cap or caller cancellation. An omitted provider control supplies no assertion -about the provider's own default. See the -[defaults reference](../formats.md#defaults-and-overrides). +## Backend Inspection -## Proposed Backend Inspection Contract +Proposed entry point: ```go func (e *Engine) InspectBackend( @@ -118,286 +136,225 @@ func (e *Engine) InspectBackend( ) (*BackendInspection, error) ``` -Use a dedicated public result rather than returning `BackendConfig`. -`BackendConfig` is an input type whose pointers express omitted and explicit -values; inspection describes the normalized effective definition after -composition and defaulting. Do not expose internal domain types. +The result should expose the full backend identity, base endpoint, credential +environment-variable name, provider defaults, effective concurrency/queue +settings, and diagnostic settings and destinations. These are backend defaults, +not settings after a particular profile or request overrides them. Capacity +values describe policy, not current occupancy. -The proposed result includes: +A dedicated top-level inspection result is recommended because `BackendConfig` +is a presence-aware source input. This does not require a new type for every +nested value: reuse suitable public values where their semantics match. Exact +field names and nesting can be selected during implementation within the public +representation decisions below; internal types remain behind the facade. -| Information | Meaning | -| --- | --- | -| `BackendID` | Full selected runtime identity. | -| `Endpoint` | Captured backend base endpoint. | -| `APIKeyEnv` | Configured environment-variable name, never its resolved value. | -| `ExtraParams` | Backend provider defaults, before profile or request overrides. | -| `ConcurrencyLimit` | Configured generation concurrency limit. | -| `QueueCapacity` | Effective queue capacity, including construction-time defaults. | -| `SmokeTests` | Diagnostic settings and effective destinations, including the authentication kind and optional inference default profile reference. | - -Scalar fields describe effective values rather than reproducing source-field -presence. Diagnostic optionality must remain expressible, including no configured -authentication adapter or inference default. Exact nested type names are -implementation choices. -Follow existing typed inspection conventions without adding a stable JSON -serialization contract for this feature. Applications can define their own -output formats. - -### Selection And Errors - -Follow the existing namespace selection rules: a scoped engine view qualifies -slashless IDs with its backend namespace; qualified IDs pass through unchanged. -An unscoped view looks up the supplied ID directly. Return the full selected ID. -Do not add fallback search across namespaces, implicit backend selection, or -special handling for bundled backends. Use the existing ID rules, including -case sensitivity, rather than introducing new normalization. - -A nil or zero-value engine should match `ErrInvalidConfig`; a blank or otherwise -invalid selection should match `ErrInvalidRequest`. Cancellation should retain -the context error, and failures should return no partial result. The public -classification for a missing backend is open decision 1. +Follow existing namespace-view selection: qualify slashless caller IDs with the +backend scope, pass qualified IDs through, and report the selected full ID. +Use the same identity rules for bundled and caller-supplied backends. The new +operation should follow existing inspection conventions for invalid engines, +invalid selections, and context cancellation, without adding cross-error +precedence guarantees. Missing-backend classification is open decision 1. ### Effective Diagnostic Destinations -Include the effective reachability destination and, when an authentication -adapter applies, its effective destination. Apply explicit endpoint overrides -and existing backend-relative defaults through the same derivation logic used -by diagnostic execution. CommandKit must not reconstruct provider-specific URL -rules. Include the configured authentication kind and inference default profile -reference; an inapplicable authentication kind has no active probe destination. +Include effective reachability and applicable authentication destinations, +using explicit overrides or existing backend-relative defaults. Share the +existing derivation logic where practical; neither CommandKit nor a new +inspection-specific resolver should maintain a competing set of provider rules. +A small helper extraction is sufficient if that is all the current adapters need. -Default reachability can apply even without a `smoke_tests` block. The result -must be able to report that effective destination without fabricating an authored -block or an authentication configuration. Where distinguishing an explicit -endpoint override from a derived destination is useful, keep those facts -separate rather than overloading a field's meaning. Source history remains -owned by `InspectConfiguration`. +Default reachability applies even without authored `smoke_tests` metadata. +Report the configured authentication kind and optional inference-default profile +reference without fabricating authored settings. An authentication kind of +`none` has no active authentication destination. Decision 5 covers whether the +result also needs separate declared endpoint fields. -These are destinations under the backend's captured settings, not guarantees -about runtime overrides, the selected client's HTTP diagnostic support, or -reachability. Report a default inference profile reference without resolving it; -a missing or incomplete referenced profile must not make backend inspection fail. +Destinations describe captured backend settings, not provider reachability or +HTTP diagnostic support in an injected client. Retain the inference-default +profile as a reference; backend inspection need not resolve it or establish +inference readiness. Ordinary inspection is expected to return a result or an +error, following existing methods. If derivation reveals a meaningful case where +useful captured settings would be lost, bring that case back for a decision +rather than automatically adding partial-result and warning machinery. -### Captured State And Ownership +## Connected Views And Concrete Requests -Inspect the engine's captured backend definition without reloading sources, -reading credential values, preparing a prompt, acquiring capacity, or making -provider calls. Engine construction already validates effective definitions; -inspection should not repeat that work or create an additional acceptance path. +CommandKit can follow prompt default-profile and selected-backend references +through the existing inspections to build a connected view. This is a suggested +consumer workflow, not a required report format or a restriction on future +convenience APIs. Use stored references exactly; namespace defaults apply to +caller selections, not to reinterpreting captured references. Endpoint-only +profiles need no backend node. A consumer can retain successful lookups while +showing a failure for a related resource. -Capacity fields describe configuration, not current occupancy or queue length. -Diagnostic settings describe configuration, not provider availability, key -acceptance, or inference readiness. In particular, report a smoke-test default -profile reference without resolving or validating that profile merely to inspect -the backend. Its selection-time behavior remains owned by `SmokeTest`. +Use existing `Prepare` for concrete variables, inputs, profile selection, and +request overrides. It performs artifact/schema reads, rendering, and applicable +credential checks without inference. `PrepareExecution` and its details serve +callers that want later execution tied to the same preparation, subject to that +API's existing contract. This roadmap adds no stronger replay guarantee and +selects no second preparation workflow. -Return independently owned mutable values, including nested provider maps and -diagnostic settings. Mutating a result must not affect the engine or another -inspection. Reuse existing ownership operations and avoid consecutive copies -whose only purpose is to cross private helper boundaries. +Model parameters are not all execution policy. Per-generation timeout, transport +cap, retry settings, backend capacity, and caller deadlines are different facts. +When reported, keep their meanings distinct. Decision 2 determines whether and +how this round should expose additional engine-policy information. Report known +settings without claiming knowledge of an injected client's internal behavior. -Provider defaults should retain their values, consistent with typed target -inspection. They are application data and may contain sensitive content; -the result is not automatically safe for logging or disclosure. Do not resolve -credentials or silently substitute ConfigKit's redacted reporting values for -the typed settings. Operator provenance and redacted configuration reporting -remain available through `InspectConfiguration`. - -## Connected Reports And Concrete Request Preview - -CommandKit can assemble a connected report by following prompt default-profile -and selected-backend identities through the three inspection APIs. Promptkit -supplies the references and effective values; CommandKit chooses nested sections -or links. Use captured references exactly, without requalifying them through a -view's caller-selection namespace. Endpoint-only profiles need no backend node. -A missing related resource can be reported while preserving successfully -inspected resources; do not make each resource inspection recursively dependent -on all related objects. - -Use `Prepare` for an operator preview with concrete variables, inputs, profile -selection, and request overrides. It performs artifact/schema reads, rendering, -and applicable credential checks without inference. Do not describe it as a -pure configuration lookup. Use `PrepareExecution` and its existing details when -later execution must use the same captured preparation; a separate preview -followed by an independent run is not a promise that mutable inputs stay equal. -No second request-resolution API is selected by this roadmap. - -An execution report must distinguish per-generation timeout, built-in transport -cap, retry policy, and backend capacity. Effective model parameters alone are -not the entire execution policy. Caller deadlines and injected-client behavior -can add constraints that Promptkit cannot predict for a future invocation. -How to expose the missing engine-policy facts is open decision 2. Do not infer -them from model fields or ask CommandKit to duplicate Promptkit defaults. - -CommandKit owns text/YAML presentation and report disclosure. Prompt templates, -input content, and arbitrary provider parameters may be sensitive; preserve -existing credential exclusions without claiming that every inspected value is -safe to publish. No CommandKit implementation is included in this repository's -roadmap; coordinate its consumer adaptation when the Promptkit APIs land. +CommandKit owns CLI commands, formatting, and disclosure. Coordinate downstream +adoption when the Promptkit additions land; this roadmap does not prescribe +CommandKit's internal adapter design or require a new stable CLI serialization +contract in Promptkit. ## Typed Backend Inheritance Add `Extends *string` to `BackendConfig`, matching `ProfileConfig.Extends`. -When non-nil, `BackendSource` should encode it as `extends: {id: value}` in the -backend record. Nil omits the field. Preserve the supplied value and let -ConfigKit resolve and validate inheritance through the existing source-loading -path, just as it does for profiles. +`BackendSource` should encode a non-nil value as `extends: {id: value}` and omit +the field when nil. ConfigKit owns inheritance resolution and its diagnostics. +The helper exposes existing sibling inheritance, rather than performing parent +lookup or filling inherited fields itself. -The purpose is to let programmatic callers derive a backend variant without -switching from `BackendSource` to a raw mapping. The helper must not resolve a -parent, fill inherited fields, or become a second inheritance implementation. -It remains a source constructor rather than an engine registry mutation API. +Keep existing root/collection identity, replacement, and null semantics. Raw +mappings remain available for explicit null and other generic controls outside +the helper's convenience surface. See the +[inheritance reference](../formats.md#resource-inheritance) and +[profile helper](../../profiles.go). -This exposes existing sibling inheritance, not a new configuration feature. -Parent IDs select siblings in the same root or collection resource list; -namespace views do not qualify these configuration references. Existing -same-ID replacement, omitted-field, and explicit-null behavior remain owned by -ConfigKit composition. Raw mappings remain available when callers need explicit -null or other generic configuration controls that the typed helper cannot express. -See the [inheritance reference](../formats.md#resource-inheritance) and the -existing [profile source helper](../../profiles.go). +## Related Work And Scope -## Scope Of API Consistency +Typed prompt construction is open decision 3. Listing return types remain a +possible later cleanup: `ListPrompts` returns an ID-only `PromptIdentity` wrapper, +while backend/profile listings return strings. Standardizing on strings may be +useful at a breaking-change window, but is not required for these inspections. -Backends, profiles, and prompts already share source layering, collection -membership, namespace selection, sibling inheritance, and construction-time -definition validation. The gaps addressed here are in public convenience and -inspection APIs, not in those underlying configuration capabilities. - -Inspection remains purpose-specific while becoming complete for operator use: -prompt templates are captured but unrendered, profile settings include backend -completion, and backend settings include effective diagnostic destinations. -Provenance and authored representation stay with `InspectConfiguration`. - -`ListPrompts` currently returns `[]PromptIdentity`, whose only field is `ID`; -`ListProfiles` and `ListBackends` return `[]string`. At a suitable breaking-change -window, prefer considering `[]string` for all three over adding thin identity -wrappers for backends and profiles. That would simplify consumer code, but -requires a deliberate compatibility decision and migration instructions. Keep -the existing return types in this work; listing consistency does not block the -backend additions. - -Collections, schemas, and engine configuration remain distinct concepts rather -than additional peers requiring identical resource APIs. Prompt requirement -inspection and backend capacity/diagnostic settings retain their specialized -roles. Profile/backend hashes are not added for symmetry; the existing +Collections, schemas, and engine options have different roles; symmetry does +not require identical APIs for them. The [deferred fingerprint proposal](deferred.md#semantic-execution-target-fingerprints) -owns that separate question. +remains separate. Live capacity monitoring, provider health guarantees, and +source round-tripping are not selected deliverables, rather than permanent +prohibitions on future work. ## Open Decisions ### 1. How Should An Unknown Backend Be Classified? -**Recommendation:** Add public `ErrBackendNotFound` for this explicit lookup, -matching the role of `ErrProfileNotFound` and `ErrPromptNotFound`. The registry -already has an internal backend-not-found sentinel. Map it at the facade so -applications can distinguish a missing backend from malformed input without -inspecting text or internal packages. +**Recommendation:** Add public `ErrBackendNotFound`, analogous to the existing +prompt/profile sentinels. The registry already identifies missing backends; +public mapping lets consumers distinguish a missing ID without parsing text. +Keep existing operations' classifications unless a change is explicitly chosen. -Limit this addition to the new inspection operation unless existing callers -justify a broader change; do not automatically reclassify smoke-test or engine -construction errors. +**Alternative:** Use `ErrInvalidRequest`, avoiding another exported sentinel but +combining malformed selection and a well-formed missing identity. Consumers that +need the distinction would require inventory lookup or other error detail. -**Alternative:** Classify an absent ID as `ErrInvalidRequest`. This avoids -another exported sentinel but combines a well-formed missing identity with an -invalid selection. Consumers needing different presentation would then require -an inventory lookup or additional error detail. A sentinel is the smaller -consumer-facing solution when that distinction matters. +### 2. How Much Engine-Policy Reporting Belongs In This Round? -### 2. Where Should Effective Engine Execution Policy Be Exposed? +The previous draft made a comprehensive execution-policy section a completion +requirement before assessing the cost of carrying transport and retry facts +through preparation. That commitment is now open, including the option to defer. -Resource inspection and `PreparedRun` do not currently provide a complete typed -view of the built-in transport cap and retry policy. These can originate in -Go engine options, not just the composed configuration inspected by ConfigKit. +**Recommendation:** Start with facts already held by the relevant owner and add +small fields to an existing prepared view where they are inexpensive to carry +accurately. Evaluate transport cap and effective retry settings individually. +Avoid a normalized policy model covering every client, invocation, and deadline. +If a useful fact requires substantial new plumbing, explicitly defer that fact +and document the report's boundary instead of inventing a default or guarantee. +The field set and location need agreement before implementation. -**Recommendation:** Extend the existing prepared-result view with a small, -separately named execution-policy section for facts applicable to that request. -Report the effective built-in retry setting and transport cap, distinguishing -injected-client ownership where Promptkit cannot supply those facts. Keep model -settings, backend capacity, and caller context constraints distinct. Reuse the -same policy selection as execution, including request overrides; do not add a -second preview resolver or claim to know an injected client's internal policy. -This makes the concrete preview useful without another top-level inspection API. - -**Alternative:** Add a separate engine-settings inspection API for captured -engine defaults, then let CommandKit combine that with resource inspection and -request-specific preparation. This also serves operators who have no concrete -request, but adds a public surface and requires a clear distinction between -engine defaults and request overrides. Select the surface before implementation; -neither option authorizes inspection of arbitrary client internals. +**Alternative:** Add a dedicated engine-settings view for captured defaults, +paired with prepared results for request-specific settings. This is useful to +operators without a concrete request, but adds an API and requires distinguishing +defaults from overrides. A third option is to defer engine-policy reporting +entirely for this round, delivering the resource/presence improvements first; +that has the lowest cost but leaves transport/retry reporting to later work. ### 3. Should Typed Prompt Construction Be Added In This Round? -**Recommendation:** Keep `PromptConfig` and `PromptSource` as a separate -discussion item until a programmatic prompt-authoring use case establishes -their value. File sources and ConfigKit mappings already support prompt -definitions. The absence of a helper is a Go convenience gap, not a missing -configuration capability, and does not block the backend additions. +**Recommendation:** Keep `PromptConfig`/`PromptSource` separate until a +programmatic authoring need justifies the public input model. Files and ConfigKit +mappings already support prompts; this does not block operator inspection. -**Alternative:** Add a typed prompt source alongside the existing backend and -profile helpers. It would improve discoverability, compiler assistance, and -consistency for applications constructing prompts in Go. Its cost is a larger -public input model covering messages, inline/file content, inputs, output -contracts, session templates, and omission semantics. If selected, define that -scope separately, including sibling inheritance and source context for relative -content files, while retaining ConfigKit's ownership of composition and raw -mapping support for explicit null. Do not implement a second prompt parser or -validation path merely to provide the helper. +**Alternative:** Add the typed helper for discoverability and compiler assistance. +It would need useful message/input/output and omission semantics, inheritance, +and source context for relative content files. Its maintenance cost is larger +than adding backend `Extends`; composition can still delegate to ConfigKit. -## Implementation Scope And Verification +### 4. How Should Presence And Public Result Values Be Represented? -1. Resolve the error and engine-policy decisions and define public result fields - and GoDoc. Typed prompt construction is a separate scope decision; it does - not block the selected inspection improvements. -2. Expand prompt inspection from captured metadata and message content. Add - presence information to profile and prepared results using the execution - path's existing resolved state. Preserve existing values and error behavior. -3. Connect namespace-aware backend inspection to captured lookup and add shared - diagnostic destination derivation. Keep the public facade responsible for - owned results; avoid a new backend cache, loader, or recursive inspector. -4. Implement the selected engine-policy reporting surface without repeating - policy resolution. Keep the concrete-request preview on existing preparation. -5. Add `BackendConfig.Extends` and encode it through `BackendSource`, leaving - composition with ConfigKit. -6. Update public GoDoc, consumer guidance, relevant internal documentation, and - release notes when implementation lands. Coordinate CommandKit changes for - the fuller reports and omission-aware formatting, without modifying its CLI - contract from this roadmap. +The operator requirement is accurate omission, not a particular Go structure, +new parallel report schema, or fixed JSON policy. -Use focused public tests for these consequential behaviors: +**Recommendation:** Reuse existing public nested values and expose the existing +`ExecutionTargetPresence` semantics alongside effective profile/prepared targets +where practical. This takes advantage of data already used by execution and +limits new translation. CommandKit can omit absent fields in its own output. +Document how values and presence interact rather than introducing an independent +presence/default/provenance system. Prefer additive changes when straightforward. -- Captured imported message content, description, session template, roles, - cache controls, and output contract appear without rendering, rereading files, - or requiring a resolvable default profile. -- Profile and prepared results identify omitted/suppressed controls as absent - and preserve explicit-zero and nonzero controls across inheritance and request - overrides. Details retain the same frozen settings and presence as their - prepared execution; no new distinction between reasons for omission is needed. -- Backend inspection handles root/scoped/qualified identities, bundled and - caller-defined records, capacity defaults, optional metadata, explicit and - derived diagnostic destinations, errors, and cancellation. Missing credential - environment values and missing smoke default profiles do not prevent lookup. -- Mutating returned templates, maps, diagnostic settings, or prepared details - cannot change captured state or another result. Inspection performs no - provider calls; preparation retains its established resource-I/O boundary. -- Engine-policy facts match the selected built-in or injected-client behavior - and applicable request overrides under the chosen reporting surface. -- Typed backend inheritance through real ConfigKit composition retains a parent - field and applies a child override. Reuse generic inheritance coverage rather - than retesting all of ConfigKit's composition rules. +**Alternative:** Use optional-valued fields in an inspection-specific settings +result, making absence directly expressible to consumers. That can simplify +reporting, but duplicates an execution-settings surface and needs conversion +and maintenance as fields evolve. Choose it only if the consumer simplification +outweighs that cost. Ordinary JSON tags or a deliberate serialization change can +also be considered; neither mandatory JSON stability for new results nor a ban +on useful serialization is selected. Existing documented JSON contracts still +need an explicit compatibility decision before changing. -Use controlled offline fixtures and reuse existing namespace, configuration, -ownership, and preparation coverage. CommandKit should own report-format tests -showing absent settings omitted and explicit zero preserved. Follow the -[testing policy](../policy/testing.md); avoid tests coupled to private projection -helpers or new recurring release gates. +### 5. Are Both Declared And Effective Diagnostic Endpoints Needed? -Completion means an operator can inspect captured prompt content, effective -profile settings with accurate omission semantics, and backend settings with -effective diagnostic destinations, then use ordinary preparation to preview a -concrete request. The chosen engine-policy surface supplies the additional facts -needed to describe execution policy without inventing provider or client defaults. -Typed backend inheritance is available through the source helper. Existing -composition and execution semantics remain intact. Typed prompt construction, -listing return-type changes, provider health guarantees, and live capacity -monitoring are not completion requirements. +Effective destinations are selected scope. The previous draft also suggested +keeping declared overrides separately, which could create two representations +and unnecessary origin metadata without a demonstrated reporting need. + +**Recommendation:** Report the effective destination, authentication kind, and +default profile reference. Use `InspectConfiguration` for authored values and +provenance. Reuse existing optional values where helpful; no duplicate declared +endpoint or derived-status flag is required merely for symmetry. + +**Alternative:** Include declared overrides alongside effective destinations when +operators need that distinction in one backend view. Existing metadata may make +this cheap, but the result then needs clear field meanings and consumer handling +for both. Prefer directly available fields over a new provenance subsystem. + +## Implementation And Verification + +1. Resolve open public-contract choices relevant to selected work. Agree on any + engine-policy additions separately so an unresolved optional expansion does + not silently become a prerequisite for the resource APIs. +2. Expand prompt inspection from captured data and expose existing presence in + profile/prepared results. Reuse or modestly extend current lookup/projection + paths; choose private interface layouts during implementation. +3. Add namespace-aware backend inspection and effective destination derivation, + following the selected public error and result representation decisions. +4. Add typed backend inheritance. Implement any separately selected engine-policy + reporting using the owning components' effective values. +5. Update GoDoc, consumer/internal guidance, and release notes with implemented + behavior and any deliberate compatibility changes. Coordinate CommandKit's + richer reports and omission handling without prescribing a new CLI framework. + +Select focused offline tests for actual behavior and boundary risks: + +- Newly exposed prompt fields reflect captured inline/imported content and + order, without introducing rendering or related-resource dependencies. +- Profile and prepared output preserve explicit zero and distinguish controls + that execution omits, including relevant inheritance and override cases. +- Backend results cover effective defaults/destinations, namespace selection, + and applicable error behavior. Use cases with missing optional credentials or + unresolved diagnostic profile references protect independent inspection. +- Returned mutable values cannot alter retained state. Reuse existing ownership + tests and copy helpers rather than prescribing extra snapshots or allocations. +- Typed backend inheritance works through real ConfigKit composition; its + generic composition rules do not need another full test matrix. +- Any selected policy-reporting fields reflect their owners' effective settings. + +CommandKit can test its own output omission and explicit-zero presentation. +Follow the [testing policy](../policy/testing.md); combine cases where useful, +reuse existing coverage, and do not turn this list into a test-per-field mandate. +No new benchmark threshold or release gate follows from this roadmap. + +Completion means the selected fields give operators a materially fuller and +accurate resource view, omission-aware settings are available to report builders, +and existing preparation remains usable for a concrete request. Backend source +inheritance is exposed. The resolved decisions define additional policy fields, +public representations, and compatibility changes; exhaustive execution-policy +coverage, typed prompt construction, listing changes, and speculative future +adapters are not implicit completion conditions.