Keep inspection roadmap contracts proportionate
This commit is contained in:
@@ -2,114 +2,132 @@
|
|||||||
|
|
||||||
## Purpose And Status
|
## Purpose And Status
|
||||||
|
|
||||||
Give operators using CommandKit-based applications a complete, accurate view of
|
Help operators using CommandKit-based applications understand Promptkit's
|
||||||
Promptkit's effective prompt, profile, and backend configuration. CommandKit is
|
captured prompt, profile, and backend configuration and the settings applicable
|
||||||
the primary consumer of these inspection APIs and owns operator-facing commands
|
to a concrete request. CommandKit owns operator-facing commands and presentation;
|
||||||
and formatting. Promptkit owns resolved typed facts and their semantics.
|
Promptkit supplies facts from its existing configuration and execution paths.
|
||||||
|
|
||||||
The selected direction is accepted for planning; implementation has not begun.
|
The direction is accepted for planning; implementation has not begun. Selected
|
||||||
Expand `InspectPrompt`, expose setting presence in profile and prepared results,
|
work includes expanded prompt inspection, presence-aware profile/prepared
|
||||||
and add `InspectBackend` with effective diagnostic destinations. Retain the
|
inspection, backend inspection with effective diagnostic destinations, and
|
||||||
planned `BackendConfig.Extends` convenience addition. Remaining decisions are
|
`BackendConfig.Extends`. Open decisions below cover unresolved scope and public
|
||||||
listed below; typed prompt construction and listing return-type changes are
|
representation choices. They are not implicit implementation requirements.
|
||||||
lower-priority considerations, not prerequisites for operator inspection.
|
|
||||||
|
|
||||||
This use case supersedes the earlier recommendation to keep prompt inspection
|
## Design Balance
|
||||||
limited to interface metadata and backend diagnostics limited to declared
|
|
||||||
settings. Operators need to see captured message content and the destinations
|
|
||||||
that default diagnostic rules select. The implementation should reuse existing
|
|
||||||
capture, resolution, and adapter logic rather than build a second resolver.
|
|
||||||
|
|
||||||
## What Fully Resolved Means
|
Provide useful information already captured or cheaply derived. Prefer exposing
|
||||||
|
existing values and small shared helpers over introducing new resolution paths,
|
||||||
|
parallel report models, provenance tracking, or retained snapshots solely for
|
||||||
|
inspection. This is a design preference, not a prohibition on a small adapter
|
||||||
|
or additional field when it serves a demonstrated consumer need.
|
||||||
|
|
||||||
| View | What can be resolved |
|
“Fully resolved” describes the applicable resolution stage. It is not a promise
|
||||||
|
that inspection can reconstruct the entire authored source, report every future
|
||||||
|
configuration field automatically, predict provider defaults, or reproduce a
|
||||||
|
future invocation independently of its request and external state.
|
||||||
|
|
||||||
|
| View | Information available |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| Resource inspection | Captured values after composition, inheritance, normalization, and applicable Promptkit defaults. |
|
| Resource inspection | Captured values after composition, inheritance, normalization, and applicable Promptkit defaults. |
|
||||||
| Concrete request preparation | Selected profile, request overrides, rendered messages, inputs, session ID, and loaded schema for supplied request data. |
|
| Concrete request preparation | Selected profile, request overrides, rendered messages, input metadata, session ID, and loaded schema for supplied request data. |
|
||||||
| Execution | Provider responses, actual usage, and other observed runtime facts. |
|
| Execution | Provider responses, actual usage, and other observed runtime facts. |
|
||||||
|
|
||||||
A prompt ID alone cannot determine future request overrides or template values.
|
The planned fields below define the initial additions. Additional cheap,
|
||||||
Resource inspection therefore shows effective templates and references, while
|
useful fields can be considered during implementation without building a
|
||||||
preparation shows the concrete request. Neither should invent provider defaults,
|
uniform resource framework or promising exhaustive future coverage. If a field
|
||||||
resolve a moving model alias to a hypothetical future model, or claim execution
|
requires substantial new machinery, record the tradeoff before expanding scope.
|
||||||
success from configuration alone.
|
|
||||||
|
|
||||||
**Reporting decision:** Omit unset optional settings from operator reports.
|
### Review Findings And Existing Boundaries
|
||||||
Retain explicitly configured values, including zero, and effective Promptkit
|
|
||||||
configuration defaults. Do not print omitted controls as zero or as
|
|
||||||
`unspecified`. Suppressed provider controls should likewise be absent from an
|
|
||||||
effective-settings report. This is a presentation of execution behavior;
|
|
||||||
`InspectConfiguration` remains available when an operator wants the authored
|
|
||||||
null, omission, or inheritance history.
|
|
||||||
|
|
||||||
Promptkit must expose sufficient typed presence information for CommandKit to
|
The current implementation provides a relatively small path to the core work:
|
||||||
make those choices correctly. A zero-valued Go field alone is not sufficient.
|
|
||||||
This does not remove fields from existing public types or silently change their
|
|
||||||
existing JSON representation; CommandKit owns its report projection.
|
|
||||||
|
|
||||||
## Relationship To Existing APIs
|
- Captured prompt definitions already retain description, session template,
|
||||||
|
message text, roles, and cache-control metadata alongside compiled templates.
|
||||||
|
Inspection can expose that data without reconstructing source files or ASTs.
|
||||||
|
- Internal prepared results already retain `TargetPresence`; the root API has
|
||||||
|
an `ExecutionTargetPresence` type. Presence-aware output need not introduce
|
||||||
|
a second interpretation of configuration or provider-value conversion.
|
||||||
|
- Smoke selection already has endpoint derivation, including the OpenRouter
|
||||||
|
key URL helper. Reusing or factoring that logic need not create an adapter
|
||||||
|
capability registry or provider metadata service.
|
||||||
|
- Transport timeout and retry policy live in different components and are not
|
||||||
|
all carried by prepared results. Their reporting scope and cost need a
|
||||||
|
decision; they are no longer an unconditional completion requirement.
|
||||||
|
|
||||||
| API | Responsibility |
|
These observations are implementation evidence, not promises to retain those
|
||||||
| --- | --- |
|
private types or helper arrangements. See the current
|
||||||
| `ListBackends` | Discover captured backend identities. |
|
[source/capture](../internal/sources.md), [runner](../internal/runner.md), and
|
||||||
| Proposed `InspectBackend` | Describe one backend's effective settings by ID. |
|
[model-client](../internal/llm.md) descriptions.
|
||||||
| Expanded `InspectPrompt` | Describe the captured prompt, including effective template content. |
|
|
||||||
| Enhanced `InspectProfile` | Report backend-completed settings with the presence information needed for accurate output. |
|
|
||||||
| `Prepare` / `PreparedExecution.Details` | Describe a concrete prepared request, including setting presence. |
|
|
||||||
| `InspectConfiguration` | Inspect effective configuration and source provenance. |
|
|
||||||
| `SmokeTest` | Execute an explicitly requested diagnostic and report evidence. |
|
|
||||||
|
|
||||||
See the [consumer guide](../consumers/pkg-promptkit.md#discover-available-resources-and-collections)
|
Retain established public ownership, credential handling, captured-state, and
|
||||||
for the implemented inspection and discovery boundaries, and the
|
namespace behavior unless a change is deliberately selected. Independent
|
||||||
[architecture policy](../policy/architecture.md) for library responsibilities.
|
mutable results prevent callers from changing engine state; they do not require
|
||||||
|
fresh copies of immutable strings or repeated deep copies between private layers.
|
||||||
|
Existing `Inspect*` APIs avoid provider work and request preparation. Extending
|
||||||
|
their returned data should not silently change those behaviors. These boundaries
|
||||||
|
follow the existing API and the selected operator use case, rather than a new
|
||||||
|
requirement that all future inspection APIs have identical behavior.
|
||||||
|
|
||||||
## Complete Prompt Inspection
|
## Reporting Semantics
|
||||||
|
|
||||||
|
**Selected presentation:** Omit unset optional settings from operator reports.
|
||||||
|
Preserve explicit values, including zero, and applicable Promptkit defaults.
|
||||||
|
Suppressed provider controls are also absent from an effective-settings report.
|
||||||
|
Neither zero nor `unspecified` should stand in for an omitted control.
|
||||||
|
|
||||||
|
This concerns the effective setting, not an explanation of every reason it was
|
||||||
|
omitted. ConfigKit inspection remains useful for authored values and history.
|
||||||
|
Promptkit supplies sufficient presence information; CommandKit chooses report
|
||||||
|
layout. Representation and serialization choices remain open in decision 4.
|
||||||
|
|
||||||
|
Do not infer provider defaults from absent controls. Likewise, a configured
|
||||||
|
model alias is the alias, not a prediction of the model it will select later.
|
||||||
|
Credential source names and requirements are useful inspection facts; resolved
|
||||||
|
credential values remain excluded under existing contracts. Templates and
|
||||||
|
arbitrary provider parameters are application data, not automatically safe
|
||||||
|
for disclosure. No new general-purpose sanitization guarantee is selected.
|
||||||
|
|
||||||
|
## Expanded Prompt Inspection
|
||||||
|
|
||||||
Expand `PromptInspection` to include description, ordered message templates
|
Expand `PromptInspection` to include description, ordered message templates
|
||||||
with roles and cache-control metadata, and the session-ID template. Retain
|
with roles and cache-control metadata, and the session-ID template. Retain the
|
||||||
identity, opaque prompt hash, default profile reference, declared inputs, and
|
existing identity, opaque hash, default profile reference, input declarations,
|
||||||
the complete output contract, including schema reference and repair settings.
|
and output contract, including schema reference and repair settings.
|
||||||
|
|
||||||
For messages imported from text or Markdown files, return the effective content
|
Expose effective content captured from imported text or Markdown files, as well
|
||||||
captured at engine construction. Operators should not have to locate or reopen
|
as inline content. Return the template text available in the captured definition;
|
||||||
the original file to see what execution will render. Return templates as text,
|
source-layout reconstruction, original YAML formatting, and parsed-template
|
||||||
not internal parsed-template objects. Inspection neither renders templates nor
|
objects are unnecessary for this use case. There is no new byte-for-byte source
|
||||||
rereads content files; edits require a new engine. The schema remains a
|
round-trip requirement. Message order and text should reflect what rendering
|
||||||
reference here; concrete preparation loads the selected schema.
|
consumes. Inspection continues to use captured state without rendering or
|
||||||
|
reopening files; the schema stays a reference until ordinary preparation loads it.
|
||||||
|
|
||||||
Keep prompt inspection independent of profile selection and request readiness.
|
Prompt inspection remains useful without a supplied request or a resolvable
|
||||||
An absent or incomplete default profile must not hide an otherwise captured
|
default profile. The existing independent lookup permits CommandKit to show a
|
||||||
prompt. A prompt without a default profile remains inspectable. Return owned
|
prompt even when related execution settings need attention.
|
||||||
mutable results, preserving message and input order and request isolation.
|
|
||||||
|
|
||||||
## Presence-Aware Profile And Prepared Inspection
|
## Presence-Aware Profile And Prepared Inspection
|
||||||
|
|
||||||
`ProfileInspection` currently exposes effective model settings but not the
|
`ProfileInspection` exposes effective settings without the presence information
|
||||||
`ExecutionTargetPresence` information that distinguishes omitted controls from
|
that execution uses for explicit-zero controls. `PreparedRun` has the same
|
||||||
explicit zero values. `PreparedRun` has the same reporting gap. Expose the
|
public reporting gap, including values returned by `PreparedExecution.Details`.
|
||||||
existing presence semantics alongside the effective target in both results,
|
Expose enough existing resolved presence information to distinguish omission
|
||||||
including the `PreparedRun` returned through `PreparedExecution.Details`.
|
from explicit settings in each of those results.
|
||||||
|
|
||||||
Use the resolved target and presence captured by the execution path; do not
|
For example, an unset temperature produces no temperature entry in the report;
|
||||||
infer presence from numeric values or re-run ConfigKit composition. Preserve
|
an explicit zero produces `temperature: 0`. A concrete request's override appears
|
||||||
backend completion, profile inheritance, explicit suppression, and request
|
in its prepared view. Backend completion, inheritance, and request precedence
|
||||||
precedence. Keep credential requirement metadata distinct from the credential
|
remain the execution path's responsibility. Inspectors need not reconstruct
|
||||||
source name; no credential value is returned or checked by resource inspection.
|
that history or distinguish all historical reasons for an absent setting.
|
||||||
|
|
||||||
For example, the report for a profile that leaves temperature unset should have
|
Reuse the captured target and presence semantics, including existing handling
|
||||||
no temperature entry. A profile explicitly setting temperature to zero should
|
of nonzero values. Decision 4 weighs exposing the current presence type against
|
||||||
show `temperature: 0`. A concrete request that overrides that profile should
|
another public representation; it does not reopen the omission-reporting decision.
|
||||||
show its effective setting instead. Presence metadata is for accurate report
|
See [defaults and overrides](../formats.md#defaults-and-overrides).
|
||||||
construction; CommandKit need not expose an implementation-oriented bit map to
|
|
||||||
operators.
|
|
||||||
|
|
||||||
Keep the meaning of effective defaults explicit. A configured zero generation
|
## Backend Inspection
|
||||||
timeout disables that particular deadline; it does not disable the transport
|
|
||||||
cap or caller cancellation. An omitted provider control supplies no assertion
|
|
||||||
about the provider's own default. See the
|
|
||||||
[defaults reference](../formats.md#defaults-and-overrides).
|
|
||||||
|
|
||||||
## Proposed Backend Inspection Contract
|
Proposed entry point:
|
||||||
|
|
||||||
```go
|
```go
|
||||||
func (e *Engine) InspectBackend(
|
func (e *Engine) InspectBackend(
|
||||||
@@ -118,286 +136,225 @@ func (e *Engine) InspectBackend(
|
|||||||
) (*BackendInspection, error)
|
) (*BackendInspection, error)
|
||||||
```
|
```
|
||||||
|
|
||||||
Use a dedicated public result rather than returning `BackendConfig`.
|
The result should expose the full backend identity, base endpoint, credential
|
||||||
`BackendConfig` is an input type whose pointers express omitted and explicit
|
environment-variable name, provider defaults, effective concurrency/queue
|
||||||
values; inspection describes the normalized effective definition after
|
settings, and diagnostic settings and destinations. These are backend defaults,
|
||||||
composition and defaulting. Do not expose internal domain types.
|
not settings after a particular profile or request overrides them. Capacity
|
||||||
|
values describe policy, not current occupancy.
|
||||||
|
|
||||||
The proposed result includes:
|
A dedicated top-level inspection result is recommended because `BackendConfig`
|
||||||
|
is a presence-aware source input. This does not require a new type for every
|
||||||
|
nested value: reuse suitable public values where their semantics match. Exact
|
||||||
|
field names and nesting can be selected during implementation within the public
|
||||||
|
representation decisions below; internal types remain behind the facade.
|
||||||
|
|
||||||
| Information | Meaning |
|
Follow existing namespace-view selection: qualify slashless caller IDs with the
|
||||||
| --- | --- |
|
backend scope, pass qualified IDs through, and report the selected full ID.
|
||||||
| `BackendID` | Full selected runtime identity. |
|
Use the same identity rules for bundled and caller-supplied backends. The new
|
||||||
| `Endpoint` | Captured backend base endpoint. |
|
operation should follow existing inspection conventions for invalid engines,
|
||||||
| `APIKeyEnv` | Configured environment-variable name, never its resolved value. |
|
invalid selections, and context cancellation, without adding cross-error
|
||||||
| `ExtraParams` | Backend provider defaults, before profile or request overrides. |
|
precedence guarantees. Missing-backend classification is open decision 1.
|
||||||
| `ConcurrencyLimit` | Configured generation concurrency limit. |
|
|
||||||
| `QueueCapacity` | Effective queue capacity, including construction-time defaults. |
|
|
||||||
| `SmokeTests` | Diagnostic settings and effective destinations, including the authentication kind and optional inference default profile reference. |
|
|
||||||
|
|
||||||
Scalar fields describe effective values rather than reproducing source-field
|
|
||||||
presence. Diagnostic optionality must remain expressible, including no configured
|
|
||||||
authentication adapter or inference default. Exact nested type names are
|
|
||||||
implementation choices.
|
|
||||||
Follow existing typed inspection conventions without adding a stable JSON
|
|
||||||
serialization contract for this feature. Applications can define their own
|
|
||||||
output formats.
|
|
||||||
|
|
||||||
### Selection And Errors
|
|
||||||
|
|
||||||
Follow the existing namespace selection rules: a scoped engine view qualifies
|
|
||||||
slashless IDs with its backend namespace; qualified IDs pass through unchanged.
|
|
||||||
An unscoped view looks up the supplied ID directly. Return the full selected ID.
|
|
||||||
Do not add fallback search across namespaces, implicit backend selection, or
|
|
||||||
special handling for bundled backends. Use the existing ID rules, including
|
|
||||||
case sensitivity, rather than introducing new normalization.
|
|
||||||
|
|
||||||
A nil or zero-value engine should match `ErrInvalidConfig`; a blank or otherwise
|
|
||||||
invalid selection should match `ErrInvalidRequest`. Cancellation should retain
|
|
||||||
the context error, and failures should return no partial result. The public
|
|
||||||
classification for a missing backend is open decision 1.
|
|
||||||
|
|
||||||
### Effective Diagnostic Destinations
|
### Effective Diagnostic Destinations
|
||||||
|
|
||||||
Include the effective reachability destination and, when an authentication
|
Include effective reachability and applicable authentication destinations,
|
||||||
adapter applies, its effective destination. Apply explicit endpoint overrides
|
using explicit overrides or existing backend-relative defaults. Share the
|
||||||
and existing backend-relative defaults through the same derivation logic used
|
existing derivation logic where practical; neither CommandKit nor a new
|
||||||
by diagnostic execution. CommandKit must not reconstruct provider-specific URL
|
inspection-specific resolver should maintain a competing set of provider rules.
|
||||||
rules. Include the configured authentication kind and inference default profile
|
A small helper extraction is sufficient if that is all the current adapters need.
|
||||||
reference; an inapplicable authentication kind has no active probe destination.
|
|
||||||
|
|
||||||
Default reachability can apply even without a `smoke_tests` block. The result
|
Default reachability applies even without authored `smoke_tests` metadata.
|
||||||
must be able to report that effective destination without fabricating an authored
|
Report the configured authentication kind and optional inference-default profile
|
||||||
block or an authentication configuration. Where distinguishing an explicit
|
reference without fabricating authored settings. An authentication kind of
|
||||||
endpoint override from a derived destination is useful, keep those facts
|
`none` has no active authentication destination. Decision 5 covers whether the
|
||||||
separate rather than overloading a field's meaning. Source history remains
|
result also needs separate declared endpoint fields.
|
||||||
owned by `InspectConfiguration`.
|
|
||||||
|
|
||||||
These are destinations under the backend's captured settings, not guarantees
|
Destinations describe captured backend settings, not provider reachability or
|
||||||
about runtime overrides, the selected client's HTTP diagnostic support, or
|
HTTP diagnostic support in an injected client. Retain the inference-default
|
||||||
reachability. Report a default inference profile reference without resolving it;
|
profile as a reference; backend inspection need not resolve it or establish
|
||||||
a missing or incomplete referenced profile must not make backend inspection fail.
|
inference readiness. Ordinary inspection is expected to return a result or an
|
||||||
|
error, following existing methods. If derivation reveals a meaningful case where
|
||||||
|
useful captured settings would be lost, bring that case back for a decision
|
||||||
|
rather than automatically adding partial-result and warning machinery.
|
||||||
|
|
||||||
### Captured State And Ownership
|
## Connected Views And Concrete Requests
|
||||||
|
|
||||||
Inspect the engine's captured backend definition without reloading sources,
|
CommandKit can follow prompt default-profile and selected-backend references
|
||||||
reading credential values, preparing a prompt, acquiring capacity, or making
|
through the existing inspections to build a connected view. This is a suggested
|
||||||
provider calls. Engine construction already validates effective definitions;
|
consumer workflow, not a required report format or a restriction on future
|
||||||
inspection should not repeat that work or create an additional acceptance path.
|
convenience APIs. Use stored references exactly; namespace defaults apply to
|
||||||
|
caller selections, not to reinterpreting captured references. Endpoint-only
|
||||||
|
profiles need no backend node. A consumer can retain successful lookups while
|
||||||
|
showing a failure for a related resource.
|
||||||
|
|
||||||
Capacity fields describe configuration, not current occupancy or queue length.
|
Use existing `Prepare` for concrete variables, inputs, profile selection, and
|
||||||
Diagnostic settings describe configuration, not provider availability, key
|
request overrides. It performs artifact/schema reads, rendering, and applicable
|
||||||
acceptance, or inference readiness. In particular, report a smoke-test default
|
credential checks without inference. `PrepareExecution` and its details serve
|
||||||
profile reference without resolving or validating that profile merely to inspect
|
callers that want later execution tied to the same preparation, subject to that
|
||||||
the backend. Its selection-time behavior remains owned by `SmokeTest`.
|
API's existing contract. This roadmap adds no stronger replay guarantee and
|
||||||
|
selects no second preparation workflow.
|
||||||
|
|
||||||
Return independently owned mutable values, including nested provider maps and
|
Model parameters are not all execution policy. Per-generation timeout, transport
|
||||||
diagnostic settings. Mutating a result must not affect the engine or another
|
cap, retry settings, backend capacity, and caller deadlines are different facts.
|
||||||
inspection. Reuse existing ownership operations and avoid consecutive copies
|
When reported, keep their meanings distinct. Decision 2 determines whether and
|
||||||
whose only purpose is to cross private helper boundaries.
|
how this round should expose additional engine-policy information. Report known
|
||||||
|
settings without claiming knowledge of an injected client's internal behavior.
|
||||||
|
|
||||||
Provider defaults should retain their values, consistent with typed target
|
CommandKit owns CLI commands, formatting, and disclosure. Coordinate downstream
|
||||||
inspection. They are application data and may contain sensitive content;
|
adoption when the Promptkit additions land; this roadmap does not prescribe
|
||||||
the result is not automatically safe for logging or disclosure. Do not resolve
|
CommandKit's internal adapter design or require a new stable CLI serialization
|
||||||
credentials or silently substitute ConfigKit's redacted reporting values for
|
contract in Promptkit.
|
||||||
the typed settings. Operator provenance and redacted configuration reporting
|
|
||||||
remain available through `InspectConfiguration`.
|
|
||||||
|
|
||||||
## Connected Reports And Concrete Request Preview
|
|
||||||
|
|
||||||
CommandKit can assemble a connected report by following prompt default-profile
|
|
||||||
and selected-backend identities through the three inspection APIs. Promptkit
|
|
||||||
supplies the references and effective values; CommandKit chooses nested sections
|
|
||||||
or links. Use captured references exactly, without requalifying them through a
|
|
||||||
view's caller-selection namespace. Endpoint-only profiles need no backend node.
|
|
||||||
A missing related resource can be reported while preserving successfully
|
|
||||||
inspected resources; do not make each resource inspection recursively dependent
|
|
||||||
on all related objects.
|
|
||||||
|
|
||||||
Use `Prepare` for an operator preview with concrete variables, inputs, profile
|
|
||||||
selection, and request overrides. It performs artifact/schema reads, rendering,
|
|
||||||
and applicable credential checks without inference. Do not describe it as a
|
|
||||||
pure configuration lookup. Use `PrepareExecution` and its existing details when
|
|
||||||
later execution must use the same captured preparation; a separate preview
|
|
||||||
followed by an independent run is not a promise that mutable inputs stay equal.
|
|
||||||
No second request-resolution API is selected by this roadmap.
|
|
||||||
|
|
||||||
An execution report must distinguish per-generation timeout, built-in transport
|
|
||||||
cap, retry policy, and backend capacity. Effective model parameters alone are
|
|
||||||
not the entire execution policy. Caller deadlines and injected-client behavior
|
|
||||||
can add constraints that Promptkit cannot predict for a future invocation.
|
|
||||||
How to expose the missing engine-policy facts is open decision 2. Do not infer
|
|
||||||
them from model fields or ask CommandKit to duplicate Promptkit defaults.
|
|
||||||
|
|
||||||
CommandKit owns text/YAML presentation and report disclosure. Prompt templates,
|
|
||||||
input content, and arbitrary provider parameters may be sensitive; preserve
|
|
||||||
existing credential exclusions without claiming that every inspected value is
|
|
||||||
safe to publish. No CommandKit implementation is included in this repository's
|
|
||||||
roadmap; coordinate its consumer adaptation when the Promptkit APIs land.
|
|
||||||
|
|
||||||
## Typed Backend Inheritance
|
## Typed Backend Inheritance
|
||||||
|
|
||||||
Add `Extends *string` to `BackendConfig`, matching `ProfileConfig.Extends`.
|
Add `Extends *string` to `BackendConfig`, matching `ProfileConfig.Extends`.
|
||||||
When non-nil, `BackendSource` should encode it as `extends: {id: value}` in the
|
`BackendSource` should encode a non-nil value as `extends: {id: value}` and omit
|
||||||
backend record. Nil omits the field. Preserve the supplied value and let
|
the field when nil. ConfigKit owns inheritance resolution and its diagnostics.
|
||||||
ConfigKit resolve and validate inheritance through the existing source-loading
|
The helper exposes existing sibling inheritance, rather than performing parent
|
||||||
path, just as it does for profiles.
|
lookup or filling inherited fields itself.
|
||||||
|
|
||||||
The purpose is to let programmatic callers derive a backend variant without
|
Keep existing root/collection identity, replacement, and null semantics. Raw
|
||||||
switching from `BackendSource` to a raw mapping. The helper must not resolve a
|
mappings remain available for explicit null and other generic controls outside
|
||||||
parent, fill inherited fields, or become a second inheritance implementation.
|
the helper's convenience surface. See the
|
||||||
It remains a source constructor rather than an engine registry mutation API.
|
[inheritance reference](../formats.md#resource-inheritance) and
|
||||||
|
[profile helper](../../profiles.go).
|
||||||
|
|
||||||
This exposes existing sibling inheritance, not a new configuration feature.
|
## Related Work And Scope
|
||||||
Parent IDs select siblings in the same root or collection resource list;
|
|
||||||
namespace views do not qualify these configuration references. Existing
|
|
||||||
same-ID replacement, omitted-field, and explicit-null behavior remain owned by
|
|
||||||
ConfigKit composition. Raw mappings remain available when callers need explicit
|
|
||||||
null or other generic configuration controls that the typed helper cannot express.
|
|
||||||
See the [inheritance reference](../formats.md#resource-inheritance) and the
|
|
||||||
existing [profile source helper](../../profiles.go).
|
|
||||||
|
|
||||||
## Scope Of API Consistency
|
Typed prompt construction is open decision 3. Listing return types remain a
|
||||||
|
possible later cleanup: `ListPrompts` returns an ID-only `PromptIdentity` wrapper,
|
||||||
|
while backend/profile listings return strings. Standardizing on strings may be
|
||||||
|
useful at a breaking-change window, but is not required for these inspections.
|
||||||
|
|
||||||
Backends, profiles, and prompts already share source layering, collection
|
Collections, schemas, and engine options have different roles; symmetry does
|
||||||
membership, namespace selection, sibling inheritance, and construction-time
|
not require identical APIs for them. The
|
||||||
definition validation. The gaps addressed here are in public convenience and
|
|
||||||
inspection APIs, not in those underlying configuration capabilities.
|
|
||||||
|
|
||||||
Inspection remains purpose-specific while becoming complete for operator use:
|
|
||||||
prompt templates are captured but unrendered, profile settings include backend
|
|
||||||
completion, and backend settings include effective diagnostic destinations.
|
|
||||||
Provenance and authored representation stay with `InspectConfiguration`.
|
|
||||||
|
|
||||||
`ListPrompts` currently returns `[]PromptIdentity`, whose only field is `ID`;
|
|
||||||
`ListProfiles` and `ListBackends` return `[]string`. At a suitable breaking-change
|
|
||||||
window, prefer considering `[]string` for all three over adding thin identity
|
|
||||||
wrappers for backends and profiles. That would simplify consumer code, but
|
|
||||||
requires a deliberate compatibility decision and migration instructions. Keep
|
|
||||||
the existing return types in this work; listing consistency does not block the
|
|
||||||
backend additions.
|
|
||||||
|
|
||||||
Collections, schemas, and engine configuration remain distinct concepts rather
|
|
||||||
than additional peers requiring identical resource APIs. Prompt requirement
|
|
||||||
inspection and backend capacity/diagnostic settings retain their specialized
|
|
||||||
roles. Profile/backend hashes are not added for symmetry; the existing
|
|
||||||
[deferred fingerprint proposal](deferred.md#semantic-execution-target-fingerprints)
|
[deferred fingerprint proposal](deferred.md#semantic-execution-target-fingerprints)
|
||||||
owns that separate question.
|
remains separate. Live capacity monitoring, provider health guarantees, and
|
||||||
|
source round-tripping are not selected deliverables, rather than permanent
|
||||||
|
prohibitions on future work.
|
||||||
|
|
||||||
## Open Decisions
|
## Open Decisions
|
||||||
|
|
||||||
### 1. How Should An Unknown Backend Be Classified?
|
### 1. How Should An Unknown Backend Be Classified?
|
||||||
|
|
||||||
**Recommendation:** Add public `ErrBackendNotFound` for this explicit lookup,
|
**Recommendation:** Add public `ErrBackendNotFound`, analogous to the existing
|
||||||
matching the role of `ErrProfileNotFound` and `ErrPromptNotFound`. The registry
|
prompt/profile sentinels. The registry already identifies missing backends;
|
||||||
already has an internal backend-not-found sentinel. Map it at the facade so
|
public mapping lets consumers distinguish a missing ID without parsing text.
|
||||||
applications can distinguish a missing backend from malformed input without
|
Keep existing operations' classifications unless a change is explicitly chosen.
|
||||||
inspecting text or internal packages.
|
|
||||||
|
|
||||||
Limit this addition to the new inspection operation unless existing callers
|
**Alternative:** Use `ErrInvalidRequest`, avoiding another exported sentinel but
|
||||||
justify a broader change; do not automatically reclassify smoke-test or engine
|
combining malformed selection and a well-formed missing identity. Consumers that
|
||||||
construction errors.
|
need the distinction would require inventory lookup or other error detail.
|
||||||
|
|
||||||
**Alternative:** Classify an absent ID as `ErrInvalidRequest`. This avoids
|
### 2. How Much Engine-Policy Reporting Belongs In This Round?
|
||||||
another exported sentinel but combines a well-formed missing identity with an
|
|
||||||
invalid selection. Consumers needing different presentation would then require
|
|
||||||
an inventory lookup or additional error detail. A sentinel is the smaller
|
|
||||||
consumer-facing solution when that distinction matters.
|
|
||||||
|
|
||||||
### 2. Where Should Effective Engine Execution Policy Be Exposed?
|
The previous draft made a comprehensive execution-policy section a completion
|
||||||
|
requirement before assessing the cost of carrying transport and retry facts
|
||||||
|
through preparation. That commitment is now open, including the option to defer.
|
||||||
|
|
||||||
Resource inspection and `PreparedRun` do not currently provide a complete typed
|
**Recommendation:** Start with facts already held by the relevant owner and add
|
||||||
view of the built-in transport cap and retry policy. These can originate in
|
small fields to an existing prepared view where they are inexpensive to carry
|
||||||
Go engine options, not just the composed configuration inspected by ConfigKit.
|
accurately. Evaluate transport cap and effective retry settings individually.
|
||||||
|
Avoid a normalized policy model covering every client, invocation, and deadline.
|
||||||
|
If a useful fact requires substantial new plumbing, explicitly defer that fact
|
||||||
|
and document the report's boundary instead of inventing a default or guarantee.
|
||||||
|
The field set and location need agreement before implementation.
|
||||||
|
|
||||||
**Recommendation:** Extend the existing prepared-result view with a small,
|
**Alternative:** Add a dedicated engine-settings view for captured defaults,
|
||||||
separately named execution-policy section for facts applicable to that request.
|
paired with prepared results for request-specific settings. This is useful to
|
||||||
Report the effective built-in retry setting and transport cap, distinguishing
|
operators without a concrete request, but adds an API and requires distinguishing
|
||||||
injected-client ownership where Promptkit cannot supply those facts. Keep model
|
defaults from overrides. A third option is to defer engine-policy reporting
|
||||||
settings, backend capacity, and caller context constraints distinct. Reuse the
|
entirely for this round, delivering the resource/presence improvements first;
|
||||||
same policy selection as execution, including request overrides; do not add a
|
that has the lowest cost but leaves transport/retry reporting to later work.
|
||||||
second preview resolver or claim to know an injected client's internal policy.
|
|
||||||
This makes the concrete preview useful without another top-level inspection API.
|
|
||||||
|
|
||||||
**Alternative:** Add a separate engine-settings inspection API for captured
|
|
||||||
engine defaults, then let CommandKit combine that with resource inspection and
|
|
||||||
request-specific preparation. This also serves operators who have no concrete
|
|
||||||
request, but adds a public surface and requires a clear distinction between
|
|
||||||
engine defaults and request overrides. Select the surface before implementation;
|
|
||||||
neither option authorizes inspection of arbitrary client internals.
|
|
||||||
|
|
||||||
### 3. Should Typed Prompt Construction Be Added In This Round?
|
### 3. Should Typed Prompt Construction Be Added In This Round?
|
||||||
|
|
||||||
**Recommendation:** Keep `PromptConfig` and `PromptSource` as a separate
|
**Recommendation:** Keep `PromptConfig`/`PromptSource` separate until a
|
||||||
discussion item until a programmatic prompt-authoring use case establishes
|
programmatic authoring need justifies the public input model. Files and ConfigKit
|
||||||
their value. File sources and ConfigKit mappings already support prompt
|
mappings already support prompts; this does not block operator inspection.
|
||||||
definitions. The absence of a helper is a Go convenience gap, not a missing
|
|
||||||
configuration capability, and does not block the backend additions.
|
|
||||||
|
|
||||||
**Alternative:** Add a typed prompt source alongside the existing backend and
|
**Alternative:** Add the typed helper for discoverability and compiler assistance.
|
||||||
profile helpers. It would improve discoverability, compiler assistance, and
|
It would need useful message/input/output and omission semantics, inheritance,
|
||||||
consistency for applications constructing prompts in Go. Its cost is a larger
|
and source context for relative content files. Its maintenance cost is larger
|
||||||
public input model covering messages, inline/file content, inputs, output
|
than adding backend `Extends`; composition can still delegate to ConfigKit.
|
||||||
contracts, session templates, and omission semantics. If selected, define that
|
|
||||||
scope separately, including sibling inheritance and source context for relative
|
|
||||||
content files, while retaining ConfigKit's ownership of composition and raw
|
|
||||||
mapping support for explicit null. Do not implement a second prompt parser or
|
|
||||||
validation path merely to provide the helper.
|
|
||||||
|
|
||||||
## Implementation Scope And Verification
|
### 4. How Should Presence And Public Result Values Be Represented?
|
||||||
|
|
||||||
1. Resolve the error and engine-policy decisions and define public result fields
|
The operator requirement is accurate omission, not a particular Go structure,
|
||||||
and GoDoc. Typed prompt construction is a separate scope decision; it does
|
new parallel report schema, or fixed JSON policy.
|
||||||
not block the selected inspection improvements.
|
|
||||||
2. Expand prompt inspection from captured metadata and message content. Add
|
|
||||||
presence information to profile and prepared results using the execution
|
|
||||||
path's existing resolved state. Preserve existing values and error behavior.
|
|
||||||
3. Connect namespace-aware backend inspection to captured lookup and add shared
|
|
||||||
diagnostic destination derivation. Keep the public facade responsible for
|
|
||||||
owned results; avoid a new backend cache, loader, or recursive inspector.
|
|
||||||
4. Implement the selected engine-policy reporting surface without repeating
|
|
||||||
policy resolution. Keep the concrete-request preview on existing preparation.
|
|
||||||
5. Add `BackendConfig.Extends` and encode it through `BackendSource`, leaving
|
|
||||||
composition with ConfigKit.
|
|
||||||
6. Update public GoDoc, consumer guidance, relevant internal documentation, and
|
|
||||||
release notes when implementation lands. Coordinate CommandKit changes for
|
|
||||||
the fuller reports and omission-aware formatting, without modifying its CLI
|
|
||||||
contract from this roadmap.
|
|
||||||
|
|
||||||
Use focused public tests for these consequential behaviors:
|
**Recommendation:** Reuse existing public nested values and expose the existing
|
||||||
|
`ExecutionTargetPresence` semantics alongside effective profile/prepared targets
|
||||||
|
where practical. This takes advantage of data already used by execution and
|
||||||
|
limits new translation. CommandKit can omit absent fields in its own output.
|
||||||
|
Document how values and presence interact rather than introducing an independent
|
||||||
|
presence/default/provenance system. Prefer additive changes when straightforward.
|
||||||
|
|
||||||
- Captured imported message content, description, session template, roles,
|
**Alternative:** Use optional-valued fields in an inspection-specific settings
|
||||||
cache controls, and output contract appear without rendering, rereading files,
|
result, making absence directly expressible to consumers. That can simplify
|
||||||
or requiring a resolvable default profile.
|
reporting, but duplicates an execution-settings surface and needs conversion
|
||||||
- Profile and prepared results identify omitted/suppressed controls as absent
|
and maintenance as fields evolve. Choose it only if the consumer simplification
|
||||||
and preserve explicit-zero and nonzero controls across inheritance and request
|
outweighs that cost. Ordinary JSON tags or a deliberate serialization change can
|
||||||
overrides. Details retain the same frozen settings and presence as their
|
also be considered; neither mandatory JSON stability for new results nor a ban
|
||||||
prepared execution; no new distinction between reasons for omission is needed.
|
on useful serialization is selected. Existing documented JSON contracts still
|
||||||
- Backend inspection handles root/scoped/qualified identities, bundled and
|
need an explicit compatibility decision before changing.
|
||||||
caller-defined records, capacity defaults, optional metadata, explicit and
|
|
||||||
derived diagnostic destinations, errors, and cancellation. Missing credential
|
|
||||||
environment values and missing smoke default profiles do not prevent lookup.
|
|
||||||
- Mutating returned templates, maps, diagnostic settings, or prepared details
|
|
||||||
cannot change captured state or another result. Inspection performs no
|
|
||||||
provider calls; preparation retains its established resource-I/O boundary.
|
|
||||||
- Engine-policy facts match the selected built-in or injected-client behavior
|
|
||||||
and applicable request overrides under the chosen reporting surface.
|
|
||||||
- Typed backend inheritance through real ConfigKit composition retains a parent
|
|
||||||
field and applies a child override. Reuse generic inheritance coverage rather
|
|
||||||
than retesting all of ConfigKit's composition rules.
|
|
||||||
|
|
||||||
Use controlled offline fixtures and reuse existing namespace, configuration,
|
### 5. Are Both Declared And Effective Diagnostic Endpoints Needed?
|
||||||
ownership, and preparation coverage. CommandKit should own report-format tests
|
|
||||||
showing absent settings omitted and explicit zero preserved. Follow the
|
|
||||||
[testing policy](../policy/testing.md); avoid tests coupled to private projection
|
|
||||||
helpers or new recurring release gates.
|
|
||||||
|
|
||||||
Completion means an operator can inspect captured prompt content, effective
|
Effective destinations are selected scope. The previous draft also suggested
|
||||||
profile settings with accurate omission semantics, and backend settings with
|
keeping declared overrides separately, which could create two representations
|
||||||
effective diagnostic destinations, then use ordinary preparation to preview a
|
and unnecessary origin metadata without a demonstrated reporting need.
|
||||||
concrete request. The chosen engine-policy surface supplies the additional facts
|
|
||||||
needed to describe execution policy without inventing provider or client defaults.
|
**Recommendation:** Report the effective destination, authentication kind, and
|
||||||
Typed backend inheritance is available through the source helper. Existing
|
default profile reference. Use `InspectConfiguration` for authored values and
|
||||||
composition and execution semantics remain intact. Typed prompt construction,
|
provenance. Reuse existing optional values where helpful; no duplicate declared
|
||||||
listing return-type changes, provider health guarantees, and live capacity
|
endpoint or derived-status flag is required merely for symmetry.
|
||||||
monitoring are not completion requirements.
|
|
||||||
|
**Alternative:** Include declared overrides alongside effective destinations when
|
||||||
|
operators need that distinction in one backend view. Existing metadata may make
|
||||||
|
this cheap, but the result then needs clear field meanings and consumer handling
|
||||||
|
for both. Prefer directly available fields over a new provenance subsystem.
|
||||||
|
|
||||||
|
## Implementation And Verification
|
||||||
|
|
||||||
|
1. Resolve open public-contract choices relevant to selected work. Agree on any
|
||||||
|
engine-policy additions separately so an unresolved optional expansion does
|
||||||
|
not silently become a prerequisite for the resource APIs.
|
||||||
|
2. Expand prompt inspection from captured data and expose existing presence in
|
||||||
|
profile/prepared results. Reuse or modestly extend current lookup/projection
|
||||||
|
paths; choose private interface layouts during implementation.
|
||||||
|
3. Add namespace-aware backend inspection and effective destination derivation,
|
||||||
|
following the selected public error and result representation decisions.
|
||||||
|
4. Add typed backend inheritance. Implement any separately selected engine-policy
|
||||||
|
reporting using the owning components' effective values.
|
||||||
|
5. Update GoDoc, consumer/internal guidance, and release notes with implemented
|
||||||
|
behavior and any deliberate compatibility changes. Coordinate CommandKit's
|
||||||
|
richer reports and omission handling without prescribing a new CLI framework.
|
||||||
|
|
||||||
|
Select focused offline tests for actual behavior and boundary risks:
|
||||||
|
|
||||||
|
- Newly exposed prompt fields reflect captured inline/imported content and
|
||||||
|
order, without introducing rendering or related-resource dependencies.
|
||||||
|
- Profile and prepared output preserve explicit zero and distinguish controls
|
||||||
|
that execution omits, including relevant inheritance and override cases.
|
||||||
|
- Backend results cover effective defaults/destinations, namespace selection,
|
||||||
|
and applicable error behavior. Use cases with missing optional credentials or
|
||||||
|
unresolved diagnostic profile references protect independent inspection.
|
||||||
|
- Returned mutable values cannot alter retained state. Reuse existing ownership
|
||||||
|
tests and copy helpers rather than prescribing extra snapshots or allocations.
|
||||||
|
- Typed backend inheritance works through real ConfigKit composition; its
|
||||||
|
generic composition rules do not need another full test matrix.
|
||||||
|
- Any selected policy-reporting fields reflect their owners' effective settings.
|
||||||
|
|
||||||
|
CommandKit can test its own output omission and explicit-zero presentation.
|
||||||
|
Follow the [testing policy](../policy/testing.md); combine cases where useful,
|
||||||
|
reuse existing coverage, and do not turn this list into a test-per-field mandate.
|
||||||
|
No new benchmark threshold or release gate follows from this roadmap.
|
||||||
|
|
||||||
|
Completion means the selected fields give operators a materially fuller and
|
||||||
|
accurate resource view, omission-aware settings are available to report builders,
|
||||||
|
and existing preparation remains usable for a concrete request. Backend source
|
||||||
|
inheritance is exposed. The resolved decisions define additional policy fields,
|
||||||
|
public representations, and compatibility changes; exhaustive execution-policy
|
||||||
|
coverage, typed prompt construction, listing changes, and speculative future
|
||||||
|
adapters are not implicit completion conditions.
|
||||||
|
|||||||
Reference in New Issue
Block a user