2026-11-21 16:48:34 +00:00
|
|
|
|
# Configuration Handoff And Repair Review
|
|
|
|
|
|
|
|
|
|
|
|
Reviewed implementation stages 1–17 against
|
|
|
|
|
|
[the implementation plan](implementation.md),
|
|
|
|
|
|
[the feature roadmap](next-release-considerations.md), and accepted ADRs
|
|
|
|
|
|
[0001](../adr/0001-configuration-boundary.md) and
|
|
|
|
|
|
[0002](../adr/0002-shared-configuration-data-model.md).
|
|
|
|
|
|
Implementation baseline: `40f8a6825359b4923645256e230a050dfdb00710`;
|
|
|
|
|
|
reviewed changes begin after `248d0db`.
|
|
|
|
|
|
|
|
|
|
|
|
## Findings
|
|
|
|
|
|
|
|
|
|
|
|
### 1. Narrow The Pre-Handoff Mutation Wording To The Accepted Expectation
|
|
|
|
|
|
|
|
|
|
|
|
Status: open. Priority: P3. Implementation stage: 16.
|
|
|
|
|
|
|
|
|
|
|
|
Locations: `engine.go:280` and `docs/consumers/pkg-promptkit.md:132`.
|
|
|
|
|
|
|
|
|
|
|
|
`NewFromConfig` says callers **must not mutate** selected settings between loading
|
|
|
|
|
|
and handoff; the consumer guide repeats that blanket instruction. The accepted
|
|
|
|
|
|
roadmap instead asks callers to hand off before changes that invalidate the
|
|
|
|
|
|
frozen inspection/provenance evidence, and expressly treats this as a usage
|
|
|
|
|
|
expectation rather than a new enforcement mechanism. The published wording
|
|
|
|
|
|
prohibits even value-preserving reassignment or copying, which needs no such
|
|
|
|
|
|
restriction.
|
|
|
|
|
|
|
|
|
|
|
|
Required change: describe the narrower consistency expectation in both owners.
|
|
|
|
|
|
Explain that construction consumes the resolved settings while inspection retains
|
|
|
|
|
|
the completed load's frozen evidence, so callers should keep those consistent at
|
|
|
|
|
|
handoff. Preserve the existing protection against mutation after construction.
|
|
|
|
|
|
No mutation detection, extra validation pass, or new error is needed.
|
|
|
|
|
|
|
|
|
|
|
|
Verification: review both passages against the feature roadmap's ownership
|
|
|
|
|
|
paragraph; ensure they no longer impose an unconditional mutation prohibition.
|
|
|
|
|
|
This is a documentation correction and does not require a new runtime test.
|
|
|
|
|
|
|
|
|
|
|
|
### 2. Keep Application-Field Sensitivity Policy With The Application
|
|
|
|
|
|
|
|
|
|
|
|
Status: open. Priority: P3. Implementation stage: 16.
|
|
|
|
|
|
|
|
|
|
|
|
Location: `docs/formats.md:235–236`.
|
|
|
|
|
|
|
|
|
|
|
|
The backend format reference says an unrecognized backend field is application
|
|
|
|
|
|
data and **must not contain secrets**. The accepted extension-point contract
|
|
|
|
|
|
assigns interpretation, validation, and sensitivity rules for those fields to
|
|
|
|
|
|
the application. The implementation follows that contract: inline fields remain
|
|
|
|
|
|
in native inspection, application sensitivity predicates compose with PromptKit's,
|
|
|
|
|
|
and those fields are omitted from resource reports and provider parameters.
|
|
|
|
|
|
The added prohibition unnecessarily narrows the application-owned data contract.
|
|
|
|
|
|
|
|
|
|
|
|
Required change: retain the useful warning that a raw `api_key` extension is not
|
|
|
|
|
|
a PromptKit credential input and is not automatically redacted by PromptKit.
|
|
|
|
|
|
State that the application owns sensitivity declarations and disclosure policy
|
|
|
|
|
|
for its extension fields, rather than prohibiting their contents. Review the
|
|
|
|
|
|
adjacent profile wording for the same distinction between PromptKit credentials
|
|
|
|
|
|
and application-owned data. Do not add automatic credential behavior or a new
|
|
|
|
|
|
validator for application fields.
|
|
|
|
|
|
|
|
|
|
|
|
Verification: compare the revised text with the feature roadmap's
|
|
|
|
|
|
application-specific-fields contract and existing focused-inspection sensitivity
|
|
|
|
|
|
coverage. No change to execution or redaction implementation is needed.
|
|
|
|
|
|
|
2026-11-21 17:27:06 +00:00
|
|
|
|
### 3. Preserve Effective Input Metadata Supplied By Application Defaults
|
|
|
|
|
|
|
|
|
|
|
|
Status: open. Priority: P2. Implementation stages: 5–6 and 13.
|
|
|
|
|
|
|
|
|
|
|
|
Locations: `internal/configuration/load.go:208–216`,
|
|
|
|
|
|
`internal/configuration/prompts.go:174–180`, and
|
|
|
|
|
|
`internal/configuration/resources.go:81–87`.
|
|
|
|
|
|
|
|
|
|
|
|
The new application-owned loading path preserves ConfigKit `Defaults` and
|
|
|
|
|
|
application resolution callbacks, but the existing input-inspection machinery
|
|
|
|
|
|
still equates `FieldInfo.Explicit == false` with an absent effective value.
|
|
|
|
|
|
`finalFieldState` consequently discards metadata for values supplied by Go
|
|
|
|
|
|
defaults or recorded derived defaults. These are valid final values, not missing
|
|
|
|
|
|
configuration.
|
|
|
|
|
|
|
|
|
|
|
|
Reproduced through the public API with a completed ConfigKit load whose
|
|
|
|
|
|
`Definition.Defaults` returns a prompt input equivalent to:
|
|
|
|
|
|
|
|
|
|
|
|
```go
|
|
|
|
|
|
promptkit.PromptInputSettings{
|
|
|
|
|
|
Name: "doc", Required: true,
|
|
|
|
|
|
ContentType: new("text/plain"), Description: new("default description"),
|
|
|
|
|
|
}
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
With a valid prompt and default profile, `NewFromConfig` succeeds. The final
|
|
|
|
|
|
typed input retains those values; native metadata describes their defaulted,
|
|
|
|
|
|
nonnull presence (`Explicit: false`, `Null: false`, default layer), but:
|
|
|
|
|
|
|
|
|
|
|
|
- `InspectPrompt` returns `Required: nil` instead of the effective true value.
|
|
|
|
|
|
- `InspectRequirements` correctly classifies `doc` as required but returns nil
|
|
|
|
|
|
content type and description, contrary to its effective-metadata contract.
|
|
|
|
|
|
- `ValidateRequest` without `doc` fails with `ErrPromptRender` and a missing
|
|
|
|
|
|
required-input error, confirming that execution uses the true value that
|
|
|
|
|
|
resource inspection fails to report.
|
|
|
|
|
|
|
|
|
|
|
|
A second public-API reproduction using an application `Resolve` callback and
|
|
|
|
|
|
`RecordDerivedDefault` for an omitted `required` and `content_type` has the same
|
|
|
|
|
|
result. Source-loaded inputs ordinarily have explicit metadata, which is why
|
|
|
|
|
|
the migrated source fixtures and current handoff tests do not expose this gap.
|
|
|
|
|
|
|
|
|
|
|
|
Required change: capture effective input metadata from the final typed values
|
|
|
|
|
|
and native final metadata without treating source explicitness as a universal
|
|
|
|
|
|
presence test. Preserve the existing ordinary omission/null and explicit
|
|
|
|
|
|
false/empty-string behavior. Keep source provenance in native inspection;
|
|
|
|
|
|
do not inspect raw sources, replay history, reject application defaults, or
|
|
|
|
|
|
rerun resolution to repair the report.
|
|
|
|
|
|
|
|
|
|
|
|
Verification: add public integration regressions for both Go defaults and
|
|
|
|
|
|
recorded application-derived defaults. Compare `InspectPrompt`,
|
|
|
|
|
|
`InspectRequirements`, and preparation/preflight for the same captured input,
|
|
|
|
|
|
and retain existing source-presence cases. Include a collection case if the fix
|
|
|
|
|
|
introduces separate root/collection handling; the current capture logic is shared.
|
|
|
|
|
|
|
2026-11-21 17:49:05 +00:00
|
|
|
|
### 4. Avoid Expanding Inheritance Provenance For Input-Presence Flags
|
|
|
|
|
|
|
|
|
|
|
|
Status: open for consideration. Priority: P3. Implementation stage: 6.
|
|
|
|
|
|
Related to finding 3; consider addressing them together.
|
|
|
|
|
|
|
|
|
|
|
|
Locations: `internal/configuration/load.go:173–178` and `208–216`.
|
|
|
|
|
|
|
|
|
|
|
|
Capture calls `Metadata.Lookup` three times per declared input, then retains only
|
|
|
|
|
|
the `Explicit` and `Null` flags. In the selected ConfigKit v1.4.0 implementation,
|
|
|
|
|
|
`Lookup` also materializes and copies the field's complete inheritance-step list.
|
|
|
|
|
|
ConfigKit keeps that evidence compact internally; the full projection is useful
|
|
|
|
|
|
for operator inspection, but unnecessary for these flags. Its public
|
|
|
|
|
|
`Metadata.IsExplicit` method performs the effective-field query without that
|
|
|
|
|
|
projection.
|
|
|
|
|
|
|
|
|
|
|
|
Complexity: for `I` input declarations and inheritance depths `h_i`, these
|
|
|
|
|
|
presence queries perform `O(I + sum(h_i))` provenance work, apart from path
|
|
|
|
|
|
processing. A chain of `N` prompts inheriting a constant-sized input declaration
|
|
|
|
|
|
has depths `0..N-1`, producing quadratic ancestry-step work and cumulative
|
|
|
|
|
|
temporary allocations merely to capture linear-sized presence data. Reading one
|
|
|
|
|
|
metadata snapshot, as selected in the plan, does not eliminate this per-query
|
|
|
|
|
|
expansion.
|
|
|
|
|
|
|
|
|
|
|
|
Refactoring opportunity: combine effective typed values/pointer presence with
|
|
|
|
|
|
`Metadata.IsExplicit` where source explicitness is actually needed. Nullable
|
|
|
|
|
|
content-type and description pointers already distinguish retained values from
|
|
|
|
|
|
absent/null metadata for their public reports. Avoid requesting a complete origin
|
|
|
|
|
|
solely to classify presence, while correcting the default-value handling in
|
|
|
|
|
|
finding 3. Presence classification can be linear in the number of declarations,
|
|
|
|
|
|
plus ordinary path-query costs, independent of inheritance-chain length.
|
|
|
|
|
|
|
|
|
|
|
|
Keep full `Lookup` operations where source origins are genuinely needed for
|
|
|
|
|
|
resource binding. This finding does not propose a different inheritance algorithm,
|
|
|
|
|
|
metadata snapshot policy, inspection contract, or source-origin fallback.
|
|
|
|
|
|
|
|
|
|
|
|
Verification: preserve inherited, omitted/null, explicit-false, empty-string, and
|
|
|
|
|
|
application-default metadata outcomes. Confirm from the call path that the
|
|
|
|
|
|
presence-only phase no longer projects inheritance steps. No timing threshold or
|
|
|
|
|
|
real-world benchmark is needed to establish that algorithmic change.
|
|
|
|
|
|
|
|
|
|
|
|
### 5. Share Immutable Backend Defaults Across Profiles Without Overrides
|
|
|
|
|
|
|
|
|
|
|
|
Status: open for consideration. Priority: P3. Implementation stage: 6.
|
|
|
|
|
|
|
|
|
|
|
|
Locations: `internal/configuration/targets.go:27–29` and `68–78`,
|
|
|
|
|
|
`internal/backend/registry.go:39–48`, and
|
|
|
|
|
|
`internal/domain/smoke_diagnostics.go:124–128`.
|
|
|
|
|
|
|
|
|
|
|
|
Every captured profile selecting a backend calls the registry's defensive-copy
|
|
|
|
|
|
lookup. That lookup deep-copies the backend's provider parameters, and the
|
|
|
|
|
|
profile retains the copied map even when it has no provider-parameter overrides.
|
|
|
|
|
|
Many profiles selecting one backend therefore retain many identical immutable
|
|
|
|
|
|
trees. This is an inherited cost reused by the new capture path, not a newly
|
|
|
|
|
|
introduced regression. The plan selects precomputed execution profiles and
|
|
|
|
|
|
ownership boundaries, not duplication of identical defaults within the owned
|
|
|
|
|
|
snapshot.
|
|
|
|
|
|
|
|
|
|
|
|
Complexity: for `P` profiles sharing one backend with `S` provider-value nodes and
|
|
|
|
|
|
no profile-level provider maps, this step costs `Theta(P*S)` copying time and
|
|
|
|
|
|
retained parameter storage, despite only `O(P+S)` distinct input data. The cost is
|
|
|
|
|
|
quadratic when both dimensions grow together. Existing ID sorting is additional
|
|
|
|
|
|
work and is not the target of this finding.
|
|
|
|
|
|
|
|
|
|
|
|
Refactoring opportunity: obtain an owned backend-default snapshot once per
|
|
|
|
|
|
backend and share its provider map read-only among profiles without overrides.
|
|
|
|
|
|
Keep separate merged maps when a profile needs its own provider parameters.
|
|
|
|
|
|
For the no-override case, default-map copying and storage can become `O(P+S)`;
|
|
|
|
|
|
the existing sorting and public result-copy costs remain unchanged. A small
|
|
|
|
|
|
construction-local cache or use of already owned immutable definitions is
|
|
|
|
|
|
sufficient; a general persistent-map framework is unnecessary.
|
|
|
|
|
|
|
|
|
|
|
|
Preserve defensive `GetBackend` semantics and copies at runtime/caller/client
|
|
|
|
|
|
boundaries. `GetProfile` already detaches the selected target's provider map, so
|
|
|
|
|
|
sharing construction-owned defaults need not expose shared mutable state to
|
|
|
|
|
|
consumers or merge caller requests into retained configuration.
|
|
|
|
|
|
|
|
|
|
|
|
Verification: review copy placement to establish the reduction, and retain
|
|
|
|
|
|
backend/profile mutation isolation, override-merge, prepared-state ownership,
|
|
|
|
|
|
and injected-client mutation coverage. Do not make internal pointer sharing a
|
|
|
|
|
|
public guarantee or introduce benchmark thresholds.
|
|
|
|
|
|
|
2026-11-21 17:27:06 +00:00
|
|
|
|
## Initial Review Results And Validation
|
2026-11-21 16:48:34 +00:00
|
|
|
|
|
|
|
|
|
|
- Stage 1: ConfigKit v1.4.0, unchanged Go minimum, and commit-pinned attribution
|
|
|
|
|
|
match the selected module and its notices.
|
|
|
|
|
|
- Stages 2–3: configuration and runtime inputs resolve mode-dependent repair
|
|
|
|
|
|
budgets; explicit zero and null/omission, whole-contract replacement, prepared
|
|
|
|
|
|
retention, and existing repair accounting are covered.
|
|
|
|
|
|
- Stages 4–9: shared settings aliases, composed stateless rules, final-metadata
|
|
|
|
|
|
resource capture, native focused inspection, explicit source mounting, and
|
|
|
|
|
|
shared constructor/assembly paths match the target architecture.
|
|
|
|
|
|
- Stages 10–15: migrated tests and maintained offline examples exercise the
|
|
|
|
|
|
changed boundaries, including concurrent loads, cross-filesystem inheritance,
|
|
|
|
|
|
application extensions, ownership, and standalone/handoff equivalence.
|
|
|
|
|
|
- Stages 16–17: canonical documentation is updated, subject to the two wording
|
|
|
|
|
|
corrections above. No additional functional or architectural defect was found.
|
|
|
|
|
|
|
|
|
|
|
|
Fresh checks passed with Go 1.27.4, `GOTOOLCHAIN=local`, and `GOWORK=off`:
|
|
|
|
|
|
`go test -race ./...` and `go vet ./...`. Changed Go files have no `gofmt`
|
|
|
|
|
|
differences, and the implementation diff passes `git diff --check`.
|
|
|
|
|
|
|
|
|
|
|
|
This review changes only this findings document; it does not implement fixes.
|
2026-11-21 17:27:06 +00:00
|
|
|
|
|
|
|
|
|
|
## Integration-Focused Follow-Up
|
|
|
|
|
|
|
|
|
|
|
|
The second review revisited stages 1–17 at their interaction boundaries: the
|
|
|
|
|
|
ConfigKit upgrade and native metadata semantics; repair-input conversion and
|
|
|
|
|
|
prepared budgets; shared settings and application callbacks; capture, inspection,
|
|
|
|
|
|
mounting, and constructor wiring; migrated tests and examples; and their contracts
|
|
|
|
|
|
with the existing runtime. Finding 3 records the newly reproduced integration
|
|
|
|
|
|
gap. Findings 1 and 2 remain unchanged.
|
|
|
|
|
|
|
|
|
|
|
|
Additional disposable external-package checks, outside the repository, passed
|
|
|
|
|
|
under the race detector for both `NewEngine` and `NewFromConfig`. They combined
|
|
|
|
|
|
configured default repair with continuation, appended messages, direct session
|
|
|
|
|
|
selection, namespace views, frozen prepared execution, structured output,
|
|
|
|
|
|
numeric-presence settings, usage/observer accounting, capacity release, and
|
|
|
|
|
|
single-pass smoke diagnostics. Both default-metadata reproductions failed their
|
|
|
|
|
|
expected inspection assertions, including under the race detector; those failures
|
|
|
|
|
|
are the evidence for finding 3, not failures in the existing test suite.
|
|
|
|
|
|
|
|
|
|
|
|
Fresh targeted race-enabled tests passed for the public handoff and runtime
|
|
|
|
|
|
contracts, configuration and use-case packages, maintained examples, capacity,
|
|
|
|
|
|
generation, observation, rendering, and validation. No other integration defect
|
|
|
|
|
|
was found. The prior full-suite and vet results remain applicable to unchanged
|
|
|
|
|
|
implementation code. No repository implementation or test files were changed by
|
|
|
|
|
|
this review.
|
2026-11-21 17:49:05 +00:00
|
|
|
|
|
|
|
|
|
|
## Algorithmic Review
|
|
|
|
|
|
|
|
|
|
|
|
The third review considered stages 1–17 in order, excluding algorithm choices
|
|
|
|
|
|
expressly selected by the plan. Findings 4 and 5 record two local optimization
|
|
|
|
|
|
opportunities; findings 1–3 remain unchanged. No additional ordinary correctness
|
|
|
|
|
|
bug was found.
|
|
|
|
|
|
|
|
|
|
|
|
- Stages 1–3: checked the selected metadata API costs, constant-time budget
|
|
|
|
|
|
defaulting/conversion, and the unchanged bounded repair/preparation paths.
|
|
|
|
|
|
- Stages 4–6: checked shared data declarations, per-resource normalization and
|
|
|
|
|
|
validation, known-value ownership traversal, presence capture, and profile
|
|
|
|
|
|
construction. Nested resource/input loops traverse distinct data; the two
|
|
|
|
|
|
findings concern hidden work in helper calls rather than loop syntax.
|
|
|
|
|
|
- Stages 7–9: native reporting remains delegated; diagnostic copying is linear
|
|
|
|
|
|
in returned records/locations, and the constructors add no repeated load or
|
|
|
|
|
|
configuration-sized work to ordinary execution.
|
|
|
|
|
|
- Stages 10–15: checked migrated test and example helpers for additional passes
|
|
|
|
|
|
or algorithm changes; they do not introduce another production loader or
|
|
|
|
|
|
runtime resource scan.
|
|
|
|
|
|
- Stages 16–17: checked the findings against the accepted ownership, inspection,
|
|
|
|
|
|
ordering, and configuration-boundary decisions. Deliberate lexical inventories,
|
|
|
|
|
|
native ConfigKit algorithms, and full-prompt repair remain outside reconsideration.
|
|
|
|
|
|
|
|
|
|
|
|
This was static complexity analysis of PromptKit and the relevant calls into the
|
|
|
|
|
|
selected ConfigKit release. No performance measurements or benchmarks were run.
|
|
|
|
|
|
Implementation code is unchanged, so prior functional validation remains
|
|
|
|
|
|
applicable; this pass validates the findings-document diff only.
|