2026-11-21 16:48:34 +00:00
|
|
|
|
# Configuration Handoff And Repair Review
|
|
|
|
|
|
|
|
|
|
|
|
Reviewed implementation stages 1–17 against
|
|
|
|
|
|
[the implementation plan](implementation.md),
|
|
|
|
|
|
[the feature roadmap](next-release-considerations.md), and accepted ADRs
|
|
|
|
|
|
[0001](../adr/0001-configuration-boundary.md) and
|
|
|
|
|
|
[0002](../adr/0002-shared-configuration-data-model.md).
|
|
|
|
|
|
Implementation baseline: `40f8a6825359b4923645256e230a050dfdb00710`;
|
|
|
|
|
|
reviewed changes begin after `248d0db`.
|
|
|
|
|
|
|
|
|
|
|
|
## Findings
|
|
|
|
|
|
|
|
|
|
|
|
### 1. Narrow The Pre-Handoff Mutation Wording To The Accepted Expectation
|
|
|
|
|
|
|
|
|
|
|
|
Status: open. Priority: P3. Implementation stage: 16.
|
|
|
|
|
|
|
|
|
|
|
|
Locations: `engine.go:280` and `docs/consumers/pkg-promptkit.md:132`.
|
|
|
|
|
|
|
|
|
|
|
|
`NewFromConfig` says callers **must not mutate** selected settings between loading
|
|
|
|
|
|
and handoff; the consumer guide repeats that blanket instruction. The accepted
|
|
|
|
|
|
roadmap instead asks callers to hand off before changes that invalidate the
|
|
|
|
|
|
frozen inspection/provenance evidence, and expressly treats this as a usage
|
|
|
|
|
|
expectation rather than a new enforcement mechanism. The published wording
|
|
|
|
|
|
prohibits even value-preserving reassignment or copying, which needs no such
|
|
|
|
|
|
restriction.
|
|
|
|
|
|
|
|
|
|
|
|
Required change: describe the narrower consistency expectation in both owners.
|
|
|
|
|
|
Explain that construction consumes the resolved settings while inspection retains
|
|
|
|
|
|
the completed load's frozen evidence, so callers should keep those consistent at
|
|
|
|
|
|
handoff. Preserve the existing protection against mutation after construction.
|
|
|
|
|
|
No mutation detection, extra validation pass, or new error is needed.
|
|
|
|
|
|
|
|
|
|
|
|
Verification: review both passages against the feature roadmap's ownership
|
|
|
|
|
|
paragraph; ensure they no longer impose an unconditional mutation prohibition.
|
|
|
|
|
|
This is a documentation correction and does not require a new runtime test.
|
|
|
|
|
|
|
|
|
|
|
|
### 2. Keep Application-Field Sensitivity Policy With The Application
|
|
|
|
|
|
|
|
|
|
|
|
Status: open. Priority: P3. Implementation stage: 16.
|
|
|
|
|
|
|
|
|
|
|
|
Location: `docs/formats.md:235–236`.
|
|
|
|
|
|
|
|
|
|
|
|
The backend format reference says an unrecognized backend field is application
|
|
|
|
|
|
data and **must not contain secrets**. The accepted extension-point contract
|
|
|
|
|
|
assigns interpretation, validation, and sensitivity rules for those fields to
|
|
|
|
|
|
the application. The implementation follows that contract: inline fields remain
|
|
|
|
|
|
in native inspection, application sensitivity predicates compose with PromptKit's,
|
|
|
|
|
|
and those fields are omitted from resource reports and provider parameters.
|
|
|
|
|
|
The added prohibition unnecessarily narrows the application-owned data contract.
|
|
|
|
|
|
|
|
|
|
|
|
Required change: retain the useful warning that a raw `api_key` extension is not
|
|
|
|
|
|
a PromptKit credential input and is not automatically redacted by PromptKit.
|
|
|
|
|
|
State that the application owns sensitivity declarations and disclosure policy
|
|
|
|
|
|
for its extension fields, rather than prohibiting their contents. Review the
|
|
|
|
|
|
adjacent profile wording for the same distinction between PromptKit credentials
|
|
|
|
|
|
and application-owned data. Do not add automatic credential behavior or a new
|
|
|
|
|
|
validator for application fields.
|
|
|
|
|
|
|
|
|
|
|
|
Verification: compare the revised text with the feature roadmap's
|
|
|
|
|
|
application-specific-fields contract and existing focused-inspection sensitivity
|
|
|
|
|
|
coverage. No change to execution or redaction implementation is needed.
|
|
|
|
|
|
|
2026-11-21 17:27:06 +00:00
|
|
|
|
### 3. Preserve Effective Input Metadata Supplied By Application Defaults
|
|
|
|
|
|
|
|
|
|
|
|
Status: open. Priority: P2. Implementation stages: 5–6 and 13.
|
|
|
|
|
|
|
|
|
|
|
|
Locations: `internal/configuration/load.go:208–216`,
|
|
|
|
|
|
`internal/configuration/prompts.go:174–180`, and
|
|
|
|
|
|
`internal/configuration/resources.go:81–87`.
|
|
|
|
|
|
|
|
|
|
|
|
The new application-owned loading path preserves ConfigKit `Defaults` and
|
|
|
|
|
|
application resolution callbacks, but the existing input-inspection machinery
|
|
|
|
|
|
still equates `FieldInfo.Explicit == false` with an absent effective value.
|
|
|
|
|
|
`finalFieldState` consequently discards metadata for values supplied by Go
|
|
|
|
|
|
defaults or recorded derived defaults. These are valid final values, not missing
|
|
|
|
|
|
configuration.
|
|
|
|
|
|
|
|
|
|
|
|
Reproduced through the public API with a completed ConfigKit load whose
|
|
|
|
|
|
`Definition.Defaults` returns a prompt input equivalent to:
|
|
|
|
|
|
|
|
|
|
|
|
```go
|
|
|
|
|
|
promptkit.PromptInputSettings{
|
|
|
|
|
|
Name: "doc", Required: true,
|
|
|
|
|
|
ContentType: new("text/plain"), Description: new("default description"),
|
|
|
|
|
|
}
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
With a valid prompt and default profile, `NewFromConfig` succeeds. The final
|
|
|
|
|
|
typed input retains those values; native metadata describes their defaulted,
|
|
|
|
|
|
nonnull presence (`Explicit: false`, `Null: false`, default layer), but:
|
|
|
|
|
|
|
|
|
|
|
|
- `InspectPrompt` returns `Required: nil` instead of the effective true value.
|
|
|
|
|
|
- `InspectRequirements` correctly classifies `doc` as required but returns nil
|
|
|
|
|
|
content type and description, contrary to its effective-metadata contract.
|
|
|
|
|
|
- `ValidateRequest` without `doc` fails with `ErrPromptRender` and a missing
|
|
|
|
|
|
required-input error, confirming that execution uses the true value that
|
|
|
|
|
|
resource inspection fails to report.
|
|
|
|
|
|
|
|
|
|
|
|
A second public-API reproduction using an application `Resolve` callback and
|
|
|
|
|
|
`RecordDerivedDefault` for an omitted `required` and `content_type` has the same
|
|
|
|
|
|
result. Source-loaded inputs ordinarily have explicit metadata, which is why
|
|
|
|
|
|
the migrated source fixtures and current handoff tests do not expose this gap.
|
|
|
|
|
|
|
|
|
|
|
|
Required change: capture effective input metadata from the final typed values
|
|
|
|
|
|
and native final metadata without treating source explicitness as a universal
|
|
|
|
|
|
presence test. Preserve the existing ordinary omission/null and explicit
|
|
|
|
|
|
false/empty-string behavior. Keep source provenance in native inspection;
|
|
|
|
|
|
do not inspect raw sources, replay history, reject application defaults, or
|
|
|
|
|
|
rerun resolution to repair the report.
|
|
|
|
|
|
|
|
|
|
|
|
Verification: add public integration regressions for both Go defaults and
|
|
|
|
|
|
recorded application-derived defaults. Compare `InspectPrompt`,
|
|
|
|
|
|
`InspectRequirements`, and preparation/preflight for the same captured input,
|
|
|
|
|
|
and retain existing source-presence cases. Include a collection case if the fix
|
|
|
|
|
|
introduces separate root/collection handling; the current capture logic is shared.
|
|
|
|
|
|
|
|
|
|
|
|
## Initial Review Results And Validation
|
2026-11-21 16:48:34 +00:00
|
|
|
|
|
|
|
|
|
|
- Stage 1: ConfigKit v1.4.0, unchanged Go minimum, and commit-pinned attribution
|
|
|
|
|
|
match the selected module and its notices.
|
|
|
|
|
|
- Stages 2–3: configuration and runtime inputs resolve mode-dependent repair
|
|
|
|
|
|
budgets; explicit zero and null/omission, whole-contract replacement, prepared
|
|
|
|
|
|
retention, and existing repair accounting are covered.
|
|
|
|
|
|
- Stages 4–9: shared settings aliases, composed stateless rules, final-metadata
|
|
|
|
|
|
resource capture, native focused inspection, explicit source mounting, and
|
|
|
|
|
|
shared constructor/assembly paths match the target architecture.
|
|
|
|
|
|
- Stages 10–15: migrated tests and maintained offline examples exercise the
|
|
|
|
|
|
changed boundaries, including concurrent loads, cross-filesystem inheritance,
|
|
|
|
|
|
application extensions, ownership, and standalone/handoff equivalence.
|
|
|
|
|
|
- Stages 16–17: canonical documentation is updated, subject to the two wording
|
|
|
|
|
|
corrections above. No additional functional or architectural defect was found.
|
|
|
|
|
|
|
|
|
|
|
|
Fresh checks passed with Go 1.27.4, `GOTOOLCHAIN=local`, and `GOWORK=off`:
|
|
|
|
|
|
`go test -race ./...` and `go vet ./...`. Changed Go files have no `gofmt`
|
|
|
|
|
|
differences, and the implementation diff passes `git diff --check`.
|
|
|
|
|
|
|
|
|
|
|
|
This review changes only this findings document; it does not implement fixes.
|
2026-11-21 17:27:06 +00:00
|
|
|
|
|
|
|
|
|
|
## Integration-Focused Follow-Up
|
|
|
|
|
|
|
|
|
|
|
|
The second review revisited stages 1–17 at their interaction boundaries: the
|
|
|
|
|
|
ConfigKit upgrade and native metadata semantics; repair-input conversion and
|
|
|
|
|
|
prepared budgets; shared settings and application callbacks; capture, inspection,
|
|
|
|
|
|
mounting, and constructor wiring; migrated tests and examples; and their contracts
|
|
|
|
|
|
with the existing runtime. Finding 3 records the newly reproduced integration
|
|
|
|
|
|
gap. Findings 1 and 2 remain unchanged.
|
|
|
|
|
|
|
|
|
|
|
|
Additional disposable external-package checks, outside the repository, passed
|
|
|
|
|
|
under the race detector for both `NewEngine` and `NewFromConfig`. They combined
|
|
|
|
|
|
configured default repair with continuation, appended messages, direct session
|
|
|
|
|
|
selection, namespace views, frozen prepared execution, structured output,
|
|
|
|
|
|
numeric-presence settings, usage/observer accounting, capacity release, and
|
|
|
|
|
|
single-pass smoke diagnostics. Both default-metadata reproductions failed their
|
|
|
|
|
|
expected inspection assertions, including under the race detector; those failures
|
|
|
|
|
|
are the evidence for finding 3, not failures in the existing test suite.
|
|
|
|
|
|
|
|
|
|
|
|
Fresh targeted race-enabled tests passed for the public handoff and runtime
|
|
|
|
|
|
contracts, configuration and use-case packages, maintained examples, capacity,
|
|
|
|
|
|
generation, observation, rendering, and validation. No other integration defect
|
|
|
|
|
|
was found. The prior full-suite and vet results remain applicable to unchanged
|
|
|
|
|
|
implementation code. No repository implementation or test files were changed by
|
|
|
|
|
|
this review.
|