Record ConfigKit inheritance integration findings

This commit is contained in:
2026-11-17 23:52:27 +00:00
parent 6f0a75e7cf
commit b396581d21

View File

@@ -1,8 +1,8 @@
# Configuration Upgrade Review Remediation
The earlier PromptKit findings below retain their original scope and status.
The ConfigKit 1.2.0 review and its two new findings are recorded in the final
section; they do not reopen or attribute earlier issues to that upgrade.
The ConfigKit 1.2.0 review and subsequent integration audit appear below; they
do not reopen or attribute earlier issues to that upgrade.
## Scope And Overall Assessment
@@ -245,9 +245,10 @@ Required changes:
and whole-record upsert; no new application inheritance machinery is needed.
Acceptance: an operator can determine the correct syntax for both a pipeline
mapping and a step record from the canonical reference. Any added snippet loads
through the current configuration boundary. Reuse the existing behavioral test;
do not add a documentation snapshot or change runtime behavior to fit the prose.
mapping and an identified record from the canonical reference. Any example
presented as a runnable pipeline must pass pipeline resolution, not just loading;
see CK-3 for step-record limitations. Reuse behavioral coverage rather than adding
a documentation snapshot or changing runtime behavior to fit the prose.
### Finding CK-2 — Describe Dependency-Ready Step Assembly Order
@@ -289,7 +290,7 @@ merging, or scheduler changes belong in this remediation.
| 1 — Dependencies/toolchain | The selected graph builds with Go 1.27.0, both approved ConfigKit versions, PromptKit v0.12.0, and YAML rc.6; tidy reports no diff. |
| 2 — API/handoff | Application imports use the new package, old types remain in the PromptKit boundary, and LoadSource/MainSourcePath retain frozen inspection and detached runtime values. |
| 3 — Normalization/origins | Paths resolve before key movement; unchanged step identities retain leaf records, changed identities retain covering structural records. Existing tests and an additional temporary combined key/path-normalization probe pass. |
| 4 — Collections | The exact step-only identity registration delegates composition to ConfigKit. Real-file tests cover imports, upsert, replace, append, clearing, record inheritance, and source-relative references. |
| 4 — Collections | The exact step-only identity registration delegates composition to ConfigKit. Real-file tests cover imports, upsert, replace, append, clearing, and source-relative references. Identified-record inheritance is checked only at loading/configuration validation; CK-3 records the missing runtime-consumer check. |
| 5 — Fixtures | Application fixtures use native controls and assignment timing; old PromptKit fixtures remain correctly scoped. The full suite passes. |
| 6 — CLI discovery | The shared loader describes File/SearchConfig sources once and returns the recorded selected file. Manual probe helpers are removed. |
| 7 — Discovery/failures | Controlled tests cover precedence, filename selection, required exhaustion, terminal failures, reports, symlinks, and debug provenance without relying on host system files. |
@@ -317,3 +318,90 @@ merging, or scheduler changes belong in this remediation.
Keep the earlier PromptKit findings separate from these two documentation fixes.
The deferred PromptKit/ConfigKit v1.2.1 upgrade and release work remain out of scope.
## ConfigKit Integration Audit
Reviewed on 2026-11-17 against `6f0a75e`, with particular attention to how the
11 implementation stages interact with the existing pipeline and PromptKit
consumers. The implementation is unchanged from `c9744ad`; the intervening
commits changed roadmaps only. CK-1 and CK-2 remain open.
The dependency/toolchain boundary, detached configuration conversion, path
normalization, CLI overrides, native discovery, selected-file reporting, and
old-format PromptKit source handoff are coherent. The maintained complete
example still reaches the existing multi-step runner and publishes its generated
registry/occurrence handoffs through an injected external client. Session,
checkpoint, warning, and output consumers continue to receive resolved domain
values, not ConfigKit source graphs. No new production correctness regression
was identified. One additional test/documentation integration gap remains.
### Finding CK-3 — Verify Identified-Record Inheritance At The Runtime Boundary
Priority: P3. Category: integration coverage and support-scope clarity. Status: open.
Affected stages: 4, 9, and 11.
`TestIdentifiedStepRecordInheritance` in
`internal/core/config/collection_composition_test.go:182–208` supplies a parent
step containing only references and a child step containing an artifact lane.
It checks `Load` and `Config.Validate`, but never calls `Config.Resolve` with a
catalog. The existing runner's resolver requires every explicit step to have at
least one artifact lane (`internal/framework/pipeline/profile.go`,
`ResolvePipeline`). The parent is an ordinary executable step, not an excluded
template. The fixture therefore cannot be used as a resolved pipeline, despite
the positive test and the reference's unqualified mention of step-record
inheritance.
Giving the parent a real artifact lane is not a simple repair to the fixture:
native inheritance recursively copies that named lane into the child, while
Notarius requires lane identifiers to be unique across steps. Adding a new
child-specific lane does not remove the inherited one. Pipeline-level inheritance
with step upsert is a different operation and remains supported and tested.
Offline reproduction through the current binary:
- A production-module equivalent of the test fixture loads for effective
inspection, but `config validate --pipeline session` fails because the parent
step must declare an artifact lane.
- A parent with `parent-spells` and a child declaring `child-spells` fails domain
validation because inherited `parent-spells` occurs in both steps.
- A native identity declaration on an extraction validator list, with
`generic/valid_json_schema` inheriting from `generic/valid_json` through
`extends: {module: generic/valid_json}`, passes production pipeline validation.
Generic identified-record inheritance is therefore usable; the gap is the
selected positive step fixture and its implied runtime applicability.
Required changes:
- Replace or reframe the positive step fixture. Cover a genuinely executable
identified-record use case in a compatible collection and verify it through
`Config.Resolve` with the appropriate catalog or the equivalent public CLI
validation path. Assert a meaningful inherited value, not only successful load.
- Make the step-specific constraints clear alongside CK-1's syntax correction.
Do not present the reference-only parent fixture as a runnable step template.
Preserve the distinction between pipeline inheritance/upsert and inheritance
between records of one final sequence.
- Preserve the existing runtime constraints. This finding does not authorize
template-only steps, implicit parent removal, lane renaming, or weaker lane-ID
uniqueness. A separate product decision would be needed for such features.
Acceptance: positive inheritance coverage reaches the actual configuration
consumer, and operator guidance distinguishes native composition from executable
pipeline validity. The complete D&D example, generated-reference validation, and
ordinary pipeline inheritance continue to behave unchanged. No production fix
is indicated by the current evidence.
### Integration Audit Validation
- Reviewed the staged changes against configuration conversion, pipeline
resolution/reference materialization, PromptKit construction/inspection,
checkpoint identity, and CLI debug/output consumers.
- `GOTOOLCHAIN=go1.27.0 GOWORK=off go test ./internal/core/config ./internal/cli ./internal/framework/llm ./internal/framework/pipeline ./internal/modules/integration`
passed. This includes the maintained complete-example run, generated-reference
handoffs, command overrides, selected-path debug provenance, and profile reports.
- Temporary CLI fixtures outside the repository reproduced both step-inheritance
failures and verified the compatible validator-record alternative. An additional
author-identified validator list with whitespace-normalized module names also
passed pipeline validation.
- The full-suite, vet, race, tidy, and build results from the preceding review
remain applicable to the unchanged implementation. No live provider call or
production-quality evaluation was performed.