Files
promptkit/docs/roadmap/implementation.md

937 lines
56 KiB
Markdown
Raw Normal View History

# ConfigKit v1 And Identified Resources Implementation Plan
Status: complete.
Implement the accepted [feature roadmap](configkit.md) as one end-to-end sprint.
Each sequentially numbered stage is one prompt-sized assignment for a Luna
coding agent. The roadmap owns intent and policy; this plan owns delivery steps.
## Resume Point And Contract Correction
Stages 1–10 have recorded implementation commits through `e01b232`. Testing in
the former Stage 11 exposed an incompatible assumption in this plan: ConfigKit
v1.0.0 audits a changed sequence as one atomic value, so a field-level record does
not cover that change. Recording the whole list would replace descendant origins
needed for relative resources and inspection.
Resume at the new Stage 11, then Stage 12, which implement the approved
execution-owned interpretation instead. The former unfinished Stages 11–20 are
now Stages 13–22. Preserve the existing uncommitted Stage 11 work and integrate
it into its corresponding assignments; do not reset or restart completed work.
Earlier completion records remain historical evidence, not proof that the revised
normalization target is already implemented. The revised instructions and
Delivery Decisions below govern the final state.
## Baseline And Working Rules
The original reviewed baseline is `07a55e45f2a3c68918c12fe5caa3e814ee00227c`, with Go
1.26.0, ConfigKit v0.8.0 at the old module path, keyed resource mappings, and
descriptive prompt-version labels. No implementation changes accompany this
plan. The local ConfigKit checkout's v1.0.0 tag resolves to
`c88aac990b23f48a31f9046b830d4d0954f57f53`; its release note and configuration
contract were reviewed for the decisions below.
Upstream references are the [v1 migration note](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c88aac990b23f48a31f9046b830d4d0954f57f53/docs/releases/v1.0.0.md)
and [configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c88aac990b23f48a31f9046b830d4d0954f57f53/docs/config.md).
The local `../ConfigKit` checkout can supply the same tagged documentation for
development; it is not a committed dependency replacement or a test prerequisite.
Before each assignment, use [development orientation](../development.md) and
read the applicable [agent](../policy/agents.md),
[architecture](../policy/architecture.md), [testing](../policy/testing.md),
[documentation](../policy/documentation.md), [OSS](../policy/oss.md), or
[release](../policy/release.md) policy. Preserve the accepted
[configuration-boundary ADR](../adr/0001-configuration-boundary.md).
- Complete stages in order. Intermediate stages may fail compilation or tests
belonging to later stages. Record those known dependencies and continue;
do not add compatibility shims, duplicate fixtures, or temporary architectures
merely to make each checkpoint green. The completed sprint must be coherent.
- Dependency/API migration precedes the list migration to isolate the kinds of
changes, not to require a separate release or a fully migrated intermediate
test suite. Convert each substantive fixture to its final form once where
practical.
- Use focused validation when the affected package can run. When it cannot,
review the changed code and defer execution to the identified later stage.
Do not report deferred checks as passing. There are no intermediate human
approval, downstream release, catalog publication, or live-provider gates.
- After a completed nonempty unit, record its completion and checks beneath its
stage and commit under the agent policy. Do not create empty commits. Update
exact GoDoc as declarations change; final documentation stages reconcile the
wider narrative.
- Keep scope to the roadmap. PromptKit's module path, generation behavior,
capacity policy, runtime overrides, and application ownership remain intact.
This sprint does not authorize tagging, pushing, or changes in other projects.
## Delivery Decisions
### Dependency, Controls, And Headers
Select `go.maximumdirect.net/configkit v1.0.0` directly and declare
`go 1.27.0`. Use a locally available Go 1.27-or-newer toolchain that meets the
consumer minimum, with the same selected toolchain for maintainer tooling and
checks. Do not pin the consumer minimum to its incidental patch release.
Keep the existing ReleaseKit selection and publication workflow unless a
concrete build incompatibility requires attention; ConfigKit's upstream tool
dependency alone is not a reason to migrate PromptKit's release machinery.
Rename actual source-root `configkit:` controls to `config:`; leave nested
application data and `configkit:"secret"` tags alone. Replace removed import
`optional` fields with `required` values. Use `required: true` where a maintained
example or fixture needs a file to exist; do not set a global required-import
default. A test of missing optional imports must keep that intended behavior.
Keep `SourceVersionModeIfPresent`. The existing `VersionRule` checks exact text
`2` rather than `1`; upstream parsing permits string `"2"` or strict decimal
integer `2` and rejects unsupported types. Headerless documents remain valid.
Source-local headers, collection `release_version`, and ConfigKit software
versions are separate concepts. A standalone prompt item's root `version` is
only a source header; there is no remaining prompt-version label.
### Typed Resources, Paths, And Identity
Use slices of resource structs for root and collection `Backends`, `Profiles`,
and `Prompts`, with `ID string` bound as `yaml:"id"` on each internal record.
Keep `Collections` as `map[string]collectionConfig`. Preserve the other field
types and allowed domain values except for removal of the prompt label.
ConfigKit consumes `extends`; internal resource structs need no parallel
inheritance field or relationship interpreter.
Register these six destination rules on the one loader definition:
| Destination | Registration | Identity |
| --- | --- | --- |
| Root `backends`, `profiles`, `prompts` | One `SequenceIdentityAt(configkit.Keys(kind), "id")` per kind | Direct `id` |
| `collections/<any-key>/backends`, `profiles`, `prompts` | One `SequenceIdentityMatching` per kind using literal `collections`, `SequenceAnyMappingKey()`, and literal kind | Direct `id` |
Use `configkit.SequenceSelector` with `SequenceLiteralKey` segments for each
dynamic selector. A literal `"*"` namespace is not the wildcard selector.
Register once without discovering source documents or enumerating namespaces.
Missing destinations need no artificial empty value or declaration.
Separate three things throughout the adapter:
- effective path: resource-list base plus `.Index(i)`, then the actual field;
- local identity: `record.ID`, validated with existing exact local-ID rules;
- runtime identity: existing namespace qualification of that local ID.
Use the index path for presence, validation errors, origins, and sensitivity.
Use the full identity for execution-owned maps. Preserve the retained ConfigKit
tree as composed lists; normalize/default owned interpretations rather than
mutating those lists or recording field/container replacements in them.
Build indexes in bounded passes over final records rather than repeatedly
searching lists by ID during execution. Keep existing inventory sorting; this
is not a promise that every operation is linear or that list order drives runs.
ConfigKit owns duplicate-identity checks, composition, final registered-identity
checks, and sibling inheritance. PromptKit owns accepted shapes, string-ID
syntax, domain settings, final references, and prompt materialization. Do not
prevalidate sources or re-resolve history. Non-string/null/missing IDs fail
through the existing typed/schema/domain boundary as appropriate; there is no
filename fallback, trimming, or new namespace constraint.
### Execution-Owned Interpretation And Inspection
Keep one ConfigKit load and one interpretation of its final effective records.
The production resolver captures detached `EntryInfo` presence/null information
as needed but does not mutate resource lists. Never retain the callback context.
ConfigKit's registered final identity checks continue to run normally.
Construct owned normalized resource interpretations during the existing semantic
validation boundary, and validate those values using ConfigKit's `Validation`
context for source-located errors. This can reuse the private record/document
types and existing domain helpers; it does not need a second schema, generic
cloning framework, or a new public resolver API. Retain the interpreted values
for post-load target assembly and prompt materialization so that normalization
does not run again on every request. If validation fails, no engine is published.
Only the owned interpretation may be modified. Copy the slices and nested
structures that normalization writes, or construct fresh execution values;
replacing a field through a shared nested pointer can otherwise mutate the
ConfigKit tree indirectly. Preserve exact IDs, references, and record-to-index
association while constructing this interpretation. Read presence and origins
from ConfigKit, not from guesses based on zero/nil interpreted values. Existing
provider-map and snapshot ownership rules remain in place.
Preserve current domain normalization/defaulting behavior, including endpoints,
credential names, diagnostic settings, positive-concurrency queue defaults,
profile controls, prompt metadata/input names, session templates, output schema
paths, roles, and cache control. Do not change trimming or explicit-null semantics
merely because the transformation moves. Validate normalized values without
mutating the configuration passed to `Validate`. Keep normalization failures and
domain errors at their existing public boundary with concrete list-field paths.
Moved checks may now produce validation-phase rather than resolution-phase
ConfigKit diagnostics; preserve meaningful public error identity and source
locations rather than an obsolete phase-specific expectation.
Configuration inspection, metadata, explanations, and diffs expose ConfigKit's
composed values/provenance. PromptKit's normalization/defaults appear in its
interpreted metadata and effective execution views, not as ConfigKit resolution
events. For example, inspection may retain a padded endpoint and an omitted or
null queue capacity, while the execution backend has the normalized endpoint
and the existing default capacity. Do not insert those defaults into
`Result.Config`, fabricate normalization history, or add a second public report.
Existing `InspectPrompt`, `InspectProfile`, preparation, and diagnostic results
continue exposing the interpreted information their contracts already own.
Resolve `content_file` using the unchanged ConfigKit field origin and
`SourceContext`; interpreted list positions remain associated with those paths.
Do not save a parallel history-based source resolver or record whole-list
normalization merely to pass the mutation audit. This is final-value domain
interpretation under ADR 0001, not raw-source validation or another composition
pass. Upstream field-granular mutation support remains a
[future reevaluation](future.md#field-level-normalization-with-preserved-list-provenance),
not a prerequisite for finishing this sprint on v1.0.0.
### Layering And Resource Context
Within one strict tier, duplicate IDs fail. Across tiers, registered sequences
default to upsert, replacing a matching record in full. Explicit append/replace
remain upstream operations; an empty append/upsert adds nothing, while explicit
replacement with `[]` clears the list. Do not recreate the old fieldwise merge
between same-ID records. Mapping overlay within inherited records is still
ConfigKit behavior, and backend-to-profile setting completion stays PromptKit
behavior; neither is removed by whole-record upsert.
All three resource kinds can use `extends: {id: parent}` within the same list.
Parents are ordinary validated resources, children retain their own identities,
and only surviving records supply parents. Self-extension cannot recover a
displaced lower-tier record. Keep partial profiles valid until request
completion; do not introduce abstract records or prompt-specific inheritance
restrictions.
Resolve each effective `content_file` through its final index-based field origin
and contributing `SourceContext`, including inherited fields and moved/replaced
records. Keep the existing construction-wide lazy metadata snapshot, synchronous
content capture, handle closure, and captured runtime lookups. Origins are
resource-resolution information from the handoff, not a source-history audit.
### Public Builders And Removed Labels
The final builder signatures are:
```go
func ProfileSource(label string, profiles []ProfileConfig) configkit.Source
func BackendSource(label string, backends []BackendConfig) configkit.Source
```
Add `ID string` to both public config structs. Preserve existing optional field
types, including `ProfileConfig.Extends *string`; when non-nil, encode it as
`extends: {id: *config.Extends}` rather than a scalar reference. Omission remains
omission. Both builders emit explicit-ID record lists in supplied order, retain
duplicates for ConfigKit to reject on loading, preserve existing ownership of
provider maps/diagnostic values, and do not independently validate definitions.
No map overload, new backend inheritance convenience, or public prompt builder
is required. Backend/prompt inheritance is available through configuration.
Remove prompt-version fields, parameters, copying, formatting, and JSON
`prompt_version` members throughout domain, application, and facade values.
Keep `PromptIdentity` as an ID-only struct and retain the existing listing return
type rather than redesigning it as a string slice. Preserve prompt hashes as
opaque equality values; removing a hashed field can change them. Keep collection
release labels, ConfigKit version evidence, session IDs, and unrelated version
identifiers. No replacement label or consumer compatibility alias is needed.
## Stage 1: Adopt The Module, Toolchain, And Renamed APIs
Scope: `go.mod`, `go.sum`, imports, and the existing ConfigKit adapter/facade.
1. Select Go 1.27 or newer locally, set the consumer minimum to 1.27.0, and
replace the old ConfigKit requirement with the new module at v1.0.0. Prepare
dependency sums through ordinary Go tooling. Do not commit workspace files or
filesystem replacements.
2. Update imports in production, tests, and examples. Migrate `Definition.Secret`
to `Definition.Sensitive`, `Resolution.InputInfo` to `Resolution.EntryInfo`,
and resolution recording calls to `RecordNormalization`/`RecordDerivedDefault`
as used. Recorded mutations must still perform their assignments; recording
methods do not assign values. Typed `Secret` references remain unchanged.
This is API migration guidance where recording is still used; the revised
production interpretation does not require resource-list mutation records.
3. Audit public inspection wrappers and their tests against the upstream release
note's complete rename table: version evidence, empty import selections,
fallback destination flags, import defaults, layer names, and history-limit
diagnostics. Change symbols/expectations actually used here, not unrelated
identifiers by global text substitution.
4. Update the ConfigKit attribution entry to the selected commit above, the new
source repository, and its verified COPYRIGHT/LICENSE notices. Review resolved
module changes, including any additional tooling dependency, proportionately.
Validation: check the module diff and imports; attempt focused adapter builds
when source controls permit. Record any toolchain/dependency-access failure
accurately. Do not use local substitution as the delivered dependency state.
2026-11-02 03:51:39 +00:00
Completion: selected Go 1.27.0 and ConfigKit v1.0.0 at the specified commit,
updated imports, adapter API names and attribution, and audited the upstream
rename table against the facade and tests. The existing ReleaseKit selection
remains unchanged; ConfigKit's separate tool dependency appears in the resolved
module graph. Validation passed: focused adapter and facade compilation, the
configuration header-check test, module/import review, formatting, and
`git diff --check`. Source controls and resource-shape fixtures remain for the
subsequent assignments.
## Stage 2: Migrate Source Controls And Import Expectations
Scope: actual configuration controls in fixtures, programmatic sources, and
maintained example assets; application resource shape can still be transitional.
1. Rename source-root controls to `config` in main documents, imported files,
collection entries, test strings, and programmatic maps. Inspect context for
each change rather than replacing every occurrence of `configkit`.
2. Convert removed `optional` fields and make intended required imports explicit.
Preserve tests whose purpose is optional selection. Keep directory patterns,
recursion, destinations, import opt-in, and filesystem roots as selected by
the caller; do not invent glob `**` semantics or directory discovery rules.
3. Update source-evidence assertions for ConfigKit v1 names and `main` layer
spelling. Test meaningful source attribution rather than preserving historical
source order, complete rendering, or incidental history-event counts.
Validation: use the existing source-helper/import tests when runnable; otherwise
record remaining resource-shape dependencies. Leave source headers for Stage 3.
2026-11-02 03:52:25 +00:00
Completion: source-root import controls now use `config`, with the selected
file imports explicitly required. Reviewed source-evidence assertions and found
no obsolete API or main-layer spellings in use. Validation passed: the native
and embedded collection input tests, directory/filesystem source-helper tests,
and inherited imported-content test on Go 1.27.0; `git diff --check` passed.
Resource formats and headers remain for the following assignments.
## Stage 3: Introduce Identified Record Shapes And Header Version 2
Scope: schema types and loader definition in `internal/configuration/load.go`.
1. Convert the six root/collection resource fields to slices and add record IDs.
Remove `promptConfig.Version`; retain document `Version` and collection
`ReleaseVersion`. Do not decode both mapping and list formats.
2. Add all six identity registrations exactly as described above. Advance the
optional checker to exact specification version 2 without changing its mode.
3. Adapt `resourceSet`/`resourceSets` to expose slices and their existing base
paths. Preserve root and arbitrary collection traversal without pre-scanning
inputs. Keep namespace syntax validation at the effective configuration.
4. Update one small adapter fixture to the final shape to exercise registered
root and dynamic destinations; do not migrate the entire suite in this stage.
Validation: inspect the registration selectors and schema; when runnable, load
one headerless root list and one version-2 collection. Resolution/validation
callers are completed in the following stages.
Completion: the typed document now uses identified resource slices, registers
all six root and collection identity rules, and checks specification version 2.
Added a focused root/collection list fixture and reviewed the selectors and
schema. The focused test cannot yet compile because field capture, resolution,
validation, indexing, prompt materialization, and older fixtures still expect
maps; those migrations are assigned to the following stages. Formatting and
`git diff --check` passed; execution of the new fixture remains deferred.
## Stage 4: Preserve Field Presence And Interpretation At List Paths
Scope: backend/profile field capture, normalization, and defaulting in
`internal/configuration/load.go` and their focused tests.
Revised target: the recorded implementation below used ConfigKit resolution
mutations; Stage 11 replaces that mechanism with execution-owned interpretation.
1. Iterate records by index, retaining the local ID for full-ID state-map keys.
Replace resource `.Key(id)` paths with `.Index(i)` for effective fields.
2. Adapt `captureFields` and related helpers to query `EntryInfo` before defaults
and normalization. Preserve absent, null, explicit zero/false/empty, inherited,
and backend-completed settings using the current semantics.
3. Interpret backend/profile records on owned copies rather than mutating the
actual ConfigKit slice elements. Keep existing endpoint, credential-name,
diagnostic-setting, and queue-capacity rules; do not record these changes
against ConfigKit list fields or entire lists.
4. Convert the existing focused default/presence fixtures. Update a same-ID
layered fixture to whole-record upsert rather than adding a hidden merge to
keep its previous expectation.
Validation: owning tests demonstrate explicit zeros/nulls and backend completion
in execution values while composed values and source origins stay unchanged.
No new readiness rule or ConfigKit normalization-event expectation.
Completion: backend and profile presence capture and resolution now use list
indexes and mutate the effective slice elements while retaining full-ID state
keys. Converted focused normalization, null/default, profile-presence, and
same-ID replacement fixtures. Their execution remains deferred: the package
still does not compile until the subsequent indexing, validation, and prompt
list migrations. Formatting and `git diff --check` passed.
Recovery required: this completion predates the atomic-list audit finding.
The revised production target and test expectations are assigned to Stage 11.
## Stage 5: Validate Final Records And Build Execution Indexes
Scope: `validateEffective`, `indexedBackends`, `indexedProfiles`, and
`internal/configuration/targets.go`.
1. Validate local IDs and settings at final index-based paths. Build the set of
full backend IDs from final records for ordinary profile-reference checks.
Keep namespace/local-ID syntax, model strings, and deferred prompt/default
readiness semantics unchanged.
2. Build backend/profile indexes from interpreted records using each record's
ID and namespace, leaving `result.Config` as the composed inspection tree.
Update capacity-policy and execution-target assembly to use those indexes and
the Stage 4 presence state. Do not inspect history or add a second ConfigKit load.
3. Keep final inventories sorted and collection membership independent of list
position. Do not conflate local IDs in separate namespaces or resource kinds.
4. Convert target-focused fixtures and retain runtime override, provider-map,
credential omission, concurrency, and queue-policy behavior.
Validation: target tests cover a root resource, namespaced resources sharing a
local ID, and partial-profile completion. Upstream duplicates must fail before
an execution map can silently overwrite a record.
Completion: final backend, profile, and prompt IDs and references are validated
at list-entry paths; backend and profile runtime indexes use full identities.
Added root and independent namespace target coverage and converted the strict
duplicate fixture to identified lists. The focused tests remain deferred because
prompt materialization and remaining configuration fixtures have not yet been
migrated. Formatting and `git diff --check` passed.
Recovery required: Stage 11 switches runtime construction from raw retained
records to their owned normalized interpretation.
## Stage 6: Materialize Prompts From Effective List Entries
Scope: prompt interpretation/capture in `internal/configuration/prompts.go` and
prompt-input presence capture in `load.go`.
1. Iterate final prompt records using their IDs for runtime keys and their list
indexes for metadata paths, message paths, and validation errors. Remove the
raw prompt-label parameter from materialization; remaining internal label
plumbing is removed in Stage 9.
2. Adapt input-description/content-type presence capture to the new paths.
Normalize/default owned prompt interpretations, not ConfigKit list fields.
Continue compiling captured message/session templates and validating
interpreted output contracts only.
3. Resolve file-backed fields from their contributing origin and SourceContext
after imports, inherited prompt definitions, upsert, and index movement.
Share the existing lazy metadata snapshot across all resources, and preserve
content-handle closure and post-construction capture.
4. Convert/extend the existing native and in-memory filesystem tests with
inherited prompt content and a replaced record from another directory. Use
distinctive synthetic file contents to prove which source contributed it.
Validation: owning prompt tests prove correct content and input metadata after
inheritance/replacement, including file edits not changing a constructed engine.
Do not normalize a whole list solely to update a field and lose its origins.
2026-11-02 03:56:23 +00:00
Completion: prompt capture, normalization, input presence, and materialization
now use effective list indexes and full runtime IDs, preserving the shared lazy
origin snapshot. Converted native, explicit-context, inherited-import, and
replacement content fixtures. Focused execution is deferred because older
loader tests still index resource maps and reference removed prompt metadata;
their conversion is assigned to later work. Formatting and `git diff --check`
passed.
Recovery required: Stage 12 moves remaining prompt normalization off ConfigKit
state and verifies resource origins after that change.
## Stage 7: Preserve Inspection And Sensitive-Value Handling
Scope: `internal/configuration/inspection.go`, configuration inspection/diff
facade tests, and path-specific metadata expectations.
1. Adapt provider-extra sensitivity matching for root and collection resource
list indexes, retaining the existing sensitive key policy and nested values.
Use ConfigKit v1's `Sensitive` hook and schema tags where already applicable.
2. Convert inspection queries and fixtures from resource-key paths to concrete
index paths. Retained output stays in composed list shape without PromptKit
normalization/default writes; runtime IDs are not synthetic inspection keys.
3. Preserve redaction in effective output, explanations, failure diagnostics,
metadata, and diffs through existing owning coverage. Include a replaced or
inherited sensitive record at a different index where it adds protection.
4. Retain the existing thin ConfigKit wrappers. Do not add inspection methods
merely because upstream v1 offers them, or invoke inspection during a run.
Validation: converted inspection/diff tests find the expected values/origins and
do not disclose synthetic credentials. Relax only assertions tied to deliberate
upstream history/diagnostic changes, not meaningful redaction expectations.
Completion: sensitivity matching and inspection paths now recognize resource
list indexes. Public inspection and diff fixtures use identified lists and
ConfigKit's atomic sequence comparison, including replaced and inherited
sensitive values, metadata, failure diagnostics, and detached reports. Validation
passed: `go test . -run '^TestConfigurationInspection' -count=1` with Go 1.27.0,
formatting, and `git diff --check`. Internal package tests remain deferred until
the remaining map-indexed loader fixtures are converted.
Recovery required: Stage 12 reconciles inspection expectations with composed
configuration versus interpreted execution values. Preserve the redaction work.
## Stage 8: Convert Public Source Builders To Explicit-ID Slices
Scope: `profiles.go`, `backends.go`, source-builder GoDoc, and owning construction
tests. Other callers are migrated in later stages.
1. Implement the exact public signatures and fields in Delivery Decisions.
Emit slice records with explicit `id`; preserve caller order and duplicates.
2. Encode non-nil profile `Extends` as an identity mapping. Preserve omitted
options, explicit false/zero values, provider-map copying, and diagnostic
configuration ownership. Leave value validation to loading.
3. Convert owning builder tests to slices. Check ordered output and that duplicate
records reach loading and are rejected rather than silently deduplicated.
Retain a public profile-variant test through real ConfigKit inheritance.
Validation: the public construction/source-builder tests demonstrate the new
input shape and ownership. Do not add map compatibility or builder-only ID rules.
Completion: profile and backend builders now accept ordered explicit-ID slices,
preserve duplicates and owned values, and encode profile inheritance by sibling
identity. Migrated their owning construction callers and added focused tests
for order, copying, duplicate rejection, and real profile inheritance. Validation
passed: the isolated source-builder contract tests and `go build .` on Go 1.27.0,
formatting, and `git diff --check`. The full root test package remains deferred
until later callers are converted to the new signatures.
## Stage 9: Remove Internal Prompt-Version Propagation
Scope: domain values, prompt requirement snapshots, hashes, use cases, and
observation recording; facade cleanup follows in Stage 10.
1. Remove prompt `Version`/`PromptVersion` from domain definitions, identities,
prepared/run/inspection/requirement/smoke values and internal snapshot types.
Remove corresponding parameters, assignments, and label-only clone helpers.
Preserve helpers still needed for collection labels or other optional values.
2. Update prompt hashing, render/requirement snapshots, runner preparation,
prepared-execution cloning, profile-independent inspection, smoke selection,
and observation recording to use prompt identity without a label.
3. Remove internal tests exclusively about absent/empty/inherited prompt labels.
Keep their independent snapshot-ownership and hash/metadata protections by
exercising retained fields, not by deleting the wider regression coverage.
Validation: focused domain/use-case tests when runnable and symbol searches for
remaining label plumbing. Do not promise old hash values or remove source-version
checks, collection release labels, or session metadata.
Completion: internal prompt definitions, identities, snapshots, execution,
inspection, smoke diagnostics, and observations no longer propagate prompt
labels. Label-only tests were removed while retained-field ownership and metadata
checks remain. Domain, observation, and prompt package tests passed, as did
focused runner and inspection tests with Go 1.27.0; symbol review and
`git diff --check` passed. Full use-case tests still have failing configuration
fixtures awaiting their scheduled list conversion; public facade cleanup follows.
## Stage 10: Remove Public Prompt Labels And Serialized Fields
Scope: `types.go`, `listing.go`, `requirements.go`, `observations.go`,
`smoke_diagnostics.go`, `convert.go`, formatting, and owning JSON tests.
1. Remove public prompt-version fields and their conversions, including
`PromptIdentity.Version`; preserve the ID-only struct and listing method shape.
2. Remove `prompt_version` JSON fields from prepared/results, observation, and
diagnostic values wherever present. Update redacted formatting and exact
GoDoc to avoid referring to removed metadata.
3. Adapt existing JSON contract tests to verify the field is absent from the
affected representative public values. Retain the existing unrelated
serialization, ownership, and sensitive-data checks.
Validation: compile the facade when remaining fixtures permit and run JSON
contract tests. Search production code for stale prompt labels, separating
unrelated version fields from the removed feature.
2026-11-02 04:04:35 +00:00
Completion: public prompt identities and execution, inspection, requirement,
observation, and diagnostic values no longer expose prompt-version metadata.
Updated the conversions and GoDoc, and retained JSON round-trip checks while
asserting the removed field is absent from representative values. The isolated
JSON contract suite passed on Go 1.27.0, as did formatting and `git diff --check`.
Remaining callers and the maintained example are assigned to later work.
## Stage 11: Recover Backend And Profile Interpretation Without List Mutation
Scope: the production definition/resolver/validator in
`internal/configuration/load.go`, target assembly in `targets.go`, and the
focused normalization/presence tests. This is the first resumption assignment.
1. Preserve the existing in-progress definition extraction and integration tests.
Remove production resolver calls that mutate resource lists, including the
prompt-list call that Stage 12 replaces. Retain the six identity registrations
and read-only presence capture; do not add covering whole-list records.
2. Build owned backend/profile interpretations from the final typed records at
the semantic validation boundary. Reuse the private typed structures and
domain normalization helpers. Keep each record's original index/path and
exact ID, and detach nested state that interpretation modifies. Report
normalization failures through the validation context using those paths.
3. Preserve backend endpoints, credential-name handling, diagnostic endpoint/kind
normalization, queue defaults, profile controls, and absent/null/explicit
values. Keep `EntryInfo` capture separate from the interpreted defaults.
Validate owned values without modifying the supplied ConfigKit configuration.
4. Retain the interpreted values for runtime backend/profile/capacity assembly;
stop those paths from assuming `result.Config` has been normalized. Do not
mutate `Result.Config` after success or rebuild definitions on ordinary runs.
5. Update the existing backend-normalization/null-default test to check two
outcomes: normalized/defaulted execution values and unchanged composed
values with original field provenance. Remove its expectation of a ConfigKit
`ChangeNormalization` event for PromptKit's transformation. Retain tests for
explicit zero/false/null and the existing public configuration-error boundary.
Validation: targeted real-loader cases with a padded endpoint and positive
concurrency plus omitted/null queue capacity load on the selected v1.0.0 without
an unrecorded-change error. Their execution values keep current behavior, while
inspection does not acquire synthesized whole-list origins/defaults. Record
unrelated old-fixture failures for their assigned stages rather than restoring
mutating resolution to satisfy them.
Completion: backend and profile normalization and defaults now operate on
detached interpreted records during validation, with presence captured from the
unchanged ConfigKit tree. Runtime indexes consume those interpreted values.
Focused real-loader tests passed for padded endpoints, nested diagnostics,
profile settings, omitted/null queue defaults, inherited targets, and preserved
composed values and origins on Go 1.27.0. Prompt interpretation follows in the
next assignment; older fixtures remain assigned to later integration work.
## Stage 12: Recover Prompt Interpretation And Preserve Inspection Origins
Scope: prompt interpretation/materialization in `load.go`/`prompts.go` and the
owning prompt/inspection tests, completing Stage 11's production recovery.
1. Move prompt description/session, input-name/content-type/description, and
output-schema-path normalization into owned interpreted records. Reuse the
Stage 11 ownership pattern and existing message role/cache-control handling.
Clone nested output/input structures that are modified; do not trim IDs or
stored resource references. Validate interpreted names/settings so normalization
cannot conceal a duplicate input name or invalid output contract.
2. Materialize prompts from interpreted records while using the unchanged
ConfigKit effective field paths, origins, and `SourceContext` for file reads.
Preserve one lazy metadata snapshot across root/collection resources, content
capture/closure, and existing prompt hash and requirement metadata behavior.
3. Extend an existing imported/inherited-content case with prompt metadata that
actually needs trimming, including a resource from another directory and a
replaced/moved record as appropriate. Verify the correct file is read and the
field retains its contributing source even when other fields normalize.
4. Reconcile inspection/diff tests with the agreed views: ConfigKit composed
values/provenance versus PromptKit-normalized metadata and execution settings.
Verify that normalizing the owned interpretation leaves the retained values
and origins unchanged, including nested structures. Keep redaction coverage
and upstream composition/inheritance history; do not manufacture records for
PromptKit transformations or duplicate the full test matrix.
5. Search production resource interpretation for leftover `Normalize`, `Default`,
or `RecordNormalization` calls writing ConfigKit list fields. Remove the
obsolete mutation-only helpers where unused. Deliberate recorded whole-list
mutations in upstream-registration tests are separate and remain legitimate.
Validation: the source-origin regression and focused normalized metadata,
inspection, and redaction tests pass without whole-list normalization. Complete
the backend/profile/prompt recovery before continuing the existing integration
work. No upstream release, extra public API, or provenance repair layer is needed.
Completion: prompt descriptions, sessions, inputs, and schema paths are now
interpreted on detached records; materialization retains the composed list paths
for source-relative content. The inherited/imported content regression now covers
trimmed metadata, replacement from another directory, retained origins, and
post-construction capture. Focused prompt and target interpretation tests passed
on Go 1.27.0, including nested ownership and duplicate normalized input names.
Production configuration code no longer records mutations to resource lists.
The broader facade inspection suite remains dependent on later fixture migration.
## Stage 13: Verify Identified Composition And Inheritance
Scope: `internal/configuration/load_test.go`, `targets_test.go`, and focused
configuration fixtures not yet converted.
1. Convert remaining loader fixtures to explicit-ID lists and identity-reference
`extends`. Keep malformed-shape cases malformed for their intended reason.
2. Cover all six identity registrations without duplicating the generic upstream
matrix: root kinds plus dynamic collection kinds, duplicate strict peers,
arbitrary namespaces including a literal `*` or numeric-looking namespace,
and equal local IDs in different kinds/namespaces.
3. Through the real PromptKit loader definition, test final registered-identity
enforcement using a package-local test seam for supported application
resolution if needed. Reuse/extract the loader definition rather than writing
a parallel test-only registration list or adding a public resolver API.
4. Cover surviving parent/child inheritance for profiles, backends, and prompts;
child identity; missing/cyclic/self parents via upstream error propagation;
valid ordinary parents; and partial profiles completed by requests.
5. Cover whole-record layered upsert, explicit append/replace, empty upsert versus
clearing replacement, and replaced lower-tier invalid domain values not being
separately validated. Preserve failure on upstream syntax/structural errors.
Validation: real-loader tests assert effective behavior and structured diagnostic
identity/paths as appropriate, not complete upstream error wording or ordering.
Completion: loader fixtures now exercise identified lists, inherited records,
strict duplicates, invalid identities and parent references, all six registered
destinations, final identity enforcement, layered upsert/append/replace, and
request completion of partial profiles. The focused real-loader composition,
target, validation, and syntax tests passed on Go 1.27.0. Header and remaining
adapter fixtures are handled in the next assignment.
## Stage 14: Complete Header, Origin, And Adapter Fixture Coverage
Scope: remaining `internal/configuration` prompt, inspection, listing, and
source-context tests and `internal/engine` configuration fixtures.
1. Finish fixture conversion without recreating label-specific tests. Preserve
local import roots, explicit-null cases, input metadata, static syntax,
snapshot ownership, and sorted inventory behavior.
2. Cover headers `2` and `"2"`, headerless entry/fragments/items, rejected
declared version 1 and malformed declared types, and rejection of the old
resource mapping shape. Do not require every document to declare a header.
3. Keep nested application `config` data distinct from source controls. Use the
existing schema's allowed/rejected shapes; do not create a new application
field just to demonstrate upstream control extraction.
4. Close any missing source-origin or sensitive-path cases identified in
Stages 6–7 and their recovery in Stages 11–12. Reuse those owning tests instead
of repeating their matrix here.
Validation: run the configuration and internal-engine packages together when
they compile. Correct production defects before adjusting meaningful assertions.
Completion: remaining adapter and internal-engine fixtures use identified lists;
coverage retains inventory ordering, null and metadata presence, prompt snapshot
ownership, syntax validation, and file origins. Added checks for version 2 in
string and integer forms, malformed headers, legacy mappings, and a headerless
imported item. Both configuration and internal-engine suites passed on Go 1.27.0.
## Stage 15: Migrate Public Configuration, Inventory, And Scope Tests
Scope: root construction, sources, inspection/diff, inheritance, profile
inspection, listing, and namespace contract tests.
1. Convert YAML, programmatic mappings, and builder callers to final lists and
explicit IDs. Remove prompt labels without changing the purpose of each test.
2. Change same-ID customization fixtures deliberately: use distinct inherited
variants for field customization, or retain same-ID records when whole-record
replacement is the behavior under test. Do not silently change replacement
into inheritance in tests intended to protect precedence.
3. Preserve full-ID listings/filtering, global namespace/collection inventories,
metadata-only collections, independent namespaces, shared scoped-engine
ownership, and unchanged exact stored references.
4. Confirm the public facade accepts new-module ConfigKit inputs and exposes
new-module inspection values; old-path type aliases/adapters are not needed.
Validation: run the affected external-package contract tests using real ConfigKit
composition and offline sources. Do not add index/list-position guarantees.
Completion: public source, construction, profile inheritance/inspection,
namespace, and inventory fixtures now use explicit-ID lists and identity
inheritance. Layered replacements supply complete records, and configuration
inspection assertions reflect composed values. Focused offline contract tests
passed on Go 1.27.0. Tests depending on the shared legacy execution fixture and
the full root package remain assigned to subsequent fixture migrations.
## Stage 16: Migrate Internal Execution And Rendering Fixtures
Scope: remaining `internal/usecase`, `internal/prompt`, `internal/domain`, and
internal observation fixtures affected by resource lists or removed labels.
1. Convert source-backed fixtures and builder callers. Remove direct internal
struct labels and assertions only about the removed metadata; retain prompt
ID, body/output, session, usage, and substantive observation assertions.
2. Reuse current runner, requirements, prepared-execution, and smoke-selection
tests to confirm captured full identities remain their selection input.
Keep tests using direct execution-owned values independent of YAML where
they already protect that separate boundary.
3. Preserve renderer behavior, structural repairs, transient retries,
cancellation, and resource snapshots through their existing owning tests.
Do not redesign those mechanisms or introduce runner-time list searches.
Validation: run the affected internal packages with final values and synthetic
sources. Resolve remaining label-related compilation errors in these packages.
Completion: internal smoke-selection and execution fixtures now use identified
resource lists, preserving their configured routing and diagnostic behavior.
The use-case, prompt, domain, and observation package suites passed on Go 1.27.0.
## Stage 17: Migrate Public Preparation And Execution Contracts
Scope: root ordinary/prepared execution, concrete-request validation,
requirements, and capacity contract fixtures.
1. Convert remaining YAML, programmatic definitions, and builder callers for
Run, Prepare, PrepareExecution/RunPrepared, ValidateRequest, and
InspectRequirements. Retain their established substantive outcomes.
2. Keep runtime overrides, artifact/schema ownership, appended messages,
cancellation, prepared-handle isolation, and shared capacity coverage at the
boundaries that already own it. Update frozen-rendering/source fixtures
without reintroducing prompt-label metadata.
3. Distinguish intentional whole-record replacement from inherited variants in
request-level fixture layering; do not weaken test expectations to accommodate
a broken adapter or add new upstream-availability requirements.
Validation: run the focused root preparation/execution/requirements tests using
existing injected clients or controlled HTTP transports, with no live provider.
Completion: public prepared execution, frozen rendering, requirement inspection,
capacity, and reliable-execution fixtures now use identified resources and no
prompt labels. Focused offline preparation, requirement, snapshot, and capacity
tests passed on Go 1.27.0. The remaining shared legacy helper and observation
test dependencies are assigned to later work, so the full root suite remains
deferred.
## Stage 18: Migrate Public Diagnostics And Observation Contracts
Scope: root smoke diagnostic, observation, prepared-observation, and adapter
boundary fixtures not completed by Stage 10's serialization changes.
1. Convert diagnostic backend/profile/prompt sources and observation fixtures to
the final shape. Keep exact configured references, selection precedence,
tri-state findings, timing/usage, partial cancellation, and disclosure checks.
2. Preserve observation event order and metadata that remain contractual while
removing only the prompt label. Reuse existing retry/repair and injected-client
evidence tests rather than repeating their transport matrices.
3. Confirm scoped smoke diagnostics and ordinary runs resolve from the same
captured indexes and retain their existing client/capacity ownership.
Validation: run these public contract tests offline; diagnostic operations in
tests use controlled transports, not actual backend endpoints or API keys.
Completion: public diagnostic, scoped-capacity, and observation fixtures use
identified lists and retain selection, disclosure, and ownership checks without
prompt labels. Corrected remaining request-validation callers of the migrated
requirements helper. Focused offline diagnostic and prepared-observation tests
passed on Go 1.27.0; other observation tests still rely on the shared legacy
fixture assigned to the next assignment and will be rerun after its conversion.
## Stage 19: Update Maintained Examples And Benchmarks
Scope: the three `examples/go-library` workflows, their assets, and configuration-
dependent benchmarks and shared test helpers still using the old representation.
1. Convert preparation collections/fragments, run prompt assets, and observation
prompt assets to identified lists. Include the item-import workflow using
`placement: sequence-item` and a headerless resource file under an explicit
collection input. Keep synthetic IDs/models and source-relative content.
2. Update Go builder callers to ordered explicit-ID slices and imports to the
new module. Keep the examples' distinct purposes and expected execution
outcomes; only update `scripts/check-examples.py` if an intentionally changed
output shape requires it.
3. Migrate benchmark fixture generators and remaining common test helpers.
Preserve the construction-wide origin-metadata optimization and the existing
benchmark's usefulness; do not add allocation/time pass thresholds or a new
benchmarking release gate.
Validation: run all three examples through the semantic checker once runnable.
Use a short focused construction benchmark if needed to investigate a suspected
repeated-scan regression, not to claim unmeasured performance improvements.
Completion: the three maintained examples and benchmark/shared fixtures now use
identified resources and ordered builders. The preparation example includes a
headerless item import and an inherited replacement profile. The semantic example
checker, full public contract suite, and short configuration benchmark fixture
runs passed on Go 1.27.0. The public run also verified the previously deferred
preparation and observation tests after updating their shared fixture.
## Stage 20: Document Formats And Internal Handoff
Scope: `docs/formats.md`, `docs/internal/sources.md`, and other current internal
documents whose descriptions changed.
1. Document specification 2, optional per-source headers, collection mappings
containing identified resource lists, exact ID rules, and preserved provider
model strings. Remove prompt-label fields; retain collection release labels.
2. Describe sibling identity inheritance across the three kinds, ordinary parent
validation, partial profiles, and same-ID whole-record replacement. Link to
ConfigKit v1 for generic import/control/collection semantics rather than
copying its full reference or imposing additional authoring restrictions.
3. Explain final index-based presence/resource-context interpretation, captured
runtime indexes, sensitivity, and operator-only inspection. Explain that
ConfigKit's composed values retain provenance while owned interpretations
carry PromptKit normalization/defaults; no leaf-level ConfigKit mutation or
synthetic whole-list history is required. Update current upstream references
and obsolete mapping examples without rewriting accepted ADR decision content
or historical release notes.
Validation: compare every current format example with the actual implemented
schema and owning tests. Exact Go declarations remain the API contract owner.
Completion: the format reference now describes identified lists, header 2,
sibling inheritance, whole-record replacement, and collection labels. Internal
source, runner, and observation documentation now describe captured interpreted
values and unchanged ConfigKit provenance without prompt labels. Examples were
reviewed against the schema and passing loader/example tests; `git diff --check`
passed.
## Stage 21: Document One Coherent Consumer Migration
Scope: `docs/consumers/configuration-migration.md`, the Go consumer guide, README,
remaining exported GoDoc, and toolchain/release preparation guidance.
1. Add the v0.16-era mapping-to-list upgrade to the migration guide while retaining
useful older identity/deployment guidance. Show explicit IDs, sibling identity
references, same-ID whole-record replacement, and source-builder map-to-slice
conversion with a distinct variant where customization is intended.
2. Explain the new import path and incompatible Go type identities, Go 1.27.0
minimum, actual root controls, optional imports/`required: true`, header 2,
removed prompt labels/JSON fields, changed inspection paths, and opaque hash
changes. Include the standalone-item header-versus-former-label distinction.
Explain that configuration inspection/diffs now retain composed values rather
than PromptKit-normalized/defaulted ones; the existing interpreted views serve
execution-setting inspection, with no new combined report or equality promise.
3. Update the current consumer/README examples and links to maintained artifacts.
Align toolchain/build guidance with the new minimum and the existing selected-
toolchain release procedure; do not introduce a separate release-tool upgrade,
CI wait, consumer publication check, or new platform support promise.
4. Identify this as one breaking pre-v1 minor migration. Actual version selection,
release-note creation, tagging, and publication remain normal separately
authorized release preparation. Do not make Scriptorium publication a sprint
gate or modify Scriptorium in this task.
Validation: render/read affected GoDoc; check local document links and snippets
against implemented APIs. Keep current documentation free of obsolete contracts;
old syntax may remain explicitly labeled in migration explanations and history.
Completion: the consumer migration guide covers the breaking identified-list,
ConfigKit v1, Go 1.27, source-control, builder, inspection, and prompt-label
changes while retaining earlier deployment guidance. Updated consumer examples,
README, and release-toolchain guidance. Reviewed builder GoDoc and snippets,
checked local link targets and `git diff --check`; release selection and
publication remain separate.
## Stage 22: Verify The Integrated Sprint And Record Completion
Scope: acceptance review and residual corrections, not another feature stage.
1. Trace every target section of the feature roadmap to implementation and
proportionate existing tests: new dependency/types, source controls/header,
all six identity rules, builder order/duplicates, inheritance/upsert, effective
field paths, file origins, sensitivity, removed labels, and preserved runtime
operations. Include the recovery distinction: normalized execution values
are valid without mutating ConfigKit's composed tree or its provenance.
Close actual gaps at their owning boundary.
2. Search current production code, tests, examples, module metadata, and current
docs for stale old-path imports, genuine old controls, prompt labels, keyed
resource fixtures, and version-1 headers. Classify intended migration examples,
historical release notes, unchanged tags, and unrelated version fields rather
than demanding zero textual matches.
3. With the selected Go toolchain and prepared dependencies, run:
```sh
GOTOOLCHAIN=local GOWORK=off go test -race ./...
GOTOOLCHAIN=local GOWORK=off go vet ./...
GOTOOLCHAIN=local GOWORK=off python3 scripts/check-examples.py
git diff --check
```
Format changed Go files and validate affected local Markdown links. Review
the final module graph/sums and absence of committed workspace/local
replacements. These are sprint checks, not an extra publication command or
new permanent release gates. Reuse successful checks for unchanged content.
4. Verify the only intended API breaks are those selected by the roadmap and
upstream type migration. In particular, do not add ID normalization, required
headers/imports, profile readiness, abstract parents, source auditing, or
execution-order semantics. Review final status/diff for unrelated material.
5. Record completion and validation in this plan and mark the feature roadmap
complete. Commit the final nonempty unit; leave retirement and publication to
normal release preparation. Report remaining failures honestly if the full
target has not been reached.
Validation: the acceptance trace, integrated checks above, and reviewed final
diff establish sprint completion; intermediate checks alone do not.
Completion: traced the dependency, identities, composition, interpretation,
origins, sensitivity, public builders, removed labels, and runtime behavior to
the implementation and owning tests. Added a focused check that nested `config`
remains application data. On Go 1.27.0, the full race-enabled suite, `go vet
./...`, and the three-example semantic checker passed; the added check also
passed under the race detector. Reviewed current-source searches, module graph,
local link targets, formatting, and `git diff --check`. No workspace or local
dependency replacement is committed. Publication remains separate.
## Open Questions
None required to begin the sprint. The accepted roadmap settles module identity,
resource representation, layering/inheritance, source-builder inputs, header
policy, removed labels, and the coordinated breaking upgrade. Implementation
details above serve those decisions without adding consumer guarantees. The
approved execution-owned interpretation accommodates ConfigKit v1.0.0's atomic
sequence audit; upstream field-granular normalization is not an open dependency.