promptkit/docs/roadmap/field-normalization.md

Field-Level Configuration Normalization

Status And Purpose

Implemented: PromptKit uses ConfigKit v1.1.0 and records the selected field normalizations and queue default in its resolved configuration.

Move PromptKit's existing configuration-time normalization and derived defaults into ConfigKit's recorded application resolution. Configuration inspection should then show the normalized values used to construct execution settings, with explanations of the changes, while unchanged resource references retain the source context needed to read their files.

The benefit is a more useful inspection view and removal of the parallel interpreted configuration maintained for execution. This is a focused loading change, not a new configuration format, normalization policy, or execution API.

Historical Baseline

PromptKit already uses identified lists for backends, profiles, and prompts at the root and within named collections. Six ConfigKit identity registrations cover those destinations. This roadmap does not repeat that migration.

At the baseline, the configuration adapter's Definition.Resolve captured input presence information. During validation, interpretConfiguration constructs detached backend, profile, and prompt interpretations and applies normalization and a conditional backend queue default. Runtime construction uses those interpretations; configuration inspection retains ConfigKit's composed values. Configured message roles and cache-control values are normalized later during prompt materialization.

This separation avoided recording whole resource lists as normalized under ConfigKit v1.0.0. Such a record would replace the effective origins of unchanged fields, including content_file, whose origin selects the filesystem and directory used to load message text.

The implemented behavior is documented in the format reference and source-loading implementation. The configuration ownership decision continues to govern the division of responsibility: ConfigKit owns composition; PromptKit interprets effective values and supplies domain rules.

Released Upstream Capability

ConfigKit v1.1.0 allows individual field records inside lists through the existing Normalize, RecordNormalization, and derived-default APIs:

• Equal-length ordinary lists are compared recursively by position. This covers nested prompt messages and inputs without registering new identities for them.
• Registered collections with valid, unique entry and final identities support matched field comparison. PromptKit already registers its resource lists.
• List length changes, and registered identity order or membership changes, still need list or ancestor coverage. Matching is conservative when identities are unsuitable. Final identity validation remains ConfigKit's responsibility.
• Entry queries retain their fixed resolution-entry meaning; records address final positions. Public effective-configuration diffs remain atomic for lists.

These capabilities support the proposed in-place field transformations. No additional ConfigKit API is a prerequisite. Historical source evidence retained for a matched record is distinct from its effective origin: a whole-list resolution record can still replace descendant effective origins. The migration should use individual field records for individual changes.

Target End State

One Normalized Configuration For Loading And Inspection

PromptKit's Definition.Resolve captures the entry metadata needed for existing presence/null semantics, then applies configuration-time transformations to the working configuration and records their effects at the affected fields. Definition.Validate examines the resulting values without mutating them. Runtime indexes and prompt materialization consume that same normalized configuration, producing the execution-owned domain values they need.

Remove state.interpreted and the detached interpretation copies once their normalization work has moved. Retain copies needed for runtime ownership and public detached results; removing the duplicate configuration is not a reason to expose shared mutable runtime state.

PromptKit continues to own normalization rules. ConfigKit records their effects and provenance. Composition, inheritance, precedence, and source acquisition remain upstream responsibilities; this work adds no source-level validation or history-based domain enforcement.

Transformation Scope

Use the existing transformations as the migration inventory, rather than introducing general trimming or case-folding across the configuration:

Area	Existing behavior to move into recorded resolution
Backends	Normalize base endpoints and diagnostic endpoints; trim API-key environment names; trim and lowercase smoke-test authentication kinds.
Backend capacity	Apply the existing conditional queue-capacity default.
Profiles	Normalize configured nonempty endpoint overrides; trim service tier, reasoning effort, and API-key environment names.
Prompt metadata	Trim descriptions and session-ID templates.
Prompt inputs	Trim names and supplied content-type and description values.
Prompt output	Trim a supplied schema path.
Configured messages	Normalize roles and supplied cache-control type/TTL values using the shared domain rules.

Preserve the rules' existing distinctions. For example, roles are lowercased, but cache-control type normalization only trims; the move should not make new spellings acceptable. Preserve optional pointers and existing empty/null interpretation rather than creating fields merely to record an event.

The backend queue default currently applies when concurrency is enabled and the decoded queue-capacity pointer is nil, including explicit null. ConfigKit's Default helper skips explicit input, so substituting it mechanically would change behavior. Retain the existing condition and record the resulting assignment with RecordDerivedDefault, including its concurrency dependency.

Resource IDs, namespaces, and resource references keep their existing exact matching rules. Message content and content_file values are not new normalization targets. Schema paths continue to address the selected schema source, rather than acquiring a configuration-file-relative meaning.

Request overrides, profile completion from backend settings, and caller-appended messages still need their existing runtime processing. Share domain normalizers where appropriate; avoid moving request-dependent work into loading merely to make inspection look fully executable.

Provenance And Resource Handling

Record changed scalar fields at their exact paths with concise, value-free reasons. A role edit, for example, belongs at the role field within the message, not at the message or prompt list. Avoid recording unchanged values needlessly.

Keep the current origin-based lookup for unchanged content_file values, including values inherited or retained from another source. Normalizing a sibling role or cache-control field should not change which file is opened.

A normalized field acquires a resolution origin; its earlier source evidence is available for inspection. This does not promise that every normalized value retains a file-backed effective origin. If implementation discovers a transformed field that itself requires a source-relative resource lookup, preserve the relevant entry context explicitly before changing that field, using ConfigKit's public context APIs. Do not infer a runtime resource base from arbitrary history.

The planned normalizer does not need to reorder, add, or remove resource or message entries. That describes this implementation's scope, not a new restriction on operators' ConfigKit composition workflows.

Observable Changes

Configuration inspection will show normalized configuration values and recorded derived defaults. Explanations will expose the associated resolution operations and retained source evidence through ConfigKit's existing inspection behavior. This remains configuration inspection, not a rendering of a request after profile completion, variable substitution, or caller overrides.

Configuration diffs will compare these normalized effective values. Two inputs that differ only in a normalized spelling may no longer produce a value diff; list changes that remain are still reported atomically. Existing redaction continues to apply to inspection and explanations.

The intended execution behavior, accepted configuration values, resource selection, and presence/null handling remain the same. Preserve public error classification where applicable while integrating normalization failures with the existing validation path. Do not add a promise of identical diagnostic wording, ordering, or internal callback timing. No YAML syntax or collection specification-version change is planned.

Acceptance And Documentation

Adopt the released ConfigKit v1.1.0 module without a committed local replacement, with applicable attribution updated. The companion implementation plan owns sequencing and implementation steps.

Ordinary invalid resources should retain useful rejection rather than becoming unrecorded-mutation failures because a sibling field needed normalization. ConfigKit's conservative list audit does not justify new identity rules or a separate source audit.

Focused boundary coverage should demonstrate:

• Normalized effective values and explanations for representative backend, profile, prompt, and nested message fields at root and collection destinations.
• Relative content-file loading after sibling normalization, including inherited values and values retained from a different contributing source.
• Existing absence, explicit null, empty, and zero handling, especially the queue default and optional input/message metadata.
• Existing invalid-value and identity rejection through useful public errors, without broadening accepted values or adding new restrictions.
• Execution-relevant equivalence for normalized definitions, including prompt hashes, and continued normalization of caller-appended messages.
• Existing redaction, normalized-value diff behavior, and atomic list diffs.

Coverage should target PromptKit integration rather than reproduce ConfigKit's correspondence test suite. The format reference, consumer inspection guidance, source-loading documentation, affected GoDoc, and release notes should describe the implemented inspection change and its limits. Architecture wording should reflect the callback flow without turning implementation details into contracts.