promptkit/docs/roadmap/field-normalization.md

# Field-Level Configuration Normalization

## Status And Purpose

Implemented: PromptKit uses ConfigKit v1.1.0 and records the selected field
normalizations and queue default in its resolved configuration.

Move PromptKit's existing configuration-time normalization and derived defaults
into ConfigKit's recorded application resolution. Configuration inspection should
then show the normalized values used to construct execution settings, with
explanations of the changes, while unchanged resource references retain the
source context needed to read their files.

The benefit is a more useful inspection view and removal of the parallel
interpreted configuration maintained for execution. This is a focused loading
change, not a new configuration format, normalization policy, or execution API.

## Historical Baseline

PromptKit already uses identified lists for backends, profiles, and prompts at
the root and within named collections. Six ConfigKit identity registrations
cover those destinations. This roadmap does not repeat that migration.

At the baseline, the configuration adapter's `Definition.Resolve` captured input
presence information. During validation, `interpretConfiguration` constructs
detached backend, profile, and prompt interpretations and applies normalization
and a conditional backend queue default. Runtime construction uses those
interpretations; configuration inspection retains ConfigKit's composed values.
Configured message roles and cache-control values are normalized later during
prompt materialization.

This separation avoided recording whole resource lists as normalized under
ConfigKit v1.0.0. Such a record would replace the effective origins of unchanged
fields, including `content_file`, whose origin selects the filesystem and
directory used to load message text.

The implemented behavior is documented in the [format reference](../formats.md) and
[source-loading implementation](../internal/sources.md). The
[configuration ownership decision](../adr/0001-configuration-boundary.md)
continues to govern the division of responsibility: ConfigKit owns composition;
PromptKit interprets effective values and supplies domain rules.

## Released Upstream Capability

[ConfigKit v1.1.0](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.1.0/docs/releases/v1.1.0.md)
allows individual field records inside lists through the existing `Normalize`,
`RecordNormalization`, and derived-default APIs:

- Equal-length ordinary lists are compared recursively by position. This covers
  nested prompt messages and inputs without registering new identities for them.
- Registered collections with valid, unique entry and final identities support
  matched field comparison. PromptKit already registers its resource lists.
- List length changes, and registered identity order or membership changes,
  still need list or ancestor coverage. Matching is conservative when identities
  are unsuitable. Final identity validation remains ConfigKit's responsibility.
- Entry queries retain their fixed resolution-entry meaning; records address
  final positions. Public effective-configuration diffs remain atomic for lists.

These capabilities support the proposed in-place field transformations. No
additional ConfigKit API is a prerequisite. Historical source evidence retained
for a matched record is distinct from its effective origin: a whole-list
resolution record can still replace descendant effective origins. The migration
should use individual field records for individual changes.

## Target End State

### One Normalized Configuration For Loading And Inspection

PromptKit's `Definition.Resolve` captures the entry metadata needed for existing
presence/null semantics, then applies configuration-time transformations to the
working configuration and records their effects at the affected fields.
`Definition.Validate` examines the resulting values without mutating them.
Runtime indexes and prompt materialization consume that same normalized
configuration, producing the execution-owned domain values they need.

Remove `state.interpreted` and the detached interpretation copies once their
normalization work has moved. Retain copies needed for runtime ownership and
public detached results; removing the duplicate configuration is not a reason
to expose shared mutable runtime state.

PromptKit continues to own normalization rules. ConfigKit records their effects
and provenance. Composition, inheritance, precedence, and source acquisition
remain upstream responsibilities; this work adds no source-level validation or
history-based domain enforcement.

### Transformation Scope

Use the existing transformations as the migration inventory, rather than
introducing general trimming or case-folding across the configuration:

| Area | Existing behavior to move into recorded resolution |
| --- | --- |
| Backends | Normalize base endpoints and diagnostic endpoints; trim API-key environment names; trim and lowercase smoke-test authentication kinds. |
| Backend capacity | Apply the existing conditional queue-capacity default. |
| Profiles | Normalize configured nonempty endpoint overrides; trim service tier, reasoning effort, and API-key environment names. |
| Prompt metadata | Trim descriptions and session-ID templates. |
| Prompt inputs | Trim names and supplied content-type and description values. |
| Prompt output | Trim a supplied schema path. |
| Configured messages | Normalize roles and supplied cache-control type/TTL values using the shared domain rules. |

Preserve the rules' existing distinctions. For example, roles are lowercased,
but cache-control type normalization only trims; the move should not make new
spellings acceptable. Preserve optional pointers and existing empty/null
interpretation rather than creating fields merely to record an event.

The backend queue default currently applies when concurrency is enabled and
the decoded queue-capacity pointer is nil, including explicit null. ConfigKit's
`Default` helper skips explicit input, so substituting it mechanically would
change behavior. Retain the existing condition and record the resulting
assignment with `RecordDerivedDefault`, including its concurrency dependency.

Resource IDs, namespaces, and resource references keep their existing exact
matching rules. Message content and `content_file` values are not new
normalization targets. Schema paths continue to address the selected schema
source, rather than acquiring a configuration-file-relative meaning.

Request overrides, profile completion from backend settings, and caller-appended
messages still need their existing runtime processing. Share domain normalizers
where appropriate; avoid moving request-dependent work into loading merely to
make inspection look fully executable.

### Provenance And Resource Handling

Record changed scalar fields at their exact paths with concise, value-free
reasons. A role edit, for example, belongs at the role field within the message,
not at the message or prompt list. Avoid recording unchanged values needlessly.

Keep the current origin-based lookup for unchanged `content_file` values,
including values inherited or retained from another source. Normalizing a sibling
role or cache-control field should not change which file is opened.

A normalized field acquires a resolution origin; its earlier source evidence is
available for inspection. This does not promise that every normalized value
retains a file-backed effective origin. If implementation discovers a transformed
field that itself requires a source-relative resource lookup, preserve the
relevant entry context explicitly before changing that field, using ConfigKit's
public context APIs. Do not infer a runtime resource base from arbitrary history.

The planned normalizer does not need to reorder, add, or remove resource or
message entries. That describes this implementation's scope, not a new
restriction on operators' ConfigKit composition workflows.

### Observable Changes

Configuration inspection will show normalized configuration values and recorded
derived defaults. Explanations will expose the associated resolution operations
and retained source evidence through ConfigKit's existing inspection behavior.
This remains configuration inspection, not a rendering of a request after
profile completion, variable substitution, or caller overrides.

Configuration diffs will compare these normalized effective values. Two inputs
that differ only in a normalized spelling may no longer produce a value diff;
list changes that remain are still reported atomically. Existing redaction
continues to apply to inspection and explanations.

The intended execution behavior, accepted configuration values, resource
selection, and presence/null handling remain the same. Preserve public error
classification where applicable while integrating normalization failures with
the existing validation path. Do not add a promise of identical diagnostic
wording, ordering, or internal callback timing. No YAML syntax or collection
specification-version change is planned.

## Acceptance And Documentation

Adopt the released ConfigKit v1.1.0 module without a committed local replacement,
with applicable attribution updated. The companion
[implementation plan](implementation.md) owns sequencing and implementation steps.

Ordinary invalid resources should retain useful rejection rather than becoming
unrecorded-mutation failures because a sibling field needed normalization.
ConfigKit's conservative list audit does not justify new identity rules or a
separate source audit.

Focused boundary coverage should demonstrate:

- Normalized effective values and explanations for representative backend,
  profile, prompt, and nested message fields at root and collection destinations.
- Relative content-file loading after sibling normalization, including inherited
  values and values retained from a different contributing source.
- Existing absence, explicit null, empty, and zero handling, especially the queue
  default and optional input/message metadata.
- Existing invalid-value and identity rejection through useful public errors,
  without broadening accepted values or adding new restrictions.
- Execution-relevant equivalence for normalized definitions, including prompt
  hashes, and continued normalization of caller-appended messages.
- Existing redaction, normalized-value diff behavior, and atomic list diffs.

Coverage should target PromptKit integration rather than reproduce ConfigKit's
correspondence test suite. The format reference, consumer inspection guidance,
source-loading documentation, affected GoDoc, and release notes should describe
the implemented inspection change and its limits. Architecture wording should
reflect the callback flow without turning implementation details into contracts.