promptkit/docs/roadmap/configkit.md

# ConfigKit v1 Adoption And Identified Resource Collections

Status: complete. The implemented result includes the agreed execution-owned
normalization adjustment for ConfigKit v1.0.0. Current consumer documentation
remains authoritative for supported behavior.

The completed implementation plan is preserved in Git history at `79dfd97`:
`git show 79dfd97:docs/roadmap/implementation.md`.

## Purpose

Adopt `go.maximumdirect.net/configkit` v1.0.0 and simplify authoring by using
identified lists consistently for prompts, profiles, and backends. An individual
resource file should begin with its own `id`, without a resource-name wrapper.
Collection entries can import those files beneath the appropriate resource list.
ConfigKit's sibling inheritance supplies concise variants, especially for
profiles and backends.

Prefer separately named variants when changing an existing definition. This is
authoring guidance, not a prohibition on deliberate same-ID replacement across
configuration layers. Keep ConfigKit responsible for composition and inheritance;
PromptKit interprets the final effective configuration and owns resource meaning.

## Pre-Migration Baseline

Before this migration, the repository declared Go 1.26.0 and depended on
`gitea.maximumdirect.net/eric/ConfigKit` v0.8.0. The following describes that
historical baseline; the current
[format reference](../formats.md) and [source adapter overview](../internal/sources.md)
describe the migrated behavior:

- Root `prompts`, `profiles`, and `backends` are mappings keyed by local ID.
  `collections` is a namespace-keyed mapping whose entries contain the same
  resource mappings and an optional `release_version` label.
- `CollectionInput` mounts a selected source at `collections/<namespace>` and
  enables its imports. Namespace views qualify convenient runtime names.
- ConfigKit composes inputs and resolves mapping-sibling `extends` before
  PromptKit validates, captures content, and constructs runtime indexes.
- Source-root `version: 1` or `version: "1"` is optional. The prompt `version`
  field is a separate descriptive label still carried through public values.
- Public configuration and inspection APIs expose ConfigKit types. The module
  identity change therefore affects downstream Go callers as well as internals.

A prior exploratory migration in a temporary copy passed the existing test
suite and maintained consumer examples against the tagged ConfigKit v1.0.0
checkout after mechanical API and control-block updates. It retained resource
mappings; it does not validate the list migration planned here.

## Targeted End State

### Dependency And Public API Migration

Use `go.maximumdirect.net/configkit` v1.0.0 throughout production code, tests,
examples, and current consumer guidance, and raise the Go consumer minimum to
1.27.0. Update build and release-toolchain guidance accordingly. PromptKit's own
module identity is outside this migration's scope.

Update PromptKit's existing uses of:

| Current use | Replacement |
| --- | --- |
| `Definition.Secret` | `Definition.Sensitive` |
| `Resolution.InputInfo` | `Resolution.EntryInfo` |

Migrate recording API names where still used; PromptKit's normalization and
defaults follow the execution-owned interpretation described below rather than
requiring ConfigKit resolution records for fields within resource lists.

Audit the remaining exposed inspection types and diagnostic consumers against
the complete [ConfigKit v1.0.0 migration notes](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.0.0/docs/releases/v1.0.0.md).
For example, version evidence fields were renamed, the main provenance layer is
now `main` rather than `base`, and the history-limit code changed. Do not require
identical historical rendering or source ordering where ConfigKit deliberately
changed those contracts or corrected attribution.

Coordinate consumers such as Scriptorium that pass `Source`, `Input`, `Underlay`,
`Path`, or inspection values across PromptKit's boundary. Old-path and new-path
ConfigKit types are different Go types; changing only a dependency requirement
does not bridge them. The released dependency must use the new module directly,
without a local replacement or an old-path compatibility adapter.

### Source Controls And Optional Headers

Migrate genuine source-root `configkit:` controls to `config:` in main files,
collection entries, fragments, and programmatic inputs. Controls are extracted
from each acquired document before placement. Nested `config` values are ordinary
application data subject to the existing application schema; placement does not
trigger another control interpretation pass. Do not globally rename arbitrary
data or the unchanged `configkit:"secret"` Go tag.

Also account for the intervening v0.9 import changes: imports are optional by
default, `optional` has been removed, and authors use `required: true` when a
missing import should fail. Preserve those upstream choices rather than imposing
a new PromptKit-wide requiredness policy. Directory imports can select filename
patterns and recurse; filesystem globs do not acquire recursive `**` semantics.

Advance the PromptKit configuration specification to version 2 for the new
resource shape. Accept the exact string `"2"` or strict decimal integer `2` when
a source declares a header; keep the existing if-present checking approach and
permit headerless files. Do not require a declaration in every resource file or
infer a missing header from an enclosing source. The header describes the
PromptKit specification, independently of ConfigKit's software version.

Remove the unused descriptive prompt `version` field and its propagation through
definitions, inventories, inspection, results, and observations. Remove the
corresponding `prompt_version` output fields and stale documentation. Retain the
collection `release_version` label and optional source specification headers;
these have distinct purposes. In a standalone prompt item file, a root `version`
is now solely a specification header; do not carry the former prompt label into
that position. Audit hashes and prepared snapshots affected by
the removed field without promising hash equality across this format migration.

### Resource Shape And Identity

Keep `collections` as a mapping keyed by namespace. Change each of `prompts`,
`profiles`, and `backends` to a list of mappings, both at the root and within a
collection. Each record supplies its local `id` explicitly. Filenames remain
organizational choices, not identities.

Preserve the existing nonblank, slash-free namespace and local-ID rules, exact
spelling, qualified `namespace/id` runtime identity, namespace views, and exact
resource references. A provider's `model` string remains provider data and can
contain `/`. Do not derive or rewrite stored backend/profile references from
the source's mount point. The same local ID can occur in different namespaces
or resource kinds.

Register `id` through `Definition.SequenceIdentities` for each root resource list
and each `collections` / any mapping key / resource-list destination. Use
`SequenceIdentityAt` and `SequenceIdentityMatching` with
`SequenceAnyMappingKey`; no source pre-scan or namespace enumeration is needed.
Authors do not need to repeat YAML identity declarations. ConfigKit owns
structural and final identity checks; PromptKit's schema and validation retain
responsibility for ID syntax, allowed value shapes, and domain semantics.

Build runtime indexes from the final validated records. Keep effective inspection
in the assembled list shape, including its index-based paths and provenance.
Do not impose a new execution-order meaning on list position or change the
existing inventory sorting solely because source resources become lists.

### Composed Configuration And Execution Interpretation

Accommodate ConfigKit v1.0.0's atomic sequence-mutation auditing. Retain the
composed typed configuration, field presence, and provenance supplied by
ConfigKit. Apply PromptKit's domain normalization and defaults to owned
interpretations of those final records during engine construction, preserving
existing execution behavior and domain validation.

Configuration inspection, explanations, and diffs describe ConfigKit's composed
values and provenance. They do not include PromptKit-side trimming, endpoint
normalization, or execution defaults as ConfigKit resolution history. Existing
prompt/profile inspection and preparation expose their respective interpreted
metadata or effective execution settings. These views have distinct purposes;
an authored whitespace difference can remain visible in a configuration diff
even when it has no effect on the interpreted setting.

Interpretation preserves exact identities, references, presence/null semantics,
and the association between records and effective index paths. Relative resources
such as `content_file` resolve through the contributing field's ConfigKit origin
and source context. Normalization does not replace that attribution with an
application-generated whole-list origin.

This remains interpretation of the final handoff, not source prevalidation or
another composition pass. It requires no upstream change or new public API.
Field-granular normalization within ConfigKit itself is a separate
[future idea](future.md#field-level-normalization-with-preserved-list-provenance)
to reevaluate after upstream support is available.

### Go Source-Builder Inputs

Add an `ID string` field to `ProfileConfig` and `BackendConfig`. Change
`ProfileSource` to accept `[]ProfileConfig` and `BackendSource` to accept
`[]BackendConfig`, replacing their keyed-map inputs. Emit each value as a record
with its explicit `id`, preserving the caller's slice order. Retain
`ProfileConfig.Extends` as a convenient parent-ID string and encode it as
`extends: {id: ...}`.

This makes Go and YAML resource definitions consistent and preserves duplicate
IDs when callers combine catalogs, allowing ConfigKit to reject them during
loading. The helpers continue encoding sources without independently validating
the final configuration. Preserving input order does not give it execution
significance. Callers that maintain keyed maps can convert them at this boundary.

Make this public API change as part of the coordinated pre-v1 migration rather
than retaining a second input representation for compatibility. Update callers,
examples, and migration guidance together.

### Inheritance And Layering

Identified siblings inherit using `extends: {id: parent-id}`. This applies to
backends and profiles as well as prompts; no additional PromptKit inheritance
engine or prompt-specific restriction is needed. Lookup stays within the same
resource list. Ordinary qualified references to other resources are separate
from this sibling relationship.

ConfigKit resolves surviving parents before children, retains the child's ID,
and applies its normal mapping and presence rules to inherited settings.
Missing parents and cycles follow upstream diagnostics. Parent records remain
ordinary definitions subject to normal validation, not automatically hidden or
abstract templates. Preserve PromptKit's existing allowance for partial profiles
that requests can complete; inheritance does not add a readiness requirement.

Within one strict tier, duplicate IDs fail. Across precedence tiers, the default
identified-list operation is upsert: a matching record is replaced in full.
Authors can select ConfigKit's append or replace modes. Under upsert or append,
`[]` contributes no records; explicit replacement with `[]` clears a list.
An ID cannot inherit its displaced lower-layer version by extending itself.

Recommend a distinct variant ID with `extends` for customization. Preserve
deliberate same-ID whole-record replacement and upstream collection operations.
Do not add a partial-record merge, ban inheritance for prompts, require new IDs
for every modification, or add constraints on source documents sharing a final
namespace. Shared prompt text can continue using `content_file`.

## Illustrative Authoring Workflow

These snippets describe the target format. They are not complete deployment
files and do not assert provider model availability.

A caller mounts an entry document under `weatherreporter` using `CollectionInput`.
The entry can declare the specification once and import headerless resources:

```yaml
version: 2
config:
  imports:
    - dir: profiles
      pattern: ["*.yaml", "*.yml"]
      recursive: true
      destination: [profiles]
      placement: sequence-item
    - dir: prompts
      pattern: ["*.yaml", "*.yml"]
      recursive: true
      destination: [prompts]
      placement: sequence-item
```

One profile file can contain a base definition:

```yaml
id: sonnet
backend: openrouter/api
model: example-model
temperature: 0.7
```

Another profile file contains its variant:

```yaml
id: sonnet-precise
extends: {id: sonnet}
temperature: 0.2
```

A prompt file can select it by qualified ID:

```yaml
id: foo.bar
default_profile: weatherreporter/sonnet-precise
messages:
  - role: user
    content: Describe the weather.
output:
  format: text
  validation_mode: none
```

The caller separately supplies `openrouter/api`. A backend collection uses the
same item-import pattern with `destination: [backends]` and can define backend
variants using sibling `extends`. Inline resource lists are equally valid;
one-file-per-item and directory layout are conveniences, not requirements.

## Acceptance And Compatibility

The integrated target supports inline lists and imported items at the root and
in arbitrarily named collections. Equal local IDs in separate namespaces remain
independent. Registered identities govern strict composition, inheritance,
layering, and final typed values through ConfigKit. PromptKit retains its domain
validation, presence-aware settings, partial profiles, and exact runtime lookup.
Domain checks use PromptKit's interpreted values without mutating ConfigKit's
retained configuration or replaying source history.

Inherited and replaced prompt fields retain their contributing resource context.
Sensitive provider values remain protected in inspection at index-based paths.
Source builders preserve supplied record order and duplicate IDs for loading.
Namespaces, discovery, inspection, preparation, concrete-request validation,
smoke diagnostics, and execution consume the same captured resource indexes.
These behaviors have focused offline coverage without duplicating ConfigKit's
generic suite or promising incidental history formatting or list positions.

The [format reference](../formats.md), [consumer migration guide](../consumers/configuration-migration.md),
[Go consumer guide](../consumers/pkg-promptkit.md), source-adapter documentation,
GoDoc, and maintained examples describe the integrated target when it lands.
Migration guidance covers the new module/types, Go minimum, controls, import
requiredness, identified lists, explicit-ID source builders, sibling inheritance,
removed prompt labels, specification headers, and index-based inspection paths.
It also explains composed configuration inspection versus PromptKit's normalized
metadata and execution settings, including the absence of ConfigKit history for
PromptKit-side normalization/defaults.
Attribution identifies the selected ConfigKit revision and applicable notices.

Deliver this as one coordinated breaking pre-v1 minor upgrade under the existing
[release policy](../policy/release.md), so applications and collection authors
migrate once. Historical release notes remain intact. Scriptorium coordination
does not make its release or internal implementation a PromptKit dependency.
ConfigKit's own release-tooling migration does not require a change to
PromptKit's release tooling or publication workflow.

The target replaces the old resource mappings without adding dual-format
decoding, source rewriting, catalog discovery conventions, or a compatibility
module. Intermediate implementation states need not compile or be coherent;
the companion plan targets one end-to-end sprint without intermediate human
review or external publication gates. Durable behavior belongs in its canonical
owners when complete; this roadmap remains temporary coordination material.