322 lines
16 KiB
Markdown
322 lines
16 KiB
Markdown
# ConfigKit v1 Adoption And Identified Resource Collections
|
|
|
|
Status: accepted direction; implementation pending. This roadmap describes
|
|
future behavior, not the currently supported configuration format.
|
|
|
|
## Purpose
|
|
|
|
Adopt `go.maximumdirect.net/configkit` v1.0.0 and simplify authoring by using
|
|
identified lists consistently for prompts, profiles, and backends. An individual
|
|
resource file should begin with its own `id`, without a resource-name wrapper.
|
|
Collection entries can import those files beneath the appropriate resource list.
|
|
ConfigKit's sibling inheritance supplies concise variants, especially for
|
|
profiles and backends.
|
|
|
|
Prefer separately named variants when changing an existing definition. This is
|
|
authoring guidance, not a prohibition on deliberate same-ID replacement across
|
|
configuration layers. Keep ConfigKit responsible for composition and inheritance;
|
|
PromptKit interprets the final effective configuration and owns resource meaning.
|
|
|
|
## Current Baseline
|
|
|
|
The repository currently declares Go 1.26.0 and depends on
|
|
`gitea.maximumdirect.net/eric/ConfigKit` v0.8.0. The
|
|
[format reference](../formats.md) and [source adapter overview](../internal/sources.md)
|
|
describe the implemented behavior:
|
|
|
|
- Root `prompts`, `profiles`, and `backends` are mappings keyed by local ID.
|
|
`collections` is a namespace-keyed mapping whose entries contain the same
|
|
resource mappings and an optional `release_version` label.
|
|
- `CollectionInput` mounts a selected source at `collections/<namespace>` and
|
|
enables its imports. Namespace views qualify convenient runtime names.
|
|
- ConfigKit composes inputs and resolves mapping-sibling `extends` before
|
|
PromptKit validates, captures content, and constructs runtime indexes.
|
|
- Source-root `version: 1` or `version: "1"` is optional. The prompt `version`
|
|
field is a separate descriptive label still carried through public values.
|
|
- Public configuration and inspection APIs expose ConfigKit types. The module
|
|
identity change therefore affects downstream Go callers as well as internals.
|
|
|
|
A prior exploratory migration in a temporary copy passed the existing test
|
|
suite and maintained consumer examples against the tagged ConfigKit v1.0.0
|
|
checkout after mechanical API and control-block updates. It retained resource
|
|
mappings; it does not validate the list migration planned here.
|
|
|
|
## Targeted End State
|
|
|
|
### Dependency And Public API Migration
|
|
|
|
Use `go.maximumdirect.net/configkit` v1.0.0 throughout production code, tests,
|
|
examples, and current consumer guidance, and raise the Go consumer minimum to
|
|
1.27.0. Update build and release-toolchain guidance accordingly. PromptKit's own
|
|
module identity is outside this migration's scope.
|
|
|
|
Update PromptKit's existing uses of:
|
|
|
|
| Current use | Replacement |
|
|
| --- | --- |
|
|
| `Definition.Secret` | `Definition.Sensitive` |
|
|
| `Resolution.InputInfo` | `Resolution.EntryInfo` |
|
|
| `Resolution.Record(..., ChangeNormalization, ...)` | `Resolution.RecordNormalization(...)` |
|
|
|
|
Audit the remaining exposed inspection types and diagnostic consumers against
|
|
the complete [ConfigKit v1.0.0 migration notes](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.0.0/docs/releases/v1.0.0.md).
|
|
For example, version evidence fields were renamed, the main provenance layer is
|
|
now `main` rather than `base`, and the history-limit code changed. Do not require
|
|
identical historical rendering or source ordering where ConfigKit deliberately
|
|
changed those contracts or corrected attribution.
|
|
|
|
Coordinate consumers such as Scriptorium that pass `Source`, `Input`, `Underlay`,
|
|
`Path`, or inspection values across PromptKit's boundary. Old-path and new-path
|
|
ConfigKit types are different Go types; changing only a dependency requirement
|
|
does not bridge them. The released dependency must use the new module directly,
|
|
without a local replacement or an old-path compatibility adapter.
|
|
|
|
### Source Controls And Optional Headers
|
|
|
|
Migrate genuine source-root `configkit:` controls to `config:` in main files,
|
|
collection entries, fragments, and programmatic inputs. Controls are extracted
|
|
from each acquired document before placement. Nested `config` values are ordinary
|
|
application data subject to the existing application schema; placement does not
|
|
trigger another control interpretation pass. Do not globally rename arbitrary
|
|
data or the unchanged `configkit:"secret"` Go tag.
|
|
|
|
Also account for the intervening v0.9 import changes: imports are optional by
|
|
default, `optional` has been removed, and authors use `required: true` when a
|
|
missing import should fail. Preserve those upstream choices rather than imposing
|
|
a new PromptKit-wide requiredness policy. Directory imports can select filename
|
|
patterns and recurse; filesystem globs do not acquire recursive `**` semantics.
|
|
|
|
Advance the PromptKit configuration specification to version 2 for the new
|
|
resource shape. Accept the exact string `"2"` or strict decimal integer `2` when
|
|
a source declares a header; keep the existing if-present checking approach and
|
|
permit headerless files. Do not require a declaration in every resource file or
|
|
infer a missing header from an enclosing source. The header describes the
|
|
PromptKit specification, independently of ConfigKit's software version.
|
|
|
|
Remove the unused descriptive prompt `version` field and its propagation through
|
|
definitions, inventories, inspection, results, and observations. Remove the
|
|
corresponding `prompt_version` output fields and stale documentation. Retain the
|
|
collection `release_version` label and optional source specification headers;
|
|
these have distinct purposes. In a standalone prompt item file, a root `version`
|
|
is now solely a specification header; do not carry the former prompt label into
|
|
that position. Audit hashes and prepared snapshots affected by
|
|
the removed field without promising hash equality across this format migration.
|
|
|
|
### Resource Shape And Identity
|
|
|
|
Keep `collections` as a mapping keyed by namespace. Change each of `prompts`,
|
|
`profiles`, and `backends` to a list of mappings, both at the root and within a
|
|
collection. Each record supplies its local `id` explicitly. Filenames remain
|
|
organizational choices, not identities.
|
|
|
|
Preserve the existing nonblank, slash-free namespace and local-ID rules, exact
|
|
spelling, qualified `namespace/id` runtime identity, namespace views, and exact
|
|
resource references. A provider's `model` string remains provider data and can
|
|
contain `/`. Do not derive or rewrite stored backend/profile references from
|
|
the source's mount point. The same local ID can occur in different namespaces
|
|
or resource kinds.
|
|
|
|
Register `id` through `Definition.SequenceIdentities` for each root resource list
|
|
and each `collections` / any mapping key / resource-list destination. Use
|
|
`SequenceIdentityAt` and `SequenceIdentityMatching` with
|
|
`SequenceAnyMappingKey`; no source pre-scan or namespace enumeration is needed.
|
|
Authors do not need to repeat YAML identity declarations. ConfigKit owns
|
|
structural and final identity checks; PromptKit's schema and validation retain
|
|
responsibility for ID syntax, allowed value shapes, and domain semantics.
|
|
|
|
Build runtime indexes from the final validated records. Keep effective inspection
|
|
in the assembled list shape, including its index-based paths and provenance.
|
|
Do not impose a new execution-order meaning on list position or change the
|
|
existing inventory sorting solely because source resources become lists.
|
|
|
|
### Go Source-Builder Inputs
|
|
|
|
Add an `ID string` field to `ProfileConfig` and `BackendConfig`. Change
|
|
`ProfileSource` to accept `[]ProfileConfig` and `BackendSource` to accept
|
|
`[]BackendConfig`, replacing their keyed-map inputs. Emit each value as a record
|
|
with its explicit `id`, preserving the caller's slice order. Retain
|
|
`ProfileConfig.Extends` as a convenient parent-ID string and encode it as
|
|
`extends: {id: ...}`.
|
|
|
|
This makes Go and YAML resource definitions consistent and preserves duplicate
|
|
IDs when callers combine catalogs, allowing ConfigKit to reject them during
|
|
loading. The helpers continue encoding sources without independently validating
|
|
the final configuration. Preserving input order does not give it execution
|
|
significance. Callers that maintain keyed maps can convert them at this boundary.
|
|
|
|
Make this public API change as part of the coordinated pre-v1 migration rather
|
|
than retaining a second input representation for compatibility. Update callers,
|
|
examples, and migration guidance together.
|
|
|
|
### Inheritance And Layering
|
|
|
|
Identified siblings inherit using `extends: {id: parent-id}`. This applies to
|
|
backends and profiles as well as prompts; no additional PromptKit inheritance
|
|
engine or prompt-specific restriction is needed. Lookup stays within the same
|
|
resource list. Ordinary qualified references to other resources are separate
|
|
from this sibling relationship.
|
|
|
|
ConfigKit resolves surviving parents before children, retains the child's ID,
|
|
and applies its normal mapping and presence rules to inherited settings.
|
|
Missing parents and cycles follow upstream diagnostics. Parent records remain
|
|
ordinary definitions subject to normal validation, not automatically hidden or
|
|
abstract templates. Preserve PromptKit's existing allowance for partial profiles
|
|
that requests can complete; inheritance does not add a readiness requirement.
|
|
|
|
Within one strict tier, duplicate IDs fail. Across precedence tiers, the default
|
|
identified-list operation is upsert: a matching record is replaced in full.
|
|
Authors can select ConfigKit's append or replace modes. Under upsert or append,
|
|
`[]` contributes no records; explicit replacement with `[]` clears a list.
|
|
An ID cannot inherit its displaced lower-layer version by extending itself.
|
|
|
|
Recommend a distinct variant ID with `extends` for customization. Preserve
|
|
deliberate same-ID whole-record replacement and upstream collection operations.
|
|
Do not add a partial-record merge, ban inheritance for prompts, require new IDs
|
|
for every modification, or add constraints on source documents sharing a final
|
|
namespace. Shared prompt text can continue using `content_file`.
|
|
|
|
## Illustrative Authoring Workflow
|
|
|
|
These snippets describe the target format. They are not complete deployment
|
|
files and do not assert provider model availability.
|
|
|
|
A caller mounts an entry document under `weatherreporter` using `CollectionInput`.
|
|
The entry can declare the specification once and import headerless resources:
|
|
|
|
```yaml
|
|
version: 2
|
|
config:
|
|
imports:
|
|
- dir: profiles
|
|
pattern: ["*.yaml", "*.yml"]
|
|
recursive: true
|
|
destination: [profiles]
|
|
placement: sequence-item
|
|
- dir: prompts
|
|
pattern: ["*.yaml", "*.yml"]
|
|
recursive: true
|
|
destination: [prompts]
|
|
placement: sequence-item
|
|
```
|
|
|
|
One profile file can contain a base definition:
|
|
|
|
```yaml
|
|
id: sonnet
|
|
backend: openrouter/api
|
|
model: example-model
|
|
temperature: 0.7
|
|
```
|
|
|
|
Another profile file contains its variant:
|
|
|
|
```yaml
|
|
id: sonnet-precise
|
|
extends: {id: sonnet}
|
|
temperature: 0.2
|
|
```
|
|
|
|
A prompt file can select it by qualified ID:
|
|
|
|
```yaml
|
|
id: foo.bar
|
|
default_profile: weatherreporter/sonnet-precise
|
|
messages:
|
|
- role: user
|
|
content: Describe the weather.
|
|
output:
|
|
format: text
|
|
validation_mode: none
|
|
```
|
|
|
|
The caller separately supplies `openrouter/api`. A backend collection uses the
|
|
same item-import pattern with `destination: [backends]` and can define backend
|
|
variants using sibling `extends`. Inline resource lists are equally valid;
|
|
one-file-per-item and directory layout are conveniences, not requirements.
|
|
|
|
## Work And Sequencing
|
|
|
|
### 1. Establish The ConfigKit v1 Baseline
|
|
|
|
Migrate the module, toolchain, affected APIs, and source controls while temporarily
|
|
retaining resource mappings. Update fixtures and examples and validate that
|
|
configuration loading, presence, inheritance, inspection, and execution retain
|
|
their intended behavior. This isolates dependency migration failures from the
|
|
later shape change; the intermediate state need not be separately released.
|
|
|
|
Review module sums and the resolved dependency graph, and update
|
|
[attribution](../oss/attribution.md) to the selected upstream revision and notices.
|
|
ConfigKit's own release-tooling migration does not by itself require changing
|
|
PromptKit's release tooling or publication workflow.
|
|
|
|
### 2. Convert The Configuration Adapter And Public Values
|
|
|
|
Change root and collection resource decoding to identified records, register all
|
|
six root/dynamic identity rules, and advance the optional specification checker.
|
|
Adapt validation and runtime indexing to read IDs from records. Remove prompt
|
|
version metadata throughout the relevant public and internal values.
|
|
|
|
Review every path-dependent operation as mapping keys become list indexes:
|
|
presence capture, profile/backend defaults, prompt input metadata, sensitive
|
|
provider fields, errors, and effective resource lookup. Resolve `content_file`
|
|
against the effective field's contributing source after inheritance and upsert.
|
|
Retain one ConfigKit load and the existing effective-value handoff; source history
|
|
must not become a second resolver.
|
|
|
|
Update `BackendSource`, `ProfileSource`, and their public configuration values to
|
|
use explicit-ID slices and encode profile inheritance as an identity reference.
|
|
Update namespace views, listing, inspection, prepared execution, and smoke-test
|
|
selection as needed to consume the same captured runtime indexes. Do not change
|
|
their selection or execution semantics merely to mirror the new configuration
|
|
representation.
|
|
|
|
### 3. Exercise The Integrated Target
|
|
|
|
Use focused behavioral coverage for the changed boundaries:
|
|
|
|
- Root and arbitrarily named collections load inline lists and imported items;
|
|
equal local IDs in separate namespaces remain independent.
|
|
- Duplicate IDs within a resource list fail, including final values introduced
|
|
through supported application resolution. Exercise the registrations through
|
|
PromptKit without duplicating ConfigKit's complete generic test suite.
|
|
- Profile and backend variants inherit settings while preserving child identity,
|
|
explicit zero/false/null behavior, and ordinary domain validation.
|
|
- Layered upsert replaces a whole record; explicit list replacement and empty
|
|
contributions follow ConfigKit semantics.
|
|
- Inherited and replaced prompt resources retain the correct source context;
|
|
provider credentials remain redacted at the new effective paths.
|
|
- Optional version-2 headers and headerless fragments load; unsupported declared
|
|
versions and obsolete resource shapes fail through their owning validators.
|
|
- Public source builders, namespace lookup, inspection, preparation, validation,
|
|
smoke tests, and execution still work with the converted catalogs.
|
|
- Source builders preserve supplied record order and duplicate IDs for loading;
|
|
profile parent-ID strings produce the expected sibling inheritance references.
|
|
|
|
Run the existing suite, relevant vet checks, and maintained offline consumer
|
|
examples under Go 1.27. Tests require no live provider calls. Reuse existing
|
|
coverage where it already proves the behavior; do not turn incidental history
|
|
formatting or list positions into stronger guarantees.
|
|
|
|
### 4. Publish One Coherent Migration
|
|
|
|
Update the [format reference](../formats.md), [consumer migration guide](../consumers/configuration-migration.md),
|
|
[Go consumer guide](../consumers/pkg-promptkit.md), source-adapter documentation,
|
|
GoDoc, and maintained examples as implementation lands. Explain the module/type
|
|
change, Go minimum, `config:` controls, import requiredness, resource lists,
|
|
the source builders' map-to-slice conversion and explicit `ID` fields,
|
|
mapping-to-record `extends` conversion, removed prompt labels, specification
|
|
headers, and index-based inspection paths together.
|
|
|
|
Plan a breaking pre-v1 minor release under the existing
|
|
[release policy](../policy/release.md). Prefer delivering both implementation
|
|
stages together so downstream applications and collection authors migrate once.
|
|
Keep historical release notes intact. Coordinate Scriptorium's dependency and
|
|
configuration updates without making its release or internal implementation a
|
|
PromptKit dependency.
|
|
|
|
The target format replaces the old resource mappings; this scope does not add
|
|
dual-format decoding, source rewriting, catalog discovery conventions, or a
|
|
compatibility module. Preserve operator choices already supplied by ConfigKit.
|
|
Once implementation is complete, move durable behavior into its canonical
|
|
owners and retire this temporary roadmap under repository practice.
|