promptkit/docs/roadmap/namespaces.md

# Namespaces, Collections, And Scoped Resource Lookup

Status: feature roadmap; not implemented. The reserved-slash naming convention,
reference-resolution rules, decisions 1–3 (scope shape, validation, and integrated
discovery filtering), and decision 4's move to application/operator-supplied
configuration are accepted target behavior. Decision 5 selects mounted collections
and optional source-version checking. Its six configuration decisions are
accepted in the collection roadmap, subject to the ConfigKit boundary. Exact Go
names and option types remain delivery details.

## Purpose

Simplifying configuration is a primary goal of this round: YAML collections
replace mandatory catalog Go-module imports, unused prompt-version selection is
removed, and specification declarations are optional. Publishers may declare a
specification in an entry document without adding headers to every file.
Conventions should ease authoring without prescribing consumer layout or
precedence. The [collection roadmap](configuration-collections-mounted-fragments.md)
owns the configuration decisions and delivery details.

Let consumers select a namespace once and use convenient local names within
that namespace across PromptKit operations. An assembled engine may contain
multiple application prompt libraries and provider profile catalogs with
overlapping local names. Resolution must remain unambiguous.

The motivating cases are:

- WeatherReporter and another application both define `summarize.alerts`.
  Each consumer selects its application namespace and uses the local name.
- OpenRouter and RakestrawHome both provide the model identified in the
  discussion as `google/gemma-4-31b-it`. Consumers distinguish the execution
  profiles by provider namespace while retaining convenient local naming.

Model names here illustrate identity and resolution, not provider availability
or a promise of maintained catalog entries.

The primary need is organization and convenient lookup within assembled
libraries, with backend, profile, and prompt collections managed independently
of PromptKit software releases. Ansible collections are a conceptual reference
for qualified names; a package manager and relocatable collection imports are
not required for this feature.

## Existing Foundation

The [format reference](../formats.md) defines prompt, profile, and backend keys
as exact literal IDs. Punctuation is already allowed, and references are not
trimmed or re-keyed. Merely adopting prefixed IDs therefore does not provide
the requested scoped lookup behavior.

Existing prompt and profile listings expose assembled identities. The
[consumer guide](../consumers/pkg-promptkit.md) describes current operations;
their Go declarations and GoDoc remain authoritative until this feature lands.

Directory-based and embedded configuration loading already exist through
`DirectoryInputs` and `FilesystemInputs`. The current engine also prepends
configuration assets from the imported OpenRouter and RakestrawHome Go modules,
plus PromptKit-owned profiles. Decision 4 removes that automatic inclusion and
dependency coupling; it does not require a replacement for ConfigKit loading.

[ADR 0001](../adr/0001-configuration-boundary.md) assigns source loading,
composition, and inheritance to ConfigKit. This proposal adds interpretation
of consumer-supplied resource names against the final effective configuration,
without a second source-composition pass.

## Accepted Identity Convention

Reserve `/` exclusively as the namespace separator in PromptKit prompt, profile,
and backend identifiers and references. A qualified identifier has exactly two
nonempty components: `namespace/local-name`. Neither component can contain `/`.
Dots remain available within local names. An unqualified identifier has no slash
and remains valid under the existing nonblank-ID rule.

Reject resource identifiers or supplied references with multiple slashes or an
empty qualified component, such as `a/b/c`, `/name`, and `namespace/`. These
rules do not apply to provider model values, endpoints, resource paths, or prompt
versions. Preserve exact spelling; do not silently trim or rewrite IDs.

| Resource | Namespace | Local name | Qualified ID |
| --- | --- | --- | --- |
| Prompt | `weatherreporter` | `summarize.alerts` | `weatherreporter/summarize.alerts` |
| Prompt | `homewatch` | `summarize.alerts` | `homewatch/summarize.alerts` |
| Profile | `openrouter` | `google.gemma-4-31b-it` | `openrouter/google.gemma-4-31b-it` |
| Profile | `rakestrawhome` | `google.gemma-4-31b-it` | `rakestrawhome/google.gemma-4-31b-it` |

Prompt, profile, and backend identities remain separate resource types. The
same namespace can contain all three, and resources in it may come from
multiple configuration sources. A namespace does not imply a directory,
package, provider model identifier, or security boundary.

A profile ID identifies execution settings. Its configured model value is
still the identifier passed to the provider; qualifying the profile ID does
not modify that value. Both example profiles may retain the configured model
`google/gemma-4-31b-it`. Different profiles may select the same model with
different settings. Prompt version labels remain separate from namespaces. The
selected collection format installs one definition per prompt ID and removes
request-side version selection and assertions. Optional labels are descriptive.

Retain unqualified and dotted IDs. Existing slash-containing IDs need review:
those with one slash and nonempty components become qualified identities, while
those outside the new grammar require migration. This intentional format break
is accepted in favor of unambiguous resolution; the migration work is described
below.

## Accepted Consumer Scope

Provide a lightweight, immutable view over an existing engine with independent
prompt, profile, and backend scopes. The following sketch illustrates the
accepted shape; method/type names and final signatures remain provisional:

```go
weather, err := engine.WithNamespaces(Namespaces{
    Prompt:  "weatherreporter",
    Profile: "openrouter",
})
// Handle err, then use local prompt and profile names through weather.
```

This sketch follows [decision 1](#1-public-scope-shape-accepted); these
declarations do not exist yet.

The view would share the engine's captured configuration, clients, and execution
machinery. Selecting a namespace would not rebuild the engine or duplicate its
backend capacity accounting. Consumers could use different views concurrently
without changing a shared current-namespace setting.

The accepted resolution rules, independent of the final scope API, are:

1. A supplied name without a slash is qualified using the selected scope for its
   resource type, then looked up exactly. Without that scope, it is looked up as
   an exact unqualified ID.
2. A missing scoped resource does not trigger a search in other namespaces or
   among unqualified IDs. Loading another collection cannot redirect lookup.
3. A valid supplied name containing one slash is fully qualified and is looked
   up exactly, regardless of scope. Malformed resource references are rejected.
4. Scope applies to caller-supplied resource names. Stored references, including
   prompt default profiles, profile backends, inheritance references, and backend
   smoke-test default profiles, retain their existing exact meaning. They are
   not rebound to the invoking consumer's namespace.
5. Omitted optional IDs retain their existing selection/default behavior; an
   omission is not converted into a namespace prefix or a lookup for an empty
   local name.
6. Where outputs, inspection, errors, or diagnostic reports identify a selected
   resource, preserve its resolved full identity. Any exposed local name should
   be distinguishable from that identity.

Keep qualification consistent across execution, preparation, request validation,
prompt/profile inspection, and applicable smoke-test selections. Namespace
selection must not change request override precedence or provider model strings.
Prompts resolve by ID alone.

### Local And Qualified References In One Operation

Syntax alone distinguishes local and qualified references. Do not decide whether
to qualify a name by checking which candidates exist; that would make resolution
depend on loaded content.

Within a `weatherreporter` prompt scope, `summarize.alerts` resolves only to
`weatherreporter/summarize.alerts`, while `homewatch/summarize.alerts` explicitly
selects that other library's prompt. A qualified profile override similarly
bypasses the profile scope. Scoped operations accept both forms without a
separate reference type or requiring a return to the original engine for
cross-namespace access.

The original engine remains the way to select an exact unqualified ID without
applying a scope. No escape syntax for unqualified IDs is introduced here.

### Separate Resource Scopes

A consumer can select prompt shorthand under `weatherreporter` and profile
shorthand under `openrouter` in the same request. Each resource type has an
independent scope, as defined in [decision 1](#1-public-scope-shape-accepted).
A prompt's configured default profile needs no caller profile scope because
the stored reference is already exact. Scope syntax and existence handling are
defined in [decision 2](#2-scope-validation-accepted).

## Discovery And Collection Organization

Namespace enumeration and filtering are integrated into PromptKit's listing
APIs. Consumers select a filter and receive matching full resource identities;
they do not need to fetch all resources and filter them themselves. Defaults,
explicit filter overrides, and result semantics are defined in
[decision 3](#3-discovery-accepted).

Derive namespace membership from the qualified identifier's namespace component:
`weather` must not include `weatherreporter/...`. Nested slash-separated
namespaces are outside the accepted grammar.

Collection namespace/release metadata and an API for reporting effective
collection declarations are desired additions. They are distinct from resource
namespace enumeration. Their configuration shape and specification-version
handling are accepted in [decision 5](#5-collection-metadata-and-specification-versioning-accepted).

## Composition And Compatibility

Qualified identities reduce accidental collisions between independently
authored libraries. Definitions with the same exact qualified identity still
follow the existing ConfigKit overlay and inheritance rules; namespaces do not
create a separate collision policy.

Leave source composition and configuration inspection with ConfigKit. Do not
introduce implicit qualification of definition keys, mounting under caller-chosen
prefixes, or rewriting of stored references as part of consumer scope. Those
would be separate configuration-design work.

ConfigKit owns all configuration processing before handing off the final,
fully resolved configuration. PromptKit must not add a collection-manifest
preflight, source-level uniqueness check, or independent version-checking pass
around that processing. Its domain interpretation and validation consume final
effective values. Source history remains available through explicit ConfigKit
inspection, not as a way to enforce extra assembly rules.

In particular, two documents contributing metadata for the same collection
namespace are governed by ordinary ConfigKit composition semantics. ConfigKit
may reject conflicts or combine contributions according to their layers; a
repeated source declaration is not an additional PromptKit error. The earlier
proposal to reject duplicate declared namespaces before composition is withdrawn.

Existing unscoped exact lookup remains available for IDs valid under the new
grammar. Compatibility with arbitrary slash-containing resource IDs is
intentionally broken. The migration must:

- Audit maintained catalogs and representative consumer definitions for invalid
  IDs and for valid one-slash IDs whose namespace meaning needs review.
- Choose explicit old-to-new mappings and check for collisions. Do not
  automatically replace slashes with dots: distinct existing IDs could collapse
  to the same new identity.
- Update renamed keys and their references together, including inheritance,
  prompt default profiles, profile backend references, backend smoke-test
  defaults, and affected examples and consumer request IDs.
- Keep provider model strings unchanged. Renaming a profile does not rename
  the upstream model.
- Document consumer action and coordinate the PromptKit format change with
  catalog adoption under the [release policy](../policy/release.md). Its catalog
  profile-data exception does not cover this general format/API behavior change;
  deliberately retire its built-in-profile preservation promise when the
  accepted removal lands. Current policy and format documents remain unchanged
  until implementation.

Renaming a backend changes its identity; duplicating it under another ID can
separate capacity accounting. Do not use blanket aliases or duplicate catalog
entries as an automatic compatibility mechanism. Exact release versions and
catalog mappings remain implementation-planning work.

The [architecture policy](../policy/architecture.md) continues to assign CLI
syntax, flags, and application policy to consumers such as Scriptorium. PromptKit
would supply reusable scope and resolution semantics.

## Work Not Included In The Initial Proposal

- Package installation, collection registries, and dependency resolution.
- Automatic rewriting of stored references when mounting or renaming collections.
  Explicit source destinations and namespace selection are supported by proposal 3.
- Implicit namespace search paths or fallback to whichever resource exists.
- Namespace-based access control or filesystem isolation.
- Changes to model names, inference behavior, or configuration precedence.

These boundaries limit the initial proposal; they do not prohibit future work
if a concrete consumer need emerges.

## Delivery Outline

1. Adopt the ConfigKit capabilities selected in decision 5 and audit existing
   identifier usage against the accepted grammar. Record migration mappings and
   application provisioning guidance. Finalize Go names and option types for
   the accepted namespace and collection APIs.
2. Remove automatic catalog inclusion, catalog-module dependencies, and built-in
   profiles under decision 4. Use caller-selected configuration sources and
   update examples/tests to supply their own definitions.
3. Implement the smallest public scope surface with shared name qualification
   and existing engine collaborators. Cover supported operations consistently.
4. Add integrated namespace filtering and representative consumer examples.
   Implement collection metadata/inventory under the accepted decision 5.
5. Verify compatibility and scoped resolution through focused offline tests,
   following the [testing policy](../policy/testing.md). Update GoDoc, the
   format reference, consumer guide, and relevant internal documentation as
   behavior lands; retire this roadmap when its work is complete.

Acceptance cases for the settled rules; the collection roadmap owns additional
configuration and metadata acceptance cases:

- Two prompt scopes resolve the same local name to their respective prompts;
  a missing name never resolves from the other scope or an unqualified entry.
- Two provider profile scopes resolve the same local name, such as
  `google.gemma-4-31b-it`, to different profiles without changing the configured
  provider model value `google/gemma-4-31b-it`.
- A qualified reference bypasses the selected scope without double qualification;
  adding another collection does not change local or qualified resolution.
- Multiple slashes and empty qualified components are rejected in resource IDs
  and references, while provider model values retain their existing syntax.
- Prompt shorthand and a profile override use their intended scopes, while
  stored default references and omitted selections retain their meaning.
- Scoped views coexist concurrently and share backend capacity with ordinary
  calls on the underlying engine.
- Scope creation rejects invalid syntax, accepts well-formed absent namespaces,
  and treats empty scope fields as no scope. Missing resources fail at lookup;
  qualified references remain usable even when the default namespace is absent.
- Listing without an explicit filter uses the resource's view scope, or lists
  all resources when no scope is selected. Explicit named, unqualified-only,
  and all-namespaces filters override that default for the current listing only.
- A valid namespace with no matches produces an empty listing. Namespace
  filtering respects resource type and exact namespace boundaries.
- Exact-ID calls using valid IDs and intentional overlays retain their behavior.
  Migrated catalog references resolve without collisions; the collection roadmap
  owns migration to one installed prompt and optional version labels.
- Listings preserve full IDs and separate optional prompt version labels.
  Qualified results round-trip through either engine or view; unqualified results round-trip
  through the engine without a corresponding scope.
- Representative validation, inspection, execution, and smoke-test paths use
  the same qualification rules without duplicating every test at every layer.
- Engines use caller-selected definitions without automatically contributing
  provider catalogs or PromptKit-owned profiles. Ordinary directory and embedded
  configuration sources retain ConfigKit composition semantics.
- PromptKit does not add source-history-based duplicate-namespace rejection or
  independently pre-validate collection definitions before the final handoff.

## Decisions And Pending Design

Decisions 1–3 are accepted; their alternatives are retained as considered but
not selected. Decision 4 records the accepted change in collection ownership
and supersedes the earlier catalog-only migration proposal. Decision 5 records
the accepted collection configuration direction and links its delivery details.

### 1. Public Scope Shape (Accepted)

**Accepted: an immutable view with independent resource scopes.** Provide one
engine operation that accepts a small value with prompt, profile, and backend
namespace fields and returns a scoped view or an error. The illustrative
`WithNamespaces` and `Namespaces` names above are provisional. Empty fields mean
no scope for that resource type. The view captures the value and delegates to
the same engine; it does not expose a mutable namespace setting. Apply the
prompt field to caller-supplied prompt IDs, the profile field to profile IDs,
and the backend field to backend IDs wherever those selections are supported.
Qualified references bypass the corresponding field.

This directly serves the select-once workflow and the common mixed case of an
application prompt namespace with a provider profile namespace. It also keeps
scope out of stored configuration and makes concurrent use independent of call
order. The cost is an additional facade surface whose supported operations must
remain consistent with the engine. Keep qualification shared rather than
duplicating selection logic in each wrapper.

Initially omit an ambiguous one-string `InNamespace` shorthand. Repeating the
same namespace in several fields happens once when creating a view and states
exactly which resource types it affects. A convenience operation can follow if
that repetition is a demonstrated burden.

**Considered alternative, not selected: one namespace per immutable view,
applied to all resource types.** A one-string operation would scope every
unqualified prompt, profile, and backend selection; callers would use qualified
references for resources in other namespaces. This is smaller and reads naturally for self-contained
application collections. It is less convenient when prompts and profiles
habitually come from different namespaces, because profile overrides would
repeatedly need qualification. Prefer it if consumers primarily use prompts'
configured defaults and rarely select profiles themselves.

**Considered alternative, not selected: explicit scope values supplied per
operation.** Add scope to request values and appropriate inspection/listing
options instead of returning a view. This makes every invocation self-contained
and suits applications that
choose a different scope for every request. It spreads scope through more call
sites and API shapes, and consumers would need their own reusable wrapper to
achieve the requested select-once experience. The immutable view is therefore
the better default for the use case discussed so far.

### 2. Scope Validation (Accepted)

**Accepted: validate syntax when creating the view; resolve existence when
an operation selects a resource.** In the scope value, the empty
string means no scope. Reject a nonempty namespace containing `/` or consisting
only of whitespace. Otherwise preserve its exact spelling, with no trimming,
case folding, or additional character whitelist. This distinguishes omission
from an unusable namespace without inventing normalization rules. Report invalid
scope construction through PromptKit's existing public error conventions.

Allow a well-formed namespace even when it has no current members. Its scoped
listing would be empty, and selection of a missing local resource would produce
the ordinary lookup error. Qualified selections would remain usable regardless
of the view's default scope. An empty optional request ID continues to mean
omission under that operation's existing contract; it is never qualified.

This keeps scope as a naming context and supports uniform application setup
across installations with different library contents. It also avoids treating
successful scope construction as validation of a prompt or execution profile.
The tradeoff is that a misspelled but syntactically valid namespace is detected
only when listed or used, rather than during view construction.

**Considered alternative, not selected: require each selected namespace to have
resources of its corresponding type when creating the view.** For an immutable
assembled engine, this can catch spelling mistakes immediately and gives startup checks useful
feedback without attempting execution. However, it rejects empty optional
collections and a view intended to use qualified references outside its default
namespace. A namespace having one member also does not establish that the
particular required prompt/profile exists. Prefer this approach only if eager
inventory validation is itself an intended consumer contract; it should not be
an accidental consequence of implementing scope creation.

### 3. Discovery (Accepted)

**Accepted: integrated namespace filtering and full-identity results.**
Enumerate the distinct named namespaces present in final effective
prompts, profiles, and backends. Allow filtering that enumeration by resource
type so a prompt picker need not show profile-only namespaces. Provide scoped
listings for the supported resource types, adding a backend inventory surface
as part of this work if needed. Namespace filtering is an explicit option in
the library listing API, not a consumer-side filtering helper. Preserve existing
unfiltered engine calls and their result contracts; the exact Go option shape
remains to be specified.

| Selection | Listing behavior |
| --- | --- |
| No explicit filter on the engine | All namespaces, including unqualified resources |
| No explicit filter on a scoped view | The selected scope for that resource type; all resources of that type if no scope is selected |
| Explicit named namespace | Resources in exactly that namespace |
| Explicit unqualified-only filter | Resources whose IDs have no namespace |
| Explicit all-namespaces filter | All resources of that type, including unqualified resources |

An explicit filter replaces the view's default for the current listing; it
does not intersect with that default or mutate the view. A named namespace
filter uses the namespace syntax rules in decision 2. A well-formed namespace
with no matching resources returns an empty listing. Represent default, named,
unqualified-only, and all-namespaces selections distinctly in the API; do not
overload an empty string to mean several of them.

Return full IDs with optional prompt version labels separately; the full ID
identifies the installed prompt. Do not invent a named namespace such as `default`
for unqualified resources. Namespace enumeration excludes that unnamed group.
Use deterministic ordering consistent
with existing listing contracts.

Qualified result IDs can be passed unchanged to either scoped or unscoped
operations. Unqualified IDs remain literal unqualified identities: when an
explicit all-namespaces or unqualified-only filter exposes one through a scoped
view, use the original engine or a view without that resource scope to select
it exactly. A listing filter does not change subsequent execution scope. This
preserves the accepted lookup rules without introducing an escape syntax.

Integrated filtering gives consumers a direct way to discover the desired
resources, while full identities keep logs and subsequent selections
unambiguous. Namespace/local-name display decomposition is separate from
filtering; a public splitting helper is not required by this decision.

**Considered alternative, not selected: add a richer inventory API returning
full ID, namespace, local name, resource type, and prompt version where
applicable.** Preserve existing list methods and expose the structured form
separately. This is attractive for
Scriptorium-style library browsers: grouping and display require no parsing,
and one inventory operation can cover all resource kinds. It creates another
public representation and overlaps existing listing APIs, so its value should
justify that maintenance. Prefer it if downstream clients routinely need a
combined catalog view rather than a resource picker or namespace filter. In
either approach, the full ID remains the authoritative selection value.

### 4. Collection Ownership And Built-In Removal (Accepted)

**Accepted: load collections as application/operator-supplied configuration,
and remove PromptKit's built-in profiles as a breaking change.** Remove the
mandatory OpenRouter and RakestrawHome catalog Go-module dependencies and their
automatic contribution to engine configuration. PromptKit owns the definition
format and execution machinery; collection publishers own their configuration
repositories, and applications/operators select and provision the collections
they use. One or more explicitly selected directories can supply backends,
profiles, and prompts through the existing ConfigKit integration.

Remove `promptkit-openrouter-standard` and `promptkit-openrouter-auto`, including
their implicit catalog dependencies. Do not replace them with another mandatory
starter bundle. Applications may embed their own defaults using the same
configuration format and loader, then layer operator configuration as desired.
Ordinary application overrides remain valid configuration and need not declare
a collection release of their own.

This decouples collection updates from PromptKit rebuilds and releases and
avoids imposing provider inventories on every consumer. The tradeoff is that
applications and deployments must supply their required configuration and
manage compatible collection revisions. PromptKit does not gain a collection
installer, automatic downloader, or implicit directory search policy.

Retain the captured-engine lifecycle: configuration changes take effect when
the application constructs and validates a replacement engine. A change to an
installed directory does not mutate an existing engine's captured definitions.
Distribution as configuration does not make new provider protocols or diagnostic
adapters data-only features; those can still require implementation work.

Prepare explicit identifier mappings, replacement configuration examples, and
consumer migration instructions. Update the built-in-profile promise in the
release policy and its canonical format documentation when removal lands.
Use a release permitted to break configuration and API contracts under the
current release policy: a minor release before v1, or a major release at/after
v1. Actual catalog mappings, packaging arrangements, and release versions remain
delivery work; independent collection releases need not track future PromptKit
software releases.

**Superseded alternatives:** retaining catalog Go modules with renamed profiles,
or retaining a small PromptKit-owned starter bundle, would simplify initial
provisioning but preserve some provider-data coupling. The accepted direction
places optional defaults in consuming applications and removes those core
dependencies instead.

### 5. Collection Metadata And Specification Versioning (Accepted)

**Accepted direction: [mounted collections](configuration-collections-mounted-fragments.md).**
Proposal 3 is the active configuration roadmap. It uses collection-local mapping
keys, one installed definition per prompt, and an optional flat prompt version.
Input destinations, relative imports, and shared source-version checks are
implemented in ConfigKit's reviewed development branch. PromptKit still needs
to adopt a suitable dependency release and implement its new schema.

Use shared specification checking with `IfPresent` and distinct collection
`release_version` metadata. Collection entries can describe their layout through
relative imports; applications select their sources and namespaces. PromptKit
interprets the final collection/resource keys as qualified IDs after ConfigKit's
handoff. It adds no manifest pre-parser, source-level namespace ownership rule,
or history-based validation pass. Stored references remain exact public IDs.

Proposals 1 and 2 were not selected and have been retired. Their original
contents and the separately preserved ConfigKit identified-record request are
recorded in the collection roadmap's
[retirement note](configuration-collections-mounted-fragments.md#retired-alternatives-and-upstream-request).
The record-sequence request is independent future upstream work and does not
block the selected mapping design.

The collection roadmap owns the candidate schema, examples, version policies,
metadata inventory, and migration requirements. Inventory describes effective
collection declarations captured by the engine, not historical manifests or
proof of unchanged published releases. Source explanation remains ConfigKit's
inspection concern.

The [six configuration decisions](configuration-collections-mounted-fragments.md#accepted-configuration-decisions)
are now accepted: optional main declarations, if-present checking without a new
strict-mode option, no global ban on resources named `version`, removal of
request-version machinery, existing ConfigKit inheritance, and flexible source
assembly with a small convenience API. Entry names and layouts are conventions;
consumers retain source and precedence choices. Cross-path inheritance and
stricter header options are deferred, not prerequisites.

No configuration-policy question remains open from that discussion. Exact API
signatures, metadata representations, dependency release selection, and migration
examples remain [delivery details](configuration-collections-mounted-fragments.md#remaining-delivery-details).
Removal of bundled catalogs and built-in profiles remains accepted.