From 8ff9aa3efdc9f16b9d2729e0a29ea083eb3c3eaf Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 6 Nov 2026 18:13:52 +0000 Subject: [PATCH] Prepare v0.20.0 release documentation --- README.md | 6 +- docs/releases/v0.20.0.md | 18 + docs/roadmap/implementation.md | 619 -------------------------------- docs/roadmap/inspect-backend.md | 488 ------------------------- docs/roadmap/remediation.md | 180 ---------- 5 files changed, 22 insertions(+), 1289 deletions(-) delete mode 100644 docs/roadmap/implementation.md delete mode 100644 docs/roadmap/inspect-backend.md delete mode 100644 docs/roadmap/remediation.md diff --git a/README.md b/README.md index ba6663e..9810e86 100644 --- a/README.md +++ b/README.md @@ -45,8 +45,10 @@ boundary and constraints that framework work must preserve. ## Release Guidance -The [planned v0.20.0 release notes](docs/releases/v0.20.0.md) describe resource -inspection, typed prompt sources, and the profile null-fallback migration. +The [v0.20.0 release notes](docs/releases/v0.20.0.md) cover resolved resource +inspection, generic JSON/YAML reports, typed prompt sources, and backend +inheritance. Upgrading consumers should review the changed inspection result +types and profile null-fallback behavior. The [v0.19.0 release notes](docs/releases/v0.19.0.md) cover audit remediation and compatibility changes to artifact ownership, programmatic sources, and schemas. diff --git a/docs/releases/v0.20.0.md b/docs/releases/v0.20.0.md index 6c6e48f..33d073d 100644 --- a/docs/releases/v0.20.0.md +++ b/docs/releases/v0.20.0.md @@ -34,4 +34,22 @@ configuration provenance, requirement analysis, and preparation remain separate. supports sibling inheritance. ConfigKit continues to own source composition, snapshotting, and relative message-file contexts. +The source helpers' [GoDoc](../../prompts.go) and +[backend configuration API](../../backends.go) describe programmatic construction. + +## Refinements + +Package GoDoc and migration guidance now consistently describe the resource +inspection contract, and the full package overview is visible in generated +documentation. Prompt inspection avoids repeating the same metadata lookup for +each declared input. + +## Upgrade + +After adapting affected inspection consumers and reviewing profile null usage: + +```sh +go get go.maximumdirect.net/promptkit@v0.20.0 +``` + [COPYRIGHT](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.20.0/COPYRIGHT) | [LICENSE](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.20.0/LICENSE) | [ATTRIBUTION](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.20.0/docs/oss/attribution.md) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 307ad74..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,619 +0,0 @@ -# Resource Inspection And Typed Sources Implementation Plan - -## Purpose And Sprint Model - -Implement the accepted [resource inspection roadmap](inspect-backend.md). That -document owns intent, scope, and product decisions; this plan owns the concrete -steps and representation choices needed to deliver them. - -At baseline `951739e`, ConfigKit v1.1.0 composed and resolved definitions once. -Promptkit captured executable prompts and backend-completed profile targets. -`InspectPrompt` exposed limited metadata; `InspectProfile` exposed an -execution target. There was no `InspectBackend` or typed `PromptSource`, and -`BackendConfig` lacked `Extends`. Existing inspection values did not have the -selected generic JSON/YAML contract. - -Each numbered stage is intended for one Luna implementation prompt, in order. -This is one end-to-end sprint: intermediate stages may fail to compile or leave -callers unmigrated. Do not build compatibility shims, optional capability probes, -dual public inspection modes, or temporary representations just to keep each -stage green. Run focused checks as their affected paths become buildable; -record deferred checks and resolve them before the final stage. There are no -intermediate human-review, publication, or downstream-release gates. - -Follow [agent](../policy/agents.md), [architecture](../policy/architecture.md), -[testing](../policy/testing.md), and [documentation](../policy/documentation.md) -policies and the [configuration boundary ADR](../adr/0001-configuration-boundary.md). -Use the [development guide](../development.md) for toolchain selection and local -commits. Commit completed nonempty units and record stage completion here, -including checks run or deferred. Intermediate commits need not be releasable. - -## Selected Implementation Decisions - -### Resource Boundary And Internal Flow - -All three public inspection operations return the selected resolved resource, -before execution completion. References remain references. Imported message -text comes from the already materialized prompt, not another file read. - -Use a narrow, private resource-inspection interface on `internal/engine.Engine`, -implemented by the configuration snapshot, alongside its existing inspection -and listing collaborators. Its methods accept a context and exact full ID and -return configuration-owned internal resource views. The engine qualifies -caller IDs with the appropriate namespace and delegates. The root facade checks -engine validity, maps errors, and projects detached public values. - -Keep configuration resource views under `internal/configuration`; they contain -no ConfigKit types in their engine-facing fields. Do not add these operations -to execution-profile, generation, capacity, or transport interfaces. Remove the -now-unused runner `InspectPrompt` and `InspectProfile` entry points when wiring -is switched. Retain execution selection helpers used by preparation and smoke -tests. - -Requirement discovery still needs only the small existing prompt metadata -lookup and requirement snapshot. Keep `promptdef.Repository.GetPromptInspection` -and its narrow `domain.PromptInspection` for that purpose, with comments making -the distinction clear. Do not expand that internal value with message bodies. -The new configuration resource view is distinct from this lightweight execution -metadata. This avoids changing `InspectRequirements` into a full-resource copy. - -Index resolved profile and backend configuration records by full ID once during -capture, reusing the already built indexed maps or references into the retained -result. Do not build full parallel configuration trees or query ConfigKit's -provenance during ordinary inspection. Prompt entries already retain captured -messages, templates, and hashes; use that capture and the selected prompt's -resolved values. Small presence metadata is appropriate where decoding has -already collapsed omission into a scalar default. - -Internal views may borrow immutable captured values. Detach mutable data at the -public projection boundary, using existing trusted copy helpers for accepted -provider maps. Do not revalidate provider values merely to copy them. This is -an ownership strategy, not a new public allocation or complexity guarantee. - -### Public Result Shapes - -Keep the existing method signatures for prompt/profile inspection. Add: - -```go -func (e *Engine) InspectBackend(ctx context.Context, backendID string) (*BackendInspection, error) -``` - -Declare the result types in `resource_inspection.go` at the module root. Move -the existing public `PromptInspection` and `ProfileInspection` declarations -there rather than leaving duplicate definitions. These are facade values, not -aliases of private types. Keep execution and extension type shapes unchanged. - -The tables below specify Go fields, types, and serialized keys. Apply both -`json` and `yaml` tags with the same key. Required fields have no `omitempty`; -all pointer fields use `omitempty` and nil means absent. A non-nil pointer to -zero, false, an empty string, or an empty collection remains present. - -| Result | Go fields and types | Serialized keys | -| --- | --- | --- | -| `ProfileInspection` | `ProfileID string` | `id` | -| | `Backend, Endpoint, Model *string` | `backend`, `endpoint`, `model` | -| | `Temperature, TopP *float64` | `temperature`, `top_p` | -| | `MaxTokens, TimeoutSeconds *int` | `max_tokens`, `timeout_seconds` | -| | `ServiceTier, ReasoningEffort *string` | `service_tier`, `reasoning_effort` | -| | `APIKeyEnv *string`, `APIKeyRequired *bool` | `api_key_env`, `api_key_required` | -| | `ExtraParams *map[string]any` | `extra_params` | -| `BackendInspection` | `BackendID string`, `Endpoint string` | `id`, `endpoint` | -| | `APIKeyEnv *string` | `api_key_env` | -| | `ConcurrencyLimit, QueueCapacity *int` | `concurrency_limit`, `queue_capacity` | -| | `ExtraParams *map[string]any` | `extra_params` | -| | `SmokeTests *BackendSmokeTestConfig` | `smoke_tests` | -| `PromptInspection` | `PromptID string`, `PromptHash string` | `id`, `prompt_hash` | -| | `DefaultProfileID, Description, SessionID *string` | `default_profile`, `description`, `session_id` | -| | `Inputs *[]PromptInputInspection` | `inputs` | -| | `Messages []PromptMessageInspection` | `messages` | -| | `OutputContract PromptOutputInspection` | `output` | - -Remove `ProfileInspection.EffectiveModelParams`; do not retain a compatibility -copy of backend-completed settings. `APIKeyRequired` becomes an optional -profile resource field. Retain the familiar prompt field names `Inputs`, -`DefaultProfileID`, and `OutputContract`, but deliberately change their types -where needed for presence. - -Nested prompt values are: - -| Type | Go fields and types | Serialized keys | -| --- | --- | --- | -| `PromptInputInspection` | `Name string`, `Required *bool`, `ContentType, Description *string` | `name`, `required`, `content_type`, `description` | -| `PromptMessageInspection` | `Role string`, `Content string`, `CacheControl *PromptCacheControlInspection` | `role`, `content`, `cache_control` | -| `PromptCacheControlInspection` | `Type CacheControlType`, `TTL *string` | `type`, `ttl` | -| `PromptOutputInspection` | `Format OutputFormat`, `ValidationMode ValidationMode`, `SchemaPath *string`, `RepairAttempts *int` | `format`, `validation_mode`, `schema_path`, `repair_attempts` | - -Required strings and enum values in these nested types are always encoded. -Imported and inline message bodies both use `content`; do not report a -`content_file` field or parsed template. Content is not rendered. A supplied -empty cache TTL stays present, unlike the existing execution cache-control -value's omission behavior. Keep `PromptInputDefinition`, `OutputContract`, and -execution `CacheControl` unchanged for their existing purposes. - -Reuse `BackendSmokeTestConfig`, `ReachabilityProbeConfig`, and -`AuthenticationProbeConfig` for backend diagnostic metadata: their pointer -fields already fit the selected semantics. Add JSON/YAML tags to their fields -(`default_profile`, `reachability`, `authentication`, `endpoint`, `kind`), all -with `omitempty`. Preserve a configured empty reachability object as `{}`; -do not fill an omitted endpoint or invent a diagnostics block. This does not -make `BackendConfig` itself a serialization contract. - -The supported generic encoding paths are `encoding/json` and the existing -`go.yaml.in/yaml/v4` dependency. Ordinary `Marshal`/`Encoder` calls must suffice. -Pointer-to-collection fields avoid needing public presence flags or custom -serializers to distinguish omitted collections from explicit `{}`/`[]`. Do not -add deserialization, source round-trip, output-order, or numeric precision -guarantees. Inspect values rather than inferring source syntax from zero values. - -### Presence And Null Interpretation - -Use resolved typed pointers and nil-versus-empty collections where they retain -presence. Capture only additional information needed for scalar fields such as -input `required` and output `repair_attempts` before they become execution -defaults. Inherited values count as present. For nullable resource fields, -resolved null and omission both produce nil in resource reports. Literal nulls -inside provider maps/arrays remain untouched. - -Keep existing normalization and derived defaults recorded during the ConfigKit -resolution callback. In particular, positive backend concurrency with no queue -setting yields the already recorded queue default in inspection. Do not invent -an explicit zero concurrency value when the resolved field is absent, or add -the framework profile timeout merely because execution uses it. - -Simplify `captureExecutionProfile` to use non-nil configured values rather than -profile-specific absent/null state. Endpoint, credential source, and whole-map -null now permit backend fallback. When `api_key_required` is true and the -profile's `APIKeyEnv` pointer is nil, suppress the backend credential source -just as for omission. An explicit empty endpoint or API-key source still clears -that value. A non-nil provider map still overlays backend values, including -an explicitly empty map that does not clear defaults. Nested provider nulls -remain literal data. - -Remove the profile-only `state.profiles` presence map, `profileFields`, and -capture helpers made unused by this change. Retain helpers needed by other -metadata or validation; do not purge presence/null handling generically. -Preserve execution `ExecutionTargetPresence`, because provider control omission -and explicit zero still matter at dispatch. Do not change required-field or -non-nullable-field validation. ConfigKit's retained result and provenance are -not rewritten to erase nulls. - -### Typed Prompt Source Shapes - -Add `BackendConfig.Extends *string` and encode it exactly like the profile helper. -Add a root `prompts.go` containing these source-input types and helper: - -```go -type PromptConfig struct { - ID string - Extends, DefaultProfile, Description, SessionID *string - Inputs []PromptInputConfig - Messages []PromptMessageConfig - Output *PromptOutputConfig -} - -type PromptInputConfig struct { - Name string - Required *bool - ContentType, Description *string -} - -type PromptMessageConfig struct { - Role, Content, ContentFile *string - CacheControl *PromptCacheControlConfig -} - -type PromptCacheControlConfig struct { - Type, TTL *string -} - -type PromptOutputConfig struct { - Format, ValidationMode, SchemaPath *string - RepairAttempts *int -} - -func PromptSource(label string, prompts []PromptConfig) configkit.Source -``` - -These types are mapping-constructor inputs, not serialized result types. Match -`BackendSource`/`ProfileSource`: always write IDs/names, dereference supplied -pointers, omit nil optional fields, preserve non-nil empty slices/objects, and -return `configkit.Mapping(label, map[string]any{"prompts": records})`. Encode -`Extends` as `extends: {id: value}`. A nil top-level prompt slice supplies an -empty `prompts` sequence, matching the other source helpers. - -Do not require a complete source record in the helper: higher layers or sibling -inheritance may complete it. Engine loading owns final validation, including -invalid content/content-file combinations. Raw mappings retain explicit-null -and generic ConfigKit operations. Do not add a Promptkit-specific content root, -filesystem adapter, recursive input converter, or inheritance resolver. - -## Stage 1: Declare The Resource Result Contract - -**Completed:** Public result fields and JSON/YAML tags reviewed against the -tables; compilation and encoding checks are deferred until the corresponding -projection and consumer migrations are complete. - -**Scope:** Public inspection values and their nested representation only. - -1. Read `types.go`, `backends.go`, and the current inspection GoDoc in `engine.go`. - Create the public result types and tags specified above. Move only the two - existing inspection declarations; leave execution types alone. -2. Add tags to the reused backend diagnostic types. Document optional pointer - and collection semantics, caller ownership, captured content, references, - credential metadata, and the resource/execution distinction. -3. Remove outdated claims that these inspection results have no serialization - contract. Do not remove those claims from unrelated types. - -**Check:** Review every field against the tables, particularly pointer maps, -input `required`, cache TTL, and output `repair_attempts`. Compilation of old -converters and callers may wait for later stages; do not add transitional fields. - -## Stage 2: Simplify Execution's Post-Handoff Null Semantics - -**Completed:** Focused configuration tests pass with the locally selected Go -toolchain. Public resource and retained-provenance assertions for inherited nulls -remain scheduled with the inspection wiring and isolation checks. - -**Scope:** Profile capture and its targeted configuration tests. - -1. In `internal/configuration/targets.go`, implement the pointer-based behavior - above. Continue applying backend defaults once, then the explicit profile - values, then ordinary request-time overrides through the existing path. -2. In `load.go`, remove only the now-unused profile tri-state capture machinery. - Preserve final effective validation and prompt metadata capture. -3. Update `targets_test.go` and relevant `load_test.go`/`interpretation_test.go` - cases. Compare null and omission for endpoint, API-key source, and whole - provider map; retain explicit-empty-string clearing, empty-map overlay, - nested provider nulls, and `api_key_required` behavior with both absent/null - and explicit profile sources. -4. Include real ConfigKit inheritance where a child's null displaces a parent's - endpoint/provider map: the inspected resource is unset and execution uses - the selected backend, not the displaced parent. Preserve the null in the - retained configuration inspection. Inspection assertions may be completed - after resource wiring exists. - -**Check:** Run the focused adapter tests when buildable. Retain outbound-control -presence and timeout behavior; removing configuration null distinctions must -not collapse runtime explicit zero or broaden accepted configuration shapes. - -## Stage 3: Project Resolved Backend And Profile Resources - -**Completed:** Exact-ID resource lookups reuse captured resolved indexes; focused -configuration tests cover partial inheritance, explicit values, backend defaults, -derived queue capacity, diagnostics, missing IDs, and cancellation. - -**Scope:** Configuration-owned views and exact-ID resource lookup. - -1. Add private-adapter result structs in `internal/configuration`, with fields - corresponding to the public profile/backend tables. Keep ConfigKit record - types and paths inside the adapter, and do not alias public root types. -2. Reuse the indexed resolved records built during `buildExecutionProfiles` - instead of discarding and recomputing those indexes for each inspection. - Values may reference the immutable retained config; avoid additional deep - retained copies of provider maps. Keep the execution registry/targets for - their current execution and smoke responsibilities. -3. Add adapter `InspectProfile(ctx, id)` and `InspectBackend(ctx, id)` lookups. - Build selected-resource views from the normalized resolved records, never - from `ExecutionProfile.ResolvedTarget` or the capacity manager. Use existing - internal not-found sentinels for absent IDs and preserve cancellation. -4. Include backend diagnostics only where configured after composition. Return - configured endpoint overrides, including inherited ones, and no derived URLs. - A missing diagnostic default profile does not prevent backend inspection. -5. Add focused adapter coverage using real ConfigKit loading: an inherited - partial profile, explicit zero/empty settings, backend defaults excluded - from that profile, and backend queue derivation and diagnostic metadata. - -**Check:** Verify resource fields against the retained resolved config and -execution fields separately against the backend-completed target. Do not turn -backend/profile references into nested reports or require execution readiness. - -## Stage 4: Capture Complete Prompt Resource Information - -**Completed:** Captured prompt resources retain inherited scalar presence and -materialized message text independently of requirement metadata. Focused -configuration tests pass, including inherited file content and explicit values. - -**Scope:** Prompt presence and resource projection, without changing execution -or requirement-discovery representations. - -1. Extend the existing prompt metadata capture just enough to retain omission - versus explicit false for each input's `required`, and omission versus - explicit zero for `output.repair_attempts`. Use ConfigKit's effective entry - information during the existing callback, including inherited presence. - Keep non-nullable-field rejection intact; do not switch those scalars to - nullable fields as a shortcut. -2. Retain or reference the selected resolved prompt record for optional strings, - inputs-list presence, schema-path presence, and per-message cache TTL - presence. Do not create another copy of the whole configuration tree. -3. Add adapter `InspectPrompt(ctx, id)` returning its resource view. Use - captured `promptEntry.definition` message text/order and existing hash, plus - the resolved resource's presence. Do not reopen message files, render - templates, or load the referenced schema/default profile. -4. Leave `GetPromptInspection` and `domain.PromptInspection` lightweight for - `InspectRequirements`; keep requirement metadata, compiled templates, and - opaque definition hashing behavior independent of the expanded resource DTO. -5. Extend prompt adapter tests for inherited/imported captured content, - optional metadata, empty inline content, cache TTL absence versus empty, - and input/output scalar presence. Reuse existing content-origin fixtures. - -**Check:** Inspect a prompt with an unresolved template variable and an absent -default profile/schema file successfully. Schema and template syntax validation -at construction remain as before. A later content-file change does not alter -the resource report or trigger a second read. - -## Stage 5: Wire Resource Inspection Through The Engine And Facade - -**Completed:** Captured resource views are wired through namespace selection -and detached public projections. An isolated assembled public-API test passes; -existing consumers and examples are migrated in the next stage. - -**Scope:** Selection, public conversion, backend entry point, and errors. - -1. Add the three-method resource-inspection interface and captured collaborator - to `internal/engine.Engine`; initialize it from the configuration snapshot - and share it naturally through `WithNamespaces`. -2. Route internal prompt/profile inspection through this collaborator instead - of the runner, and add backend inspection. Reuse `qualifyResourceID` with - each kind's namespace. Do not qualify references stored inside definitions. -3. Replace root inspection conversion with explicit projections of the new - configuration views. Copy pointer scalars and mutable maps/slices/nested - diagnostics into caller-owned values; reuse immutable strings. Use trusted - detachment for accepted nested provider values, not a serialization round - trip or a new JSON compatibility conversion. -4. Add public `ErrBackendNotFound` and `Engine.InspectBackend`. Scope the mapping - of the registry's internal not-found sentinel to this new operation; do not - globally reclassify construction, smoke, or generation errors. Follow the - existing nil/uninitialized-engine guards and invalid-ID conventions. - Preserve cancellation causes without inventing a public backend-load category. -5. Update exact method/type GoDoc, including `Engine`'s concurrent-operation - list. All three calls return no partial result on error and expose neither - credential values nor runtime occupancy. - -**Check:** A small assembled public-API test for each method builds. Missing -backend matches `ErrBackendNotFound` and not `ErrInvalidRequest`; malformed IDs -match `ErrInvalidRequest`, invalid engines match `ErrInvalidConfig`, and -cancellation remains detectable through `errors.Is`. - -## Stage 6: Retire Superseded Inspection Paths And Migrate Existing Tests - -**Completed:** Removed superseded runner inspection wrappers, migrated public -consumers and the preparation example, and retained target and requirement -coverage at their owning seams. The full ordinary `go test ./...` suite passes. - -**Scope:** Existing consumers and test ownership, without losing execution or -requirement coverage. - -1. Remove the unused runner `InspectPrompt`/`InspectProfile` wrappers and their - wrapper-only tests. Preserve `resolvePromptInspection` for requirement - selection and profile identity/execution helpers used elsewhere. -2. Update root inspection, inheritance, namespace, bundled-configuration, and - source tests for the selected result shape. Preserve literal identity text - and exact stored-reference behavior. Dereference pointers only after - checking presence; an unset value is not a zero-valued setting. -3. Where an old `InspectProfile.EffectiveModelParams` assertion tested backend - completion or framework timeout, keep that protection on `Prepare`/execution - or the focused adapter target tests. Do not simply delete the assertion or - reproduce completed settings in the new inspection result. -4. Keep `InspectRequirements` tests using the narrow metadata/snapshot seam. - Retain protection against full definition/target lookups and source access; - do not replace that seam with the expanded resource report. -5. Migrate any in-repository examples or helper fixtures that actually consume - the changed inspection fields. Leave run/prepared-result, output validation, - appended-message, and extension-client contracts unchanged. - -**Check:** Run the affected public, configuration, use-case, and example tests -when their dependency graph is coherent. Search for stale references to -`ProfileInspection.EffectiveModelParams` and old inspection scalar assumptions; -do not replace valid execution-result fields with the same name. - -## Stage 7: Verify Generic JSON And YAML Rendering - -**Completed:** Real resource results pass the same semantic presence, nested -provider-data, diagnostics, and omission assertions with standard JSON and YAML -marshalling. The existing YAML dependency is now a direct test dependency. - -**Scope:** One focused public serialization contract suite for resource results. - -1. Inspect real configured resources, then encode the returned values directly - using `encoding/json` and `go.yaml.in/yaml/v4`. Consumers in the test must not - build substitute reports, register nested serializers, or switch on fields. - Use the existing YAML version; update `go.mod`/`go.sum` only as needed to - reflect its direct test import, without upgrading dependencies. -2. Decode both outputs into ordinary structures for semantic assertions. Assert - selected names/nesting and presence, not whitespace, key order, or a golden - byte snapshot. Use a small set of rich fixtures rather than one fixture per - field or duplicating configuration validation cases. -3. Cover absent versus explicit zero/false/empty strings, absent/null versus - empty collections, and provider-data nulls. Include profile provider maps, - prompt `inputs`, input `required`, output `repair_attempts`, empty message - content/cache TTL, and backend capacity/diagnostic fields. -4. Assert no `extends`, execution-completed profile fields, parsed templates, - or synthetic diagnostics/endpoints appear. A configured empty reachability - object remains present, and a recorded queue default is present even when - it was not originally written by the operator. - -**Check:** The same semantic cases pass with both ordinary encoders. If tags -alone fail to preserve a selected case, correct the result representation or -add narrowly scoped encoding support; do not move adaptation into consumers. - -## Stage 8: Verify Selection, Isolation, And Offline Boundaries - -**Completed:** Public namespace, offline reference, ownership, concurrent -inspection, execution fallback, and retained null/provenance checks pass under -the race detector. Existing captured-content and requirement tests remain in use. - -**Scope:** Public behavior risks not already covered by the migrated suite. - -1. Extend existing namespace tests for backend inspection: scoped slashless ID, - qualified bypass, no fallback to another namespace, and full selected ID. - Include a bundled backend through the same API. Reuse existing profile/prompt - namespace coverage rather than duplicate every selection case three times. -2. Exercise independence: missing diagnostic default profiles do not prevent - backend reports; partial profiles do not require endpoints/models; prompt - reports do not resolve their default profile or schema reference. -3. Mutate representative returned pointer fields, nested diagnostics, provider - maps/arrays, and prompt input/message/cache values. Reinspection and later - preparation must retain the captured values. Include separate concurrent - inspections using caller-owned results under the race detector, reusing - existing ownership helpers where suitable. -4. Use offline boundary doubles or counting files only where needed to prove - that inspection does not render, read credentials/artifacts/schemas, or call - a provider. Preserve existing source-lifetime tests and avoid mocks of every - private call. No live diagnostic or inference requests belong in this suite. -5. Complete the resource-inspection assertions paired with Stage 2's null - inheritance case and verify that `InspectConfiguration` still shows the - retained null and provenance, separately from execution's backend fallback. - -**Check:** Each test protects a distinct public risk; consolidate overlapping -new and old cases. Run focused tests and their race-enabled versions when useful. -Do not introduce benchmark thresholds or additional release gates. - -## Stage 9: Add Typed Backend Inheritance And Prompt Sources - -**Completed:** Backend inheritance and the typed prompt mapping constructor -follow the existing ConfigKit source pattern. Existing focused programmatic -source and inspection tests pass; composition coverage follows next. - -**Scope:** Public mapping-constructor implementation and GoDoc. - -1. Add `Extends *string` to `BackendConfig` and encode it in `BackendSource` - using the existing profile helper pattern. Update its GoDoc accordingly. -2. Implement the source-input types and `PromptSource` specified above. Write - small explicit mapping construction for nested input, message, cache, and - output values. Preserve supplied order and nil-versus-empty collections. -3. Pass supported mapping values directly to ConfigKit so its snapshot behavior - applies. Preserve the other helpers' final-load validation and accepted-input - contracts. Do not copy through JSON/YAML or add reflection-based encoding. -4. Document nil/explicit values, sibling inheritance, deferred validation, - imported file context, and raw mapping escape hatches. Do not give these - source-input types the inspection result's serialization contract. - -**Check:** Compare each mapping field with the format reference and the source -shape table. Incomplete layered records remain expressible, and constructing -the helper itself neither accesses files nor resolves inheritance. - -## Stage 10: Verify Typed Sources Through Real Composition - -**Completed:** Real engine composition tests cover inherited backend and prompt -records, layered completion, snapshots, explicit values, relative inherited file -content using ConfigKit's filesystem context, and deferred final validation. - -**Scope:** Focused source-helper contract tests, not a second ConfigKit suite. - -1. Extend existing backend-source tests with a real inherited child, preserving - the child's ID and endpoint/capacity overrides through `InspectBackend`. -2. Add prompt-source tests using real `NewEngine` composition. Exercise nested - input/message/output settings, sibling inheritance, source/layer completion, - explicit false/zero/empty values, nil versus empty lists, and message order. - Reuse the new resource APIs to observe resolved values and preparation where - execution behavior is the fact being tested. -3. Mutate original nested source slices, scalar pointees, and records after - creating the source. Loading should use ConfigKit's captured source values. - Keep this representative rather than reimplementing upstream snapshot tests. -4. Include a relative `content_file` from a typed source with ConfigKit's - ordinary explicit source context. Use the dependency's existing context - input setting: construct `filesystem := configkit.NewFilesystem("fixture", fsys)` - and set `configkit.Input.Filesystem` to - `&configkit.FilesystemContext{FS: &filesystem, Root: ".", BaseDir: "templates"}`. - Include an inherited relative message path. Confirm capture uses that context - without adding a new Promptkit path-setting API. -5. Confirm malformed final definitions fail at loading, not helper construction; - rely on existing generic null, inheritance-cycle, and schema validation - coverage instead of repeating every rejection for every typed helper. - -**Check:** Run focused source/inspection tests. Existing `ProfileSource` and raw -mapping workflows remain usable; this stage is not a ConfigKit upgrade or an -authorization to edit another repository. - -## Stage 11: Update Canonical Guidance And Migration Notes - -**Completed:** Updated consumer, format, and internal guidance and prepared the -unpublished v0.20.0 note with licensing links. Changed local link targets and -whitespace were checked; the v0.20.0 tag was absent on the authoritative remote. - -**Scope:** Documentation of the implemented APIs and intentional changes. - -1. Update `docs/consumers/pkg-promptkit.md` with the three resource views, - nullable pointer/collection usage, and minimal direct JSON/YAML rendering. - Distinguish them from `InspectConfiguration` provenance, requirement - discovery, `ValidateRequest`, and concrete-request preparation/details. -2. Document migration from `EffectiveModelParams` to the profile's resource - fields and from prompt scalar metadata to optional inspection values. - Consumers needing completed request settings use existing preparation APIs; - do not add a compatibility execution view to inspection. -3. Update `docs/formats.md`'s post-handoff profile table and map explanation. - Show a concrete null migration: a profile with null endpoint/API-key source/ - provider map now permits backend defaults; explicit empty strings still clear - endpoint/key source, and backend provider maps can be edited through ordinary - composition. Do not imply a new whole-map suppression replacement exists. -4. Document typed prompt sources and backend inheritance through the source - helpers' GoDoc and consumer guidance. Keep exact field contracts in GoDoc - and the existing format owner rather than parallel exhaustive prose schemas. -5. Reconcile `docs/internal/sources.md`, `docs/internal/overview.md`, and affected - runner guidance with configuration-owned resource inspection, retained narrow - requirement metadata, and simplified execution capture. Adjust architecture - descriptions only where implemented structure changed; preserve policy and - the accepted configuration boundary ADR. -6. Prepare the next minor release note under the existing release procedure. - At this baseline `v0.19.0` is published, so use `docs/releases/v0.20.0.md` - unless another version has been published during the sprint. Explain the - inspection and configuration compatibility changes, link canonical guidance, - and include the required licensing footer. Update the README release link. - Do not edit published release notes, tag, push, or run publication commands. -7. Supply enough migration detail for CommandKit to adopt generic reporting - without per-field adapters. Changes or publication in CommandKit itself are - consumer follow-up, not prerequisites or work authorized by this sprint. - -**Check:** Validate changed local links and terminology. Remove old claims that -profile inspection is backend-completed, resource results have no JSON contract, -or null suppresses profile backend fallbacks. Leave unrelated execution and -ConfigKit composition contracts intact. - -## Stage 12: Validate And Close The Sprint - -**Completed:** Audited the selected resource, encoding, source, ownership, and -null-migration boundaries; added coverage for inherited diagnostics and explicit -serialization values. With Go 1.27.3, `GOTOOLCHAIN=local`, and `GOWORK=off`, -`go test -race ./...`, `go vet ./...`, and module tidy verification pass. Changed -Go files are formatted, local documentation targets and whitespace are checked, -and the feature roadmap and plan are retained for review. - -**Scope:** Combined verification and final cleanup of this feature only. - -1. Confirm that all three inspection operations, every selected result field, - both generic encoders, helper inheritance, and post-handoff null migration - are implemented and covered at the appropriate owning boundary. Resolve any - checks deferred by intermediate compilation failures. -2. Check that resource lookup uses captured configuration without execution - completion or a second ConfigKit interpretation; requirement discovery still - uses its narrow metadata path. Remove obsolete converters/wrappers and - private fixtures that preserve the old report behavior, without sweeping - cleanup outside this feature. -3. Format changed Go files and tidy the module only where imports changed. - Using the newest locally available Go installation under the development - guide, run the complete offline suite and vet: - - ```sh - go test -race ./... - go vet ./... - git diff --check - ``` - - Select that toolchain on `PATH`, with `GOTOOLCHAIN=local` and `GOWORK=off`. - The complete suite includes maintained example tests; do not impose another - standalone build/example/release-validation gate. No dependency or consumer - minimum upgrade is selected just to implement this feature. -4. Inspect all changes for unintended behavior and contracts. In particular, - preserve ConfigKit source ownership, literal provider nulls, existing request - overrides, output repair, provider-value support, credential handling, and - captured-state isolation. Do not add context limits, schema probes, health - promises, CLI policy, or source-round-trip guarantees. -5. Verify changed documentation links and review status/diffs under the agent - commit policy. Mark stages complete with actual validation evidence, commit - the finished unit, and report any remaining limitations accurately. Keep the - completed feature roadmap and this plan for the requested implementation - review; retirement can occur during later release preparation. - -Completion requires the full selected end state, including generic rendering -and both deliberate compatibility changes. Passing only the new backend method -or preserving the old execution-oriented profile report is not completion. diff --git a/docs/roadmap/inspect-backend.md b/docs/roadmap/inspect-backend.md deleted file mode 100644 index 6ab8d42..0000000 --- a/docs/roadmap/inspect-backend.md +++ /dev/null @@ -1,488 +0,0 @@ -# Operator-Facing Resource Inspection And Configuration API Roadmap - -## Purpose And Status - -Help operators using CommandKit-based applications inspect the configured -prompt, profile, or backend selected by ID. CommandKit owns commands and report -presentation; Promptkit exposes the selected resource's resolved configuration. - -The selected work has been implemented; this roadmap is retained for review. -It includes fuller prompt inspection, profile inspection of the resolved -profile itself, a new `InspectBackend`, `BackendConfig.Extends`, typed prompt -construction through `PromptConfig`/`PromptSource`, generically renderable -inspection results, and simplification of Promptkit's post-handoff null handling. -Execution-policy reporting -and prepared-request enhancements have been removed from this roadmap. The -decisions below record the accepted scope and representation approach. - -## Resource-Inspection Boundary - -“Fully resolved” means that ConfigKit has applied source composition and -inheritance to the selected resource. A child profile should show its resulting -keys and values, not an unresolved `extends` directive. It does not mean that -inspection assembles everything affecting execution. - -| API | Resource shown | -| --- | --- | -| `InspectPrompt` | Resolved prompt definition, including captured imported message content. | -| `InspectProfile` | Resolved profile definition, before backend-to-profile execution completion. | -| `InspectBackend` | Resolved backend definition, including configured diagnostic metadata. | - -For example, given: - -```yaml -profiles: - - id: base - backend: openrouter - model: example-model - temperature: 0.5 - - id: precise - extends: {id: base} - temperature: 0 -``` - -The resource settings shown for `precise` are: - -```yaml -id: precise -backend: openrouter -model: example-model -temperature: 0 -``` - -ConfigKit inheritance contributes the parent's model and backend reference. -Promptkit's execution-time completion would obtain settings from that backend, -but those settings do not become fields of the inspected profile. The operator -can inspect the referenced backend separately. Likewise, prompt inspection -reports a default profile reference without incorporating that profile's values. - -Use the captured, resolved configuration, including normalizations/defaults -already recorded in that configuration. Do not add framework execution defaults, -request overrides, credential values, or provider-derived defaults to fill -absent resource fields. For example, an absent profile timeout is not populated -from the framework generation timeout, while a queue default already recorded -in the resolved backend remains visible. See the -[format reference](../formats.md#source-headers-collections-and-resource-ids). - -## Design Balance And Reporting - -Provide useful resource information already captured or available with modest -projection. Reuse ConfigKit's resolved handoff and existing resource capture; -avoid another composition pass, execution resolver, parallel provenance system, -or duplicate retained graphs solely for inspection. These are proportionality -guidelines, not prohibitions on a small helper or type when it serves the API. - -**Selected presentation:** Omit absent optional settings. Preserve explicit -values, including zero. Absence is not displayed as zero or `unspecified`. -Treat null like omission for nullable resource fields after ConfigKit resolution, -under the selected post-handoff semantics below. Literal nulls inside provider -data retain their meaning; they are not optional resource-field presence states. -Accepted decision 4 selects typed results without a third state for nullable -resource fields. - -Inspection reports effective resource values rather than original YAML layout, -comments, or the history of each inherited field. `InspectConfiguration` remains -available for provenance. Existing opaque prompt identity/hash information can -remain alongside the resource view; it does not turn inspection into a source -round-trip API. - -Retain existing captured-state, namespace, ownership, and credential-handling -contracts unless a change is deliberately selected. Mutable results should not -let a consumer change retained engine state; that does not require repeated -copies between private helpers or copying immutable strings. Ordinary resource -inspection needs no template rendering, provider calls, or credential -availability checks. -Templates and arbitrary provider settings remain application data whose -presentation and disclosure are controlled by the consuming application. - -## Generic Rendering And Presence - -**Requirement:** Consumers must be able to render `InspectPrompt`, -`InspectProfile`, and `InspectBackend` results generically, without maintaining -parallel resource schemas or field-by-field adapters. Keep typed Go results for -programmatic use and make them directly encodable as JSON and YAML using -ordinary encoders. A consumer calls inspection, handles the error, and passes -the result to its selected encoder; it need not interpret individual settings -or register each nested Promptkit type. - -Promptkit owns serialization field names, nesting, and omission semantics for -these results. Use consistent names aligned with configuration fields where -applicable. CommandKit owns command selection, output format, presentation, and -delivery. Adding an inspection field should not require another CommandKit -field mapping after updating its Promptkit dependency. Consumers may still -choose custom layouts, but generic rendering must be sufficient for a complete -resource report. - -Capture presence before converting configured values to execution values that -collapse omission into zero, false, or an empty string. This applies across all -three resource kinds, including prompt input `required`, output -`repair_attempts`, and backend capacity settings. Use small inspection-specific -values where existing types such as `PromptInputDefinition` or `OutputContract` -cannot preserve that distinction. Do not change shared execution types merely -to make them suitable for inspection. Existing configuration validation rules, -including rejection of null for non-nullable fields, remain intact. - -Omit absent optional settings and nullable fields resolved to null. Retain -explicit zero, false, empty strings, and explicit empty collections where -represented in the resolved resource, along with literal nulls inside provider -data. Generic encoding must preserve these distinctions; a blanket `omitempty` -on scalar or collection fields is not sufficient. Prefer ordinary optional -fields and serialization tags, with focused encoding support only where needed -for the selected semantics. Consumers must not reconstruct presence themselves. - -This is a modest serialization contract for inspection results, not a new -reporting framework, configuration round-trip API, or promise of byte-for-byte -output stability. Formatting choices such as indentation and mapping-key order -belong to the encoder or consumer; semantic sequence order remains meaningful. -Promptkit need not provide CLI layouts or a generic reflection framework. - -## Expanded Prompt Inspection - -Expose description, ordered message templates with roles and cache-control -metadata, and the session-ID template, alongside the existing identity, hash, -default profile reference, input declarations, and output contract. Include -schema references and repair settings as part of the prompt definition. - -Captured prompt definitions already retain the effective content of inline and -imported text/Markdown messages. Return that text without rendering or reopening -files. Original source layout and parsed-template objects are unnecessary. -The schema remains a reference; inspecting the prompt does not load it. - -Keep prompt inspection independent of request readiness and default-profile -lookup. A captured prompt remains inspectable without a runnable default profile. - -## Resolved Profile Inspection - -This is a deliberate change to the former `InspectProfile` behavior. It previously -returned backend-completed `EffectiveModelParams`; the resource boundary -requires the resolved profile's own fields instead. - -Report fields supplied by the profile or inherited through ConfigKit, including -backend reference, endpoint/model where present, generation controls, credential -metadata, and provider parameters. Do not merge in backend provider defaults or -resolve an environment variable. Profiles may be partial, and their references -remain references. Distinguish omitted settings from explicit values, including -zero, using the resolved resource's information rather than inferring presence -from an execution target. Null uses the same absent representation as omission. - -Execution target presence and prepared metadata describe a later resolution -stage and are not substitutes for profile configuration presence. Use the -existing configuration capture path to expose the needed information without -introducing a second configuration interpreter. Follow accepted decision 4 for -typed results and migration from `EffectiveModelParams`. - -This roadmap adds no preparation API surface. Execution continues to apply -backend completion, framework defaults, and request overrides through its -existing path, with the deliberate configuration behavior change below. - -## Post-Handoff Null Semantics - -**Selected behavior:** For Promptkit's interpretation of nullable resource -fields, null and omission mean the same thing. Apply this consistently to -resource inspection and execution, rather than hiding a runtime distinction -only in inspection results. - -ConfigKit owns everything before the handoff of its resolved configuration, -including source composition, inheritance, and null's effects during those -operations. This work makes no upstream change or request, removes no nulls -from input sources, and does not reinterpret or repeat ConfigKit's composition. -The retained ConfigKit result and its provenance remain available through -`InspectConfiguration`; resource projection does not rewrite that tree. - -Remove the existing special treatment of null in these profile fields: - -| Field | Selected meaning of either omission or null after handoff | -| --- | --- | -| `endpoint` | Permit the selected backend's endpoint fallback. | -| `api_key_env` | Permit the ordinary backend credential-source fallback, subject to the existing `api_key_required` rule. | -| Whole-map `extra_params` | Permit the backend's configured provider parameters. | - -Retain explicit values, including zero, false, and the existing empty-string -clearing behavior for `endpoint` and `api_key_env`. Retain the ordinary map -overlay behavior: `extra_params: {}` does not clear backend defaults, and -`extra_params: ""` is not a valid mapping. Profile-level suppression of the -whole backend provider-parameter map via null is deliberately removed; do not -replace it with another clearing option without a concrete requirement. -Operators remain free to remove or replace parameters in backend definitions -through ordinary ConfigKit composition. This does not prohibit those workflows -or change ConfigKit's null semantics. - -Null entries within arbitrary provider maps and arrays remain literal provider -data. This is not recursive null removal. Required-field checks and fields that -currently reject null remain governed by their existing validation rules; this -decision does not broaden accepted configuration shapes. - -For example, ConfigKit may resolve a child's null over a parent's endpoint. -Promptkit then treats the resulting unset profile endpoint like omission and -uses the selected backend's endpoint, if available. The parent's endpoint is -not restored or recomputed by Promptkit. - -## Backend Inspection - -Proposed entry point: - -```go -func (e *Engine) InspectBackend( - ctx context.Context, - backendID string, -) (*BackendInspection, error) -``` - -Expose the full backend identity and its resolved fields: base endpoint, -credential environment-variable name, provider defaults, concurrency/queue -settings, and configured `smoke_tests` metadata. Reuse suitable public nested -values where their semantics match; exact names and nesting follow decision 4. - -Diagnostic metadata includes the configured authentication kind, profile -reference, and endpoint overrides after ConfigKit composition/inheritance. -An inherited endpoint override is a resolved configured value. An endpoint -computed later by a smoke-test adapter is not. Do not synthesize `/key` URLs, -reachability fallbacks, or a `smoke_tests` block that is absent from the resolved -backend. No shared diagnostic URL derivation work is required for this feature. - -Inspecting a backend does not resolve its smoke-test default profile or contact -an endpoint. Report the configured reference even if that profile cannot be -selected. Capacity values describe configured policy, not current occupancy. - -Follow the existing namespace-view selection rules: qualify slashless caller -IDs with the backend scope, pass qualified IDs through, and return the selected -full ID. Bundled and caller-defined resources use the same lookup rules. -Follow existing inspection conventions for invalid engines, invalid selections, -and cancellation. Missing-backend classification follows accepted decision 1. - -## Consumer Integration And Compatibility - -CommandKit can offer navigation between prompt, profile, and backend references -without combining their settings into the inspected resource. Stored references -retain their exact meaning; caller namespace defaults do not rewrite them. -A particular connected layout is a consumer choice, not a new Promptkit API. - -Coordinate CommandKit adoption of generically rendered resource results, -replacing parallel report structures and per-field conversions where they are -no longer needed. In particular, migrate consumers of `EffectiveModelParams`. -Also account for prompt inspection fields whose types need to change to retain -presence; compatibility work is not limited to `ProfileInspection`. Document -the Go and serialized-result migrations with the final types, and follow the -existing release policy. The profile semantic change must be documented as such, -not presented as merely adding more fields. - -Document the post-handoff null change as a configuration compatibility change -as well. Existing null endpoint/credential-source clearing can use the existing -explicit empty-string form when that behavior is desired. There is no new -replacement for profile-level whole-map suppression of backend parameters. -Include a migration example showing that a profile's null now permits backend -endpoint, credential-source, or provider-parameter fallback that it previously -suppressed; this changes execution as well as inspection. Update -the [format reference](../formats.md#defaults-and-overrides) when implementation -lands; it describes the current behavior until then. - -Existing preparation APIs remain available for separate concrete-request uses; -this roadmap adds neither preview commands nor execution-policy report fields. -Transport timeouts, engine retry settings, caller deadlines, and injected-client -policy are outside the inspected resource unless they are actual fields of its -resolved definition. This is the scope of these APIs, not a pending engine-policy -feature decision. - -## Typed Backend Inheritance - -Add `Extends *string` to `BackendConfig`, matching `ProfileConfig.Extends`. -`BackendSource` encodes a non-nil value as `extends: {id: value}` and omits it -when nil. ConfigKit owns sibling inheritance and its diagnostics. The source -helper does not look up parents or populate inherited fields itself. - -Keep existing root/collection identity and ConfigKit composition semantics, -including its handling of null. Raw mappings remain available for generic -controls outside the convenience helper. -See [resource inheritance](../formats.md#resource-inheritance) and the existing -[profile helper](../../profiles.go). - -## Typed Prompt Sources - -Add `PromptConfig` and `PromptSource` to complete the typed source-helper set -alongside `BackendConfig`/`BackendSource` and `ProfileConfig`/`ProfileSource`. -The proposed helper follows their calling convention: - -```go -func PromptSource(label string, prompts []PromptConfig) configkit.Source -``` - -Cover prompt identity, sibling inheritance through `Extends`, default profile, -description, session template, inputs, messages with inline or file content and -cache controls, and output settings. Choose nested input types for configuration -semantics; reuse existing types where suitable without forcing execution or -inspection values into the source-input role. - -Keep the helper a straightforward mapping constructor. Encode supplied fields -under a `prompts` list, then delegate to `configkit.Mapping`. Optional nil fields -mean omission; explicit values, including zero, false, and empty strings where -supported, must remain representable. Preserve nil-versus-empty collections -where that distinction matters to composition. Raw ConfigKit mappings remain -the escape hatch for ConfigKit's explicit-null composition operations and other -advanced controls outside the typed convenience surface. The post-handoff -interpretation above does not change how source helpers supply values to ConfigKit. - -The helper does not resolve inheritance, apply execution defaults, render -messages, read content files, or independently validate a prompt. Existing -ConfigKit composition and engine loading retain those responsibilities. Relative -content references use the ordinary input/source resource-context mechanism; -creating a typed source does not select a new filesystem root or file-loading -workflow. Reuse ConfigKit's source snapshot behavior rather than adding another -copier or recursive compatibility converter. - -Keep all three source-helper families consistent in their scope and omission -conventions. They need not reproduce every generic configuration operation or -share a reflection-based encoding framework. Maintain the field mappings and -focused behavioral tests as the configuration model evolves. - -## Related Work - -Listing return-type consistency is a possible later cleanup, not required here. -Collections, schemas, and engine options do not need identical APIs merely for symmetry. -The [deferred fingerprint proposal](deferred.md#semantic-execution-target-fingerprints) -remains separate. Live capacity monitoring, provider health guarantees, and -source round-tripping are not selected deliverables. - -## Decisions - -All five decisions below are resolved, including generic rendering and the -post-handoff null simplification in decision 4. Numbering is retained for -continuity with the discussion; the selected implementation is complete. - -### 1. How Should An Unknown Backend Be Classified? - -**Status:** Accepted and implemented. - -**Decision:** Add public `ErrBackendNotFound` for an absent backend selected by -`InspectBackend`, following the existing prompt/profile sentinel pattern. -An ordinary missing-backend lookup should match `ErrBackendNotFound` through -`errors.Is` and should not also match `ErrInvalidRequest`. Reuse the registry's -existing internal missing-backend classification at the public boundary. - -Invalid selections follow existing ID rules and match `ErrInvalidRequest`; -invalid engines match `ErrInvalidConfig`. Preserve context cancellation through -the existing inspection conventions. Invalid backend definitions remain an -engine-construction concern. Exact error wording is not a new contract. - -Initially apply the new public classification to `InspectBackend`; do not -implicitly reclassify construction or smoke-test failures. CommandKit owns -operator wording and exit-status selection. - -**Rationale:** This gives consumers a useful distinction with a small addition -consistent with prompt and profile inspection, without a new error hierarchy. - -**Alternatives considered:** Using `ErrInvalidRequest` saves one public sentinel -but combines malformed selections and missing identities. A typed generic -resource-not-found error could carry resource kind and ID, but adds a broader -contract and integration questions for existing sentinels. Neither is selected; -the caller already knows the operation and requested ID. - -### 2. Resource Inspection Does Not Include Execution-Policy Reporting - -**Status:** Resolved; the former engine-policy proposal is removed from scope. - -**Decision:** Inspect the selected resolved resource, not all settings that may -affect its execution. ConfigKit composition and inheritance are included; -backend-to-profile execution completion, framework execution defaults, request -overrides, and engine transport/retry reporting are not. - -Remove the proposed engine-settings API, prepared-result policy fields, and -prepared-result presence enhancements from this roadmap. No decision about -where to place those additions remains necessary for resource inspection. - -### 3. Should Typed Prompt Construction Be Added In This Round? - -**Status:** Accepted and implemented. - -**Decision:** Add `PromptConfig` and `PromptSource`, including inheritance, -following the thin mapping-constructor pattern described above. Retain the -backend/profile helpers and raw ConfigKit mapping support. - -**Rationale:** API symmetry and convenient programmatic construction have value -as Promptkit approaches 1.0. The existing helpers are small explicit encoders; -a prompt helper adds nested input values and mappings, not a new authoring, -composition, or validation subsystem. That bounded maintenance cost is justified -by consistent discoverability and compiler assistance across all three resources. - -**Alternatives considered:** Deferring prompt helpers would leave asymmetric Go -conveniences. Removing the existing typed helpers would simplify Promptkit's -surface but move string-keyed mapping construction to consumers and require -migration. Neither is selected. Full typed coverage of all ConfigKit operations -is unnecessary; raw mappings continue to serve those cases. - -### 4. How Should Resolved Resource Results And Presence Be Represented? - -**Status:** Accepted and implemented. - -**Decision (Option A):** Use resource-oriented typed inspection results, reusing -suitable public nested values. Change `ProfileInspection` deliberately to report -the resolved profile rather than retaining a second execution-settings view for -compatibility. Apply the same resource semantics to prompt/backend additions. -Require generic rendering under the contract above; typed access must not force -consumers to maintain parallel report types or interpret every setting. - -Represent absent settings and explicit values, including zero, false, and empty -strings where valid. Treat nullable resource fields set to null as absent in -the result and adopt the post-handoff behavior described above. This supersedes -the earlier proposal to preserve three profile-specific null distinctions. -ConfigKit composition and literal nulls within provider data retain their -existing ownership and meaning. No third resource-field presence state or -generic resource framework is needed. Execution-target presence does not -represent the uncompleted profile and must not substitute for it. - -**Rationale:** Typed results provide discoverable Go fields without making -consumers interpret a generic configuration mapping. Endpoint and credential -overrides already take precedence without first clearing a value, and explicit -empty strings already support clearing those settings. No concrete requirement -justifies retaining profile-level whole-map suppression of backend parameters. -Removing these exceptions simplifies interpretation and avoids carrying source -syntax and inheritance history into every field. - -**Alternative considered:** A caller-owned resource mapping naturally represents -missing keys and null and can reduce per-field projection work, but sacrifices -typed discoverability and introduces a broader mapping-value contract. It is not -selected as the primary resource-inspection result; `InspectConfiguration` -remains available for explicit configuration/provenance inspection. - -Document migration for existing profile and prompt inspection consumers with -the final Go types and their serialization semantics. Generic JSON/YAML encoding -is selected; bespoke consumer reports and custom serializers are not inherently -required. Do not leak internal package types or mutable captured state. - -### 5. Backend Diagnostics Show Configured Values, Not Derived Destinations - -**Status:** Resolved by the resource-inspection boundary. - -**Decision:** Return diagnostic settings present in the resolved backend, -including inherited endpoint overrides. Leave omitted endpoint settings absent; -deriving a default destination belongs to diagnostic execution. The earlier -proposal to expose both declared and computed destinations is removed. This -avoids an additional representation and provider-specific derivation machinery -in resource inspection. - -## Acceptance Criteria - -The companion [implementation plan](implementation.md) owns the staged work -and verification sequence. The target is complete when: - -- Prompt, profile, and backend inspection expose the selected resolved resource - with the boundary and presence semantics above, using ordinary JSON/YAML - encoding without per-field consumer adapters. -- Inherited values, explicit zero/false/empty settings, and provider-data nulls - are represented accurately; nullable resource nulls and omission are absent. -- Profile inspection excludes backend-completed execution values, and backend - inspection reports configured diagnostics without synthesizing destinations - or requiring diagnostic readiness. -- Prompt inspection includes captured inline/imported message content and - metadata without rendering or loading related resources. -- Execution applies the selected post-handoff null behavior while preserving - explicit clearing, map overlay, and ConfigKit's retained result and provenance. -- Selection, errors, captured-state isolation, and credential handling follow - the selected contracts. Requirement discovery retains its lightweight path. -- All three resource kinds have typed source helpers, including inheritance, - using ordinary ConfigKit composition, context, and snapshot behavior. -- Canonical guidance and the next minor release note explain changed Go - inspection types, generic rendering, source helpers, and null migration. - -Promptkit owns resource serialization checks; CommandKit owns presentation and -delivery. No engine-policy or prepared-request additions, benchmark threshold, -or new release gate is part of this feature. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 905b29f..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,180 +0,0 @@ -# Resource Inspection Implementation Review - -## Scope And Results - -Reviewed the twelve stages of [implementation.md](implementation.md) against -the accepted [feature roadmap](inspect-backend.md) and the current code and -documentation at `f083ad5`. The implementation commits run from `8add7c4` -through `f083ad5`, following planning commit `8563379`. - -The selected code architecture is implemented. Resource inspection delegates -from the facade through the internal engine to captured configuration records; -execution keeps its backend-completed targets, and requirement discovery keeps -its lightweight metadata path. Public projections detach mutable values. -Typed sources use ordinary ConfigKit mapping, composition, and resource context. - -The initial review identified one documentation finding. The subsequent -algorithmic review adds one low-priority optimization opportunity. No -correctness defect or unintended new restriction or guarantee was identified -in the reviewed implementation. The -resource-oriented profile result and changed post-handoff null fallback are -deliberately selected compatibility changes, not accidental regressions. - -Review evidence below describes the pre-remediation implementation. Completion -markers record the subsequent corrections. Both findings are now addressed. - -## Findings - -### R1. Reconcile Remaining Inspection Contract Documentation [P2] - -**Status:** Complete. **Affected stages:** 1, 11, and final reconciliation in 12. - -Completion: package GoDoc now describes the resource JSON/YAML contract and -includes backend inspection in its orientation/ownership summary. A blank -comment separator that hid most of the rendered package overview was repaired. -The migration guide now distinguishes resolved profile inspection from completed -request settings and links to the current consumer guidance. Rendered `go doc`, -current-documentation checks, local links, and whitespace were verified; no -runtime behavior or published release note changed. - -**Locations:** - -- [Package GoDoc](../../doc.go), lines 50–68: the JSON section still explicitly - lists `ProfileInspection` and `PromptInspection` among values without stable - JSON representations. -- [Configuration migration guidance](../consumers/configuration-migration.md), - lines 133–139: it still says that `InspectProfile` applies backend completion. - -These statements contradict the implemented result contracts in -[resource_inspection.go](../../resource_inspection.go), the method GoDoc in -[engine.go](../../engine.go), and the updated -[consumer guidance](../consumers/pkg-promptkit.md#inspect-a-profile-before-prompt-work). -The result types now deliberately own JSON/YAML keys, nesting, and omission -semantics. Profile inspection reports only the resolved profile's fields; -backend-completed settings belong to request preparation. - -**Impact:** Consumers consulting package documentation are told that the new -generic rendering contract is unavailable. Consumers following the migration -guide may still expect backend endpoints, credential sources, or provider -defaults in a profile report even though those fields can now be absent. This -undermines the primary migration and reporting purpose of the feature. - -**Required correction:** - -1. Update the package GoDoc's serialization description to include the selected - resource-result contract and remove the contradictory exclusions. Link to - the result/method owners for details. Preserve the distinction from source - inputs, provenance inspection, and unrelated values without a serialization - contract; do not add byte-format, ordering, or round-trip guarantees. -2. Replace the migration guide's backend-completion claim with the resolved - resource boundary and point readers needing completed request settings to - the existing preparation APIs/current consumer guidance. -3. Check the remaining current documentation for equivalent stale claims. - Published historical release notes should remain unchanged. - -**Verification:** Review the rendered package GoDoc and the affected migration -passage against the implemented types and method behavior; check edited local -links and whitespace. This is a documentation correction and does not require -another implementation path, compatibility field, or new runtime test. - -### R2. Hoist The Prompt Presence Lookup Out Of The Input Loop [P3] - -**Status:** Complete. **Affected stage:** 4. - -Completion: `InspectPrompt` fetches the selected presence slice once before -iterating over inputs. The loop now indexes that slice directly, with no new -cache or public contract. Existing inherited-presence, captured-content, generic -JSON/YAML encoding, typed-source, and isolation tests pass with Go 1.27.3 under -the race detector; `go vet ./...` also passes. No benchmark or implementation- -coupled lookup-count test was added. - -**Location:** [Prompt resource projection](../../internal/configuration/resources.go), -`InspectPrompt`, lines 77–85, especially `state.promptMetadata[id][i]` at line 81. - -Every input repeats a lookup in the same map using the same prompt ID. The map -and selected metadata slice do not change during the loop; only the slice index -changes. This loop organization was not an algorithm selected by the roadmap. - -**Complexity:** Let `I` be the selected prompt's input count, `M` its message -count, and `L` the prompt ID's byte length. With expected hash-table probing, -map lookup is constant in the number of catalog entries, but hashing a string -key can still cost `O(L)`. The current loop can therefore spend `O(I × L)` on -repeated key processing, for `O(L + I × L + I + M)` total adapter projection -time. Valid resource IDs have no fixed length limit. Go 1.27.3's ordinary -string-map lookup hashes the supplied key on each lookup; its small-map fast -path is an optimization, not a general elimination of this cost. - -**Potential refactoring:** Fetch `presence := state.promptMetadata[id]` once -before iterating over the inputs and use `presence[i].required`. This changes -the projection bound to `O(L + I + M)` with constant additional storage. The -existing `O(I + M)` result allocation and public ownership copies remain -unchanged. No new cache, API, validation, or ID limit is needed. - -This is a modest loop-invariant hoist, not a claim that inspection is quadratic -in catalog size. When ID length is treated as a fixed small constant, both -versions are linear in inputs/messages; the improvement removes an avoidable -multiplicative dependence when IDs and input counts grow independently. - -**Verification:** Existing inherited-input presence and generic encoding tests -should continue to pass, including absent versus explicit false. Source review -can verify that the invariant lookup is outside the loop. No timing threshold, -real-world performance study, or test of an exact map-call count is needed. - -## Stage-By-Stage Review Record - -| Stage | Evidence inspected and result | -| --- | --- | -| 1 — Result declarations | `resource_inspection.go` and nested diagnostic tags in `backends.go` match the selected fields and optional-pointer representation. Package-level serialization documentation still needs R1. | -| 2 — Null interpretation | `captureExecutionProfile` uses configured pointer/map presence; profile-only tri-state capture is removed. Target and isolation tests cover null/omission fallback, inherited null, empty-string clearing, map overlay, and the required-key rule. | -| 3 — Backend/profile resources | `internal/configuration/resources.go` reads indexed resolved records, not completed targets. Adapter tests cover partial/inherited profiles, derived queue defaults, missing IDs, and configured/inherited diagnostics. | -| 4 — Prompt resources | Prompt capture retains the resolved record, materialized messages, and required/repair presence. Prompt tests cover inherited imported text and omitted versus explicit values; inspection does not reopen sources or resolve referenced profiles/schemas. | -| 5 — Facade wiring | The internal engine's resource interface delegates to the snapshot after namespace qualification. Root conversion detaches pointers, collections, and nested diagnostics; backend not-found mapping is local to `InspectBackend`. Public error and cancellation tests pass. | -| 6 — Existing consumers | Superseded runner wrappers and their dedicated tests are removed. Execution helpers and the narrow requirement repository remain. Existing profile, inheritance, source, namespace, bundle, and preparation-example tests are migrated. | -| 7 — Generic encoding | `resource_serialization_test.go` encodes real results with ordinary JSON and YAML encoders and checks semantic keys, omission, explicit values, empty collections, diagnostic objects, and provider nulls. The existing YAML dependency is direct; its version is unchanged. | -| 8 — Isolation/selection | Public tests cover scoped backend lookup, qualified bypass, no namespace fallback, independent results, concurrent inspection/mutation, offline references, execution fallback, and retained null provenance. | -| 9 — Typed sources | `prompts.go` and `BackendSource` use explicit mapping construction with optional pointers/collections and sibling `extends`. They do not validate incomplete records, resolve inheritance, or read content files. | -| 10 — Real composition | Typed-source tests cover snapshots, layered completion, nested settings, order, inheritance, explicit values, relative file context, invalid final definitions, and empty top-level input. | -| 11 — Documentation | Main consumer, format, and internal guidance, README, and the unpublished v0.20.0 release note reflect the selected changes. Two remaining current-documentation contradictions require R1. | -| 12 — Combined validation | Race-enabled suite, vet, module tidy comparison, formatting, and changed Markdown link-target checks pass. The documentation finding prevents declaring reconciliation fully complete. | - -## Validation Performed - -Using Go 1.27.3 with `GOTOOLCHAIN=local`, `GOWORK=off`, and `GOPROXY=off`: - -- `go test -race ./...` passed, including the maintained offline examples. -- `go vet ./...` passed. -- `go mod tidy -diff` reported no changes. -- Formatting checks on surviving changed Go files and `git diff --check` passed. -- Local link targets in the changed Markdown documents exist. - -The review made no production-code or test changes. These checks do not include -live provider diagnostics, release publication, or downstream application changes. - -## Algorithmic Review - -This follow-up reviewed the same production revision, now at repository commit -`101a284` (the intervening commit adds only this review document). It used static -loop, call-path, ownership, and data-size analysis, not performance measurements. -The existing architecture, testing, and documentation policies apply. Explicitly -selected algorithms and dependency-owned composition/encoding were not reopened. - -| Stage | Algorithmic assessment | -| --- | --- | -| 1 — Result declarations | Types and tags introduce no traversal algorithm. Optional values preserve presence without requiring consumer-side reconstruction. | -| 2 — Null interpretation | The fixed set of profile-field checks is constant-sized. Existing backend-provider overlay and target materialization are retained decisions, not new algorithms to redesign in this audit. | -| 3 — Backend/profile resources | Resolved-record indexes are captured once. Individual inspection selects one record rather than scanning the catalog; public provider-map detachment visits the selected value graph. | -| 4 — Prompt resources | Input and message loops visit their respective sequences once, but the input loop has the avoidable repeated string-key lookup described in R2. Scalar presence capture does not scan all prompts for each input. | -| 5 — Facade wiring | Namespace qualification is proportional to ID length. Pointer projection is fixed-field work; nested provider copying is proportional to the copied value graph. No per-field whole-catalog lookup is introduced. | -| 6 — Existing consumers | Removed wrappers do not leave a second resolution pass. Requirement discovery continues using the narrow metadata/snapshot path, rather than copying the expanded resource report. | -| 7 — Generic encoding | Standard JSON/YAML encoding and presence-aware result types are selected planning decisions. No Promptkit serialization round trip or generic reflection layer was added. | -| 8 — Isolation/selection | Inspection itself performs no admission, provider work, schema compilation, or file reads. Caller-owned copies scale with the selected report rather than unrelated resources. | -| 9 — Typed sources | Constructors iterate once over supplied records and their nested input/message sequences, then delegate snapshotting to ConfigKit. These are additive nested sizes, not an all-inputs-by-all-messages cross product. | -| 10 — Real composition | Tests exercise the dependency-owned loading path; no second inheritance resolver or normalization pass was added for typed sources. | -| 11 — Documentation | No production algorithm; the previously recorded documentation correction remains R1. No additional performance guarantee is selected. | -| 12 — Combined validation | The reviewed new production paths contain no additional cross-resource quadratic scan or correctness defect requiring a finding. Existing successful suite/vet results remain applicable because production code is unchanged. | - -The R2 complexity analysis accounts for string-key processing, rather than -assuming every map lookup is free of key-size cost. The selected Go toolchain's -`internal/runtime/maps/runtime_faststr.go` was inspected to confirm that point. -No production implementation, test, benchmark, dependency, or public contract was -changed by this follow-up.