Align Narratio requirements with native CommandKit reports
This commit is contained in:
@@ -208,6 +208,32 @@ qualified IDs and explicit namespace flags when selecting namespaced resources,
|
|||||||
while retaining upstream behavior for unqualified IDs. Configurable defaults
|
while retaining upstream behavior for unqualified IDs. Configurable defaults
|
||||||
can be considered later if a need arises.
|
can be considered later if a need arises.
|
||||||
|
|
||||||
|
### Native inspection reports and consumer requirements
|
||||||
|
|
||||||
|
Decision recorded 2026-11-19: adopt native inspection reports and the native
|
||||||
|
`promptkit requirements show` interface for Narratio's early dependency
|
||||||
|
discovery. Its consumer needs and acceptance scenarios are recorded in the
|
||||||
|
[aligned Narratio proposal](narratio-feature-request.md). This direction is
|
||||||
|
accepted; Scriptorium integration remains to be implemented.
|
||||||
|
|
||||||
|
Consume native report fields without a Scriptorium-specific schema-version
|
||||||
|
envelope. Use native nonzero outcomes and diagnostics for invocation failures;
|
||||||
|
JSON report selection does not require JSON errors. Neither a separately
|
||||||
|
versioned envelope nor programmatic invocation-failure categories blocks this
|
||||||
|
migration. A future concrete need for either belongs in an upstream discussion.
|
||||||
|
|
||||||
|
PromptKit owns requirement classification, discovery completeness, and structured
|
||||||
|
discovery issues. Completeness is distinct from satisfied bindings or execution
|
||||||
|
readiness; Narratio owns comparison against its planned inputs. Keep early
|
||||||
|
discovery separate from concrete request validation. Use current exact resource
|
||||||
|
identities and native input/variable metadata, without restoring obsolete prompt
|
||||||
|
versions or inventing input schemas or variable-default semantics.
|
||||||
|
|
||||||
|
Standard prompt inspection includes captured messages; prepared preview includes
|
||||||
|
rendered content. Adopt these native reports without rebuilding Scriptorium's
|
||||||
|
previous summary projections, and explain the changed presentation in migration
|
||||||
|
guidance. Requirements discovery retains its distinct upstream content boundary.
|
||||||
|
|
||||||
## Upstream baseline
|
## Upstream baseline
|
||||||
|
|
||||||
The initial review covered CommandKit **v0.7.0**, ckconfig **v0.9.0**, and
|
The initial review covered CommandKit **v0.7.0**, ckconfig **v0.9.0**, and
|
||||||
@@ -252,33 +278,7 @@ responsibilities. Decide their mapping to the CommandKit invocation context
|
|||||||
and PromptKit operation timeout, separately from the standard per-attempt
|
and PromptKit operation timeout, separately from the standard per-attempt
|
||||||
timeout flag.
|
timeout flag.
|
||||||
|
|
||||||
### 2 Native reports and executable consumer needs
|
### 2 Application commands and standard request flags
|
||||||
|
|
||||||
Adopt native inspection reports as the starting point. Standard prompt
|
|
||||||
inspection includes captured prompt messages, unlike the current Scriptorium
|
|
||||||
summary. Document that change and confirm how native inspection and preparation
|
|
||||||
reports meet operator needs without rebuilding custom projections.
|
|
||||||
|
|
||||||
Reconcile the proposed [Narratio requirements request](narratio-feature-request.md)
|
|
||||||
with native `promptkit requirements show`. Its central need remains valid:
|
|
||||||
discover input requirements before expensive upstream work and before input
|
|
||||||
files exist. Preserve the distinction between successful discovery and complete
|
|
||||||
discovery; a successful partial report does not establish all requirements.
|
|
||||||
|
|
||||||
The proposal's legacy syntax and prompt-version assumptions need revision. Its
|
|
||||||
illustrative versioned JSON envelope and structured-error request also need a
|
|
||||||
decision: native reports do not add that Scriptorium envelope, and `--format
|
|
||||||
json` does not automatically turn diagnostics into a JSON error protocol.
|
|
||||||
Determine which consumer requirements are essential and whether any genuine
|
|
||||||
gap belongs upstream. Avoid treating the existing proposal as a mandate for a
|
|
||||||
new compatibility facade.
|
|
||||||
|
|
||||||
Keep discovery distinct from concrete request validation: validation prepares a
|
|
||||||
real request and may read input artifacts and check relevant credentials;
|
|
||||||
discovery must work before those artifacts exist. Neither guarantees a future
|
|
||||||
generation will succeed.
|
|
||||||
|
|
||||||
### 3 Application commands and standard request flags
|
|
||||||
|
|
||||||
Decide the final generation command operands and application-specific flags
|
Decide the final generation command operands and application-specific flags
|
||||||
around the reusable ckprompt request flag groups. Use their standard names,
|
around the reusable ckprompt request flag groups. Use their standard names,
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
# Feature request: inspect prompt input requirements
|
# Prompt input requirements for Narratio
|
||||||
|
|
||||||
Status: proposed. This document requests an external Scriptorium API for
|
Status: accepted direction, pending implementation through the CommandKit
|
||||||
Narratio and other executable consumers. It does not describe implemented
|
migration. This proposal records Narratio's consumer needs and their mapping
|
||||||
behavior or prescribe implementation structure.
|
to the native interface; it does not describe shipped Scriptorium behavior.
|
||||||
|
The [CommandKit notepad](commandkit-notepad.md) records the migration decisions.
|
||||||
|
|
||||||
## Consumer need
|
## Consumer need
|
||||||
|
|
||||||
@@ -13,153 +14,153 @@ identities: for example, the source
|
|||||||
`narratio.transcript.final_trimmed_markdown` might be supplied under the prompt
|
`narratio.transcript.final_trimmed_markdown` might be supplied under the prompt
|
||||||
input name `transcript`.
|
input name `transcript`.
|
||||||
|
|
||||||
Narratio needs to discover the selected prompt's expected input names before
|
Narratio needs to discover expected input names before running expensive
|
||||||
running expensive upstream stages. It can then show operators how prompt inputs
|
upstream stages and before the eventual input files exist. It can then show
|
||||||
map to pipeline products and report missing bindings before transcription or
|
operators how prompt inputs map to pipeline products and identify missing
|
||||||
generation begins. At that point, the eventual input files often do not exist.
|
bindings without reading prompt definitions or reproducing PromptKit's analysis.
|
||||||
|
|
||||||
The requested API must expose requirements without requiring callers to read
|
## Selected native operation
|
||||||
prompt definitions or reproduce Scriptorium's interpretation of them.
|
|
||||||
|
|
||||||
## Requested operation
|
Expose `scriptorium promptkit requirements show PROMPT_ID` through ckprompt.
|
||||||
|
Subprocess consumers select JSON with the standard `--format json` flag.
|
||||||
|
Default and explicit text output use the native YAML presentation.
|
||||||
|
The [upstream CLI contract][requirements-cli] owns syntax and report details.
|
||||||
|
|
||||||
Provide a read-only CLI operation that reports the effective input requirements
|
Use the same application configuration and framework source selection as
|
||||||
for a selected prompt and execution context. A proposed invocation is:
|
execution, including `--config`, `--overlay`, and applicable application
|
||||||
|
selectors. The native `--llm-profile`, `--llm-var`, `--llm-session-id`, and
|
||||||
|
prompt/profile namespace flags supply the relevant discovery context.
|
||||||
|
Preserve upstream selection semantics, including the prompt's default profile
|
||||||
|
when an explicit profile is omitted. Qualified IDs identify installed resources;
|
||||||
|
there is no separate prompt-version selector or reported prompt-version field.
|
||||||
|
|
||||||
```text
|
The selected profile establishes identity, not execution readiness. Selecting
|
||||||
scriptorium inspect requirements --prompt ID [--prompt-version VERSION]
|
another profile does not itself change the prompt's discovered requirements.
|
||||||
[--profile ID] [--var name=value ...]
|
Scriptorium's loading and request mapping must preserve the same effective
|
||||||
[--config PATH] [--config-overlay NAME]
|
selection and applicable caller context across discovery and execution.
|
||||||
[--promptkit-source PATH ...] [--schema-dir DIR]
|
|
||||||
[--format text|json]
|
|
||||||
```
|
|
||||||
|
|
||||||
Command naming is open for agreement. The required capability is distinct from
|
Discovery requires no input bindings or output-artifact path. PromptKit performs
|
||||||
the currently documented [prompt definition inspection](../cli.md#scriptorium-inspect-prompt):
|
no artifact loading, schema loading, credential resolution, message rendering,
|
||||||
the result describes the caller-facing requirements of the selected execution
|
capacity reservation, or model call for this operation. Reading configuration
|
||||||
context. Extending an existing public operation is acceptable if this contract
|
and prompt resources needed to build the captured engine is expected. The
|
||||||
remains explicit and machine-readable.
|
[PromptKit discovery contract][requirements-api] owns the analysis and its
|
||||||
|
limits.
|
||||||
|
|
||||||
Selection must follow the same public configuration, source, prompt-version,
|
## Native response and consumer interpretation
|
||||||
profile, and variable semantics as an equivalent execution request. Report the
|
|
||||||
resolved prompt ID/version and effective profile identity, including defaults
|
|
||||||
when the caller omits a selector. Unknown or ambiguous selections must produce
|
|
||||||
an actionable failure consistent with execution selection.
|
|
||||||
|
|
||||||
Input paths, input contents, and an output-artifact path are not required.
|
Consume the native JSON report directly, without a Scriptorium-specific wrapper
|
||||||
Inspection must work before upstream products are created. It must not call a
|
or schema-version field. Successful report delivery writes one JSON document to
|
||||||
model, require generation credentials, reserve generation capacity, or create
|
stdout, with diagnostics on stderr under CommandKit's standard reporting rules.
|
||||||
execution artifacts. Reading configuration and prompt resources needed for
|
Consumers requesting a report should not enable quiet mode.
|
||||||
inspection is expected.
|
|
||||||
|
|
||||||
## Response contract
|
| Field | Consumer meaning |
|
||||||
|
| --- | --- |
|
||||||
|
| `prompt_id`, `profile_id` | Exact selected resource identities. |
|
||||||
|
| `inputs` | Declared and recognizable template-referenced inputs, including exact `name`, requirement `kind`, `declared`, `content_type`, `description`, and `explanation`. |
|
||||||
|
| `variables` | Recognizable caller-variable keys with `name`, requirement `kind`, `supplied`, and `explanation`. |
|
||||||
|
| `complete` | Whether the supported dependency analysis is complete for the supplied context. |
|
||||||
|
| `issues` | Structured discovery limitations with a `code`, associated input or variable names where known, and a human-readable description. |
|
||||||
|
|
||||||
Offer a readable text view for operators and a versioned JSON response for
|
PromptKit sorts input and variable entries by exact name, and deduplicates and
|
||||||
subprocess consumers. JSON stdout must contain one response document; incidental
|
orders issues by their code and associated names. Preserve empty inventories
|
||||||
diagnostics belong on stderr. Stable ordering should make repeated inspection
|
and false values. Input metadata distinguishes absent/null values from an
|
||||||
of unchanged effective inputs easy to compare.
|
explicitly empty string. Explanations and issue descriptions are for display;
|
||||||
|
Narratio must use structured fields rather than parse their prose.
|
||||||
|
|
||||||
The response should expose:
|
The native input metadata includes content type and description. Current
|
||||||
|
PromptKit inputs have no per-input schema declaration to project. Do not infer
|
||||||
|
constraints from names or treat the output-validation schema as an input schema.
|
||||||
|
Variable `supplied` records exact-key presence, including an explicitly empty
|
||||||
|
value; this interface does not introduce a separate variable-default system.
|
||||||
|
|
||||||
- Resolved prompt and profile identities.
|
Requirements reports exclude artifact payloads, prompt contents, and unrelated
|
||||||
- Every discoverable caller-supplied input name used by the selected prompt,
|
supplied variable values. A supplied value used to resolve an input identifier
|
||||||
including template references rather than only explicit input declarations.
|
can appear as that identifier. Keep this native behavior without adding a custom
|
||||||
- Whether each input is required, optional, or conditional for this request.
|
projection of the report.
|
||||||
- Any declared format or schema constraint that affects accepted input, using
|
|
||||||
public identifiers. Distinguish an unspecified constraint from a known one;
|
|
||||||
do not infer a format from an input's name.
|
|
||||||
- A description or requirement explanation where available, sufficient for an
|
|
||||||
operator to understand a missing-binding diagnostic.
|
|
||||||
- Caller-supplied variable requirements separately from file input requirements,
|
|
||||||
including whether supplied values or defaults satisfy them. Resolved variable
|
|
||||||
values and prompt contents need not be returned.
|
|
||||||
- Whether requirement discovery is complete for the supplied context, with
|
|
||||||
explicit reasons for unresolved requirements.
|
|
||||||
|
|
||||||
For example, this is an illustrative proposed JSON shape, not an existing
|
The upstream report contract and adopted dependency versions govern the shape.
|
||||||
Scriptorium schema:
|
Migration documentation must identify relevant consumer changes; this proposal
|
||||||
|
adds no independent response-versioning mechanism or stronger compatibility
|
||||||
|
guarantee.
|
||||||
|
|
||||||
```json
|
## Conditional and incomplete discovery
|
||||||
{
|
|
||||||
"schema_version": "scriptorium.requirements.v1",
|
|
||||||
"prompt": {"id": "session_recap", "version": "1"},
|
|
||||||
"profile": {"id": "default"},
|
|
||||||
"complete": true,
|
|
||||||
"inputs": [
|
|
||||||
{"name": "glossary", "requirement": "optional", "format": null},
|
|
||||||
{"name": "transcript", "requirement": "required", "format": null}
|
|
||||||
],
|
|
||||||
"variables": [],
|
|
||||||
"unresolved": []
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The final field names and schema identifier are open for agreement. Publish
|
PromptKit classifies dependencies as required, optional, or conditional using
|
||||||
their meanings and compatibility rules so Narratio can consume the result
|
the supplied context and its supported template analysis. Recognizable references
|
||||||
without parsing human-readable text. Input names must retain the exact spelling
|
only on known skipped paths can be optional. Explicit required declarations
|
||||||
used by execution.
|
still apply. Unavailable input contents, unresolved variable values or dynamic
|
||||||
|
names, and unsupported dependency-relevant constructs can leave discovery
|
||||||
|
incomplete. Valid templates are not rejected merely because the analysis cannot
|
||||||
|
fully resolve them.
|
||||||
|
|
||||||
## Conditional and incomplete requirements
|
Narratio must inspect both the requirements and discovery completeness:
|
||||||
|
|
||||||
Use supplied variables and effective defaults when they determine whether an
|
- `complete: true` establishes discovery coverage, not satisfied bindings,
|
||||||
input is needed. An input used only by an inactive conditional branch must not
|
valid future input bytes, or execution readiness. Required inputs or missing
|
||||||
be presented as unconditionally required.
|
required variables may still be present.
|
||||||
|
- `complete: false` preserves useful findings and structured issues. An empty
|
||||||
|
partial inventory does not prove that no inputs are needed.
|
||||||
|
- A partial report can return status zero. An issue is not itself an invocation
|
||||||
|
failure. Narratio owns its policy for proceeding with incomplete dependency
|
||||||
|
information.
|
||||||
|
|
||||||
When a requirement depends on unavailable input contents, an unspecified
|
Once input artifacts exist, native `promptkit requirements validate` can check
|
||||||
variable, or a dynamically selected name, report that limitation explicitly.
|
a concrete request through preparation. It is a separate operation and must not
|
||||||
Return any requirements that are known, but do not present a partial list as a
|
replace early discovery. Neither operation guarantees later generation success.
|
||||||
complete contract. Conditional entries should explain their condition where it
|
Changes to configuration or resources between invocations can change findings.
|
||||||
can be expressed through the public API; otherwise report an unresolved reason.
|
|
||||||
|
|
||||||
An empty, complete input list means the selected context needs no caller-supplied
|
## Failures
|
||||||
file inputs. An empty, incomplete list does not establish that conclusion.
|
|
||||||
Successfully returning an incomplete inspection result may exit zero, provided
|
|
||||||
the response clearly identifies incompleteness. Narratio can then distinguish
|
|
||||||
inspection success from a completed dependency check.
|
|
||||||
|
|
||||||
## Errors and externally observable guarantees
|
Use CommandKit's native nonzero outcomes and diagnostic messages for selection,
|
||||||
|
configuration, operational, and report-delivery failures. JSON report selection
|
||||||
|
does not turn diagnostic messages into a categorized JSON error response.
|
||||||
|
Narratio can attach the diagnostic to its own artifact or pipeline context;
|
||||||
|
it must not depend on parsing the message to recover a stable error category.
|
||||||
|
|
||||||
Invalid selectors, unreadable required configuration/resources, and invalid
|
Structured discovery issue codes belong to successful inspection reports and
|
||||||
definitions that prevent inspection must return nonzero. Provide a structured
|
are distinct from invocation failures. Follow PromptKit's distinction between
|
||||||
error with a stable category and safe human-readable message in JSON mode.
|
partial discovery and failures that return no inspection. Absence of a future
|
||||||
Known incomplete analysis should use the response's incompleteness contract
|
input artifact is not a discovery failure because discovery does not load it.
|
||||||
rather than masquerade as an unknown prompt or a successful complete result.
|
|
||||||
|
|
||||||
Inspection must not fail merely because an eventual input artifact is absent.
|
The accepted consumer requirement is a nonzero return code plus failure
|
||||||
It must not expose credentials, input payloads, or rendered prompt content in
|
information. If programmatic failure categories or separate schema versioning
|
||||||
its output or errors.
|
become necessary, discuss them upstream rather than introducing local wrappers.
|
||||||
|
Neither is a prerequisite for this migration.
|
||||||
The reported requirements must agree with execution for the same effective
|
|
||||||
configuration and selectors. This is a description of input requirements at
|
|
||||||
inspection time, not a guarantee of generation success or validation of future
|
|
||||||
input bytes. Changes to prompt resources or configuration after inspection may
|
|
||||||
change the requirements of a later execution.
|
|
||||||
|
|
||||||
## Consumer acceptance scenarios
|
## Consumer acceptance scenarios
|
||||||
|
|
||||||
1. A prompt requires `transcript`, but Narratio binds only `session_transcript`.
|
1. A prompt requires `transcript`, but Narratio binds only `session_transcript`.
|
||||||
Inspection reports the exact required name, allowing Narratio to identify the
|
The native inventory identifies the exact required name before either file
|
||||||
missing binding before producing either file.
|
exists, allowing Narratio to identify the missing binding.
|
||||||
2. A template references an undeclared input. Inspection reports the reference
|
2. A template references an undeclared input. Recognizable references appear in
|
||||||
or an actionable definition error; it does not silently omit it from a
|
the inventory with `declared: false`; unsupported dependency analysis is
|
||||||
supposedly complete result.
|
explicitly incomplete rather than silently claiming full coverage.
|
||||||
3. A prompt accepts an optional glossary. Inspection identifies its optional
|
3. An optional glossary is reported with its native classification, allowing
|
||||||
status, allowing a pipeline without a glossary to remain valid.
|
Narratio to distinguish it from a required binding.
|
||||||
4. Supplied variables resolve a conditional input requirement. Inspection
|
4. Variables resolve a supported conditional dependency. Unknown conditions
|
||||||
reflects the selected context; unresolved conditions remain visibly uncertain.
|
retain conditional findings or discovery issues according to PromptKit.
|
||||||
5. Selecting another prompt version or profile reports the corresponding
|
5. Selecting another qualified prompt or profile reports the corresponding
|
||||||
effective requirements and resolved identities.
|
identities. Prompt selection changes the analyzed definition; profile
|
||||||
6. Inspection succeeds with no generated input files, no model access, and no
|
selection alone does not change its requirements.
|
||||||
generation credentials, assuming the selected configuration/resources are
|
6. Discovery works with no generated input files, model access, or generation
|
||||||
otherwise valid and readable.
|
credentials when the selected configuration and resources are readable and
|
||||||
7. An unknown prompt, ambiguous version, or invalid profile yields a structured
|
valid for discovery.
|
||||||
failure suitable for a Narratio diagnostic naming the affected artifact.
|
7. An unknown prompt or invalid profile selection returns nonzero with native
|
||||||
|
diagnostics suitable for Narratio to associate with the affected artifact.
|
||||||
|
8. Narratio treats a complete report with missing required bindings differently
|
||||||
|
from a complete, satisfied dependency check, and handles partial reports
|
||||||
|
without assuming that status zero establishes completeness.
|
||||||
|
9. Scriptorium's adapter uses consistent configuration, namespaces, and applicable
|
||||||
|
request context across discovery, validation, preview, and execution.
|
||||||
|
|
||||||
## Scope boundary
|
## Scope boundary
|
||||||
|
|
||||||
Narratio owns its source catalog, stage ordering, source-to-input bindings,
|
Narratio owns its source catalog, stage ordering, source-to-input bindings,
|
||||||
artifact dependency graph, and session availability checks. Scriptorium only
|
artifact dependency graph, and session availability checks. Scriptorium exposes
|
||||||
needs to describe its prompt requirements through this public interface.
|
PromptKit's native requirements through CommandKit. Narratio source IDs and
|
||||||
Narratio source IDs and pipeline concepts need not appear in the Scriptorium
|
pipeline concepts do not become part of that interface. A graph renderer, HTTP
|
||||||
API. A graph renderer, HTTP endpoint, batch operation, and changes to generation
|
endpoint, batch operation, and changes to generation behavior are outside this
|
||||||
behavior are outside this request.
|
request.
|
||||||
|
|
||||||
|
[requirements-cli]: https://forge.maximumdirect.net/base-layer/commandkit/src/commit/d330d7637f2e7507eca931a899397222c82bc892/docs/cli.md#promptkit-requirements
|
||||||
|
[requirements-api]: https://forge.maximumdirect.net/base-layer/promptkit/src/commit/804a4d32b23aa2e7973400bb0a322a7016574c9f/requirements.go
|
||||||
|
|||||||
Reference in New Issue
Block a user