Request prompt requirement inspection for Narratio
This commit is contained in:
165
docs/roadmap/narratio-feature-request.md
Normal file
165
docs/roadmap/narratio-feature-request.md
Normal file
@@ -0,0 +1,165 @@
|
||||
# Feature request: inspect prompt input requirements
|
||||
|
||||
Status: proposed. This document requests an external Scriptorium API for
|
||||
Narratio and other executable consumers. It does not describe implemented
|
||||
behavior or prescribe implementation structure.
|
||||
|
||||
## Consumer need
|
||||
|
||||
Narratio produces transcripts, extraction outputs, and generated artifacts in
|
||||
a staged pipeline. It binds named inputs to a Scriptorium prompt when generating
|
||||
each analysis artifact. A Narratio source ID and a prompt input name are separate
|
||||
identities: for example, the source
|
||||
`narratio.transcript.final_trimmed_markdown` might be supplied under the prompt
|
||||
input name `transcript`.
|
||||
|
||||
Narratio needs to discover the selected prompt's expected input names before
|
||||
running expensive upstream stages. It can then show operators how prompt inputs
|
||||
map to pipeline products and report missing bindings before transcription or
|
||||
generation begins. At that point, the eventual input files often do not exist.
|
||||
|
||||
The requested API must expose requirements without requiring callers to read
|
||||
prompt definitions or reproduce Scriptorium's interpretation of them.
|
||||
|
||||
## Requested operation
|
||||
|
||||
Provide a read-only CLI operation that reports the effective input requirements
|
||||
for a selected prompt and execution context. A proposed invocation is:
|
||||
|
||||
```text
|
||||
scriptorium inspect requirements --prompt ID [--prompt-version VERSION]
|
||||
[--profile ID] [--var name=value ...]
|
||||
[--config PATH] [--config-overlay NAME]
|
||||
[--promptkit-source PATH ...] [--schema-dir DIR]
|
||||
[--format text|json]
|
||||
```
|
||||
|
||||
Command naming is open for agreement. The required capability is distinct from
|
||||
the currently documented [prompt definition inspection](../cli.md#scriptorium-inspect-prompt):
|
||||
the result describes the caller-facing requirements of the selected execution
|
||||
context. Extending an existing public operation is acceptable if this contract
|
||||
remains explicit and machine-readable.
|
||||
|
||||
Selection must follow the same public configuration, source, prompt-version,
|
||||
profile, and variable semantics as an equivalent execution request. Report the
|
||||
resolved prompt ID/version and effective profile identity, including defaults
|
||||
when the caller omits a selector. Unknown or ambiguous selections must produce
|
||||
an actionable failure consistent with execution selection.
|
||||
|
||||
Input paths, input contents, and an output-artifact path are not required.
|
||||
Inspection must work before upstream products are created. It must not call a
|
||||
model, require generation credentials, reserve generation capacity, or create
|
||||
execution artifacts. Reading configuration and prompt resources needed for
|
||||
inspection is expected.
|
||||
|
||||
## Response contract
|
||||
|
||||
Offer a readable text view for operators and a versioned JSON response for
|
||||
subprocess consumers. JSON stdout must contain one response document; incidental
|
||||
diagnostics belong on stderr. Stable ordering should make repeated inspection
|
||||
of unchanged effective inputs easy to compare.
|
||||
|
||||
The response should expose:
|
||||
|
||||
- Resolved prompt and profile identities.
|
||||
- Every discoverable caller-supplied input name used by the selected prompt,
|
||||
including template references rather than only explicit input declarations.
|
||||
- Whether each input is required, optional, or conditional for this request.
|
||||
- Any declared format or schema constraint that affects accepted input, using
|
||||
public identifiers. Distinguish an unspecified constraint from a known one;
|
||||
do not infer a format from an input's name.
|
||||
- A description or requirement explanation where available, sufficient for an
|
||||
operator to understand a missing-binding diagnostic.
|
||||
- Caller-supplied variable requirements separately from file input requirements,
|
||||
including whether supplied values or defaults satisfy them. Resolved variable
|
||||
values and prompt contents need not be returned.
|
||||
- Whether requirement discovery is complete for the supplied context, with
|
||||
explicit reasons for unresolved requirements.
|
||||
|
||||
For example, this is an illustrative proposed JSON shape, not an existing
|
||||
Scriptorium schema:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": "scriptorium.requirements.v1",
|
||||
"prompt": {"id": "session_recap", "version": "1"},
|
||||
"profile": {"id": "default"},
|
||||
"complete": true,
|
||||
"inputs": [
|
||||
{"name": "glossary", "requirement": "optional", "format": null},
|
||||
{"name": "transcript", "requirement": "required", "format": null}
|
||||
],
|
||||
"variables": [],
|
||||
"unresolved": []
|
||||
}
|
||||
```
|
||||
|
||||
The final field names and schema identifier are open for agreement. Publish
|
||||
their meanings and compatibility rules so Narratio can consume the result
|
||||
without parsing human-readable text. Input names must retain the exact spelling
|
||||
used by execution.
|
||||
|
||||
## Conditional and incomplete requirements
|
||||
|
||||
Use supplied variables and effective defaults when they determine whether an
|
||||
input is needed. An input used only by an inactive conditional branch must not
|
||||
be presented as unconditionally required.
|
||||
|
||||
When a requirement depends on unavailable input contents, an unspecified
|
||||
variable, or a dynamically selected name, report that limitation explicitly.
|
||||
Return any requirements that are known, but do not present a partial list as a
|
||||
complete contract. Conditional entries should explain their condition where it
|
||||
can be expressed through the public API; otherwise report an unresolved reason.
|
||||
|
||||
An empty, complete input list means the selected context needs no caller-supplied
|
||||
file inputs. An empty, incomplete list does not establish that conclusion.
|
||||
Successfully returning an incomplete inspection result may exit zero, provided
|
||||
the response clearly identifies incompleteness. Narratio can then distinguish
|
||||
inspection success from a completed dependency check.
|
||||
|
||||
## Errors and externally observable guarantees
|
||||
|
||||
Invalid selectors, unreadable required configuration/resources, and invalid
|
||||
definitions that prevent inspection must return nonzero. Provide a structured
|
||||
error with a stable category and safe human-readable message in JSON mode.
|
||||
Known incomplete analysis should use the response's incompleteness contract
|
||||
rather than masquerade as an unknown prompt or a successful complete result.
|
||||
|
||||
Inspection must not fail merely because an eventual input artifact is absent.
|
||||
It must not expose credentials, input payloads, or rendered prompt content in
|
||||
its output or errors.
|
||||
|
||||
The reported requirements must agree with execution for the same effective
|
||||
configuration and selectors. This is a description of input requirements at
|
||||
inspection time, not a guarantee of generation success or validation of future
|
||||
input bytes. Changes to prompt resources or configuration after inspection may
|
||||
change the requirements of a later execution.
|
||||
|
||||
## Consumer acceptance scenarios
|
||||
|
||||
1. A prompt requires `transcript`, but Narratio binds only `session_transcript`.
|
||||
Inspection reports the exact required name, allowing Narratio to identify the
|
||||
missing binding before producing either file.
|
||||
2. A template references an undeclared input. Inspection reports the reference
|
||||
or an actionable definition error; it does not silently omit it from a
|
||||
supposedly complete result.
|
||||
3. A prompt accepts an optional glossary. Inspection identifies its optional
|
||||
status, allowing a pipeline without a glossary to remain valid.
|
||||
4. Supplied variables resolve a conditional input requirement. Inspection
|
||||
reflects the selected context; unresolved conditions remain visibly uncertain.
|
||||
5. Selecting another prompt version or profile reports the corresponding
|
||||
effective requirements and resolved identities.
|
||||
6. Inspection succeeds with no generated input files, no model access, and no
|
||||
generation credentials, assuming the selected configuration/resources are
|
||||
otherwise valid and readable.
|
||||
7. An unknown prompt, ambiguous version, or invalid profile yields a structured
|
||||
failure suitable for a Narratio diagnostic naming the affected artifact.
|
||||
|
||||
## Scope boundary
|
||||
|
||||
Narratio owns its source catalog, stage ordering, source-to-input bindings,
|
||||
artifact dependency graph, and session availability checks. Scriptorium only
|
||||
needs to describe its prompt requirements through this public interface.
|
||||
Narratio source IDs and pipeline concepts need not appear in the Scriptorium
|
||||
API. A graph renderer, HTTP endpoint, batch operation, and changes to generation
|
||||
behavior are outside this request.
|
||||
Reference in New Issue
Block a user