166 lines
8.0 KiB
Markdown
166 lines
8.0 KiB
Markdown
|
|
# Feature request: inspect prompt input requirements
|
||
|
|
|
||
|
|
Status: proposed. This document requests an external Scriptorium API for
|
||
|
|
Narratio and other executable consumers. It does not describe implemented
|
||
|
|
behavior or prescribe implementation structure.
|
||
|
|
|
||
|
|
## Consumer need
|
||
|
|
|
||
|
|
Narratio produces transcripts, extraction outputs, and generated artifacts in
|
||
|
|
a staged pipeline. It binds named inputs to a Scriptorium prompt when generating
|
||
|
|
each analysis artifact. A Narratio source ID and a prompt input name are separate
|
||
|
|
identities: for example, the source
|
||
|
|
`narratio.transcript.final_trimmed_markdown` might be supplied under the prompt
|
||
|
|
input name `transcript`.
|
||
|
|
|
||
|
|
Narratio needs to discover the selected prompt's expected input names before
|
||
|
|
running expensive upstream stages. It can then show operators how prompt inputs
|
||
|
|
map to pipeline products and report missing bindings before transcription or
|
||
|
|
generation begins. At that point, the eventual input files often do not exist.
|
||
|
|
|
||
|
|
The requested API must expose requirements without requiring callers to read
|
||
|
|
prompt definitions or reproduce Scriptorium's interpretation of them.
|
||
|
|
|
||
|
|
## Requested operation
|
||
|
|
|
||
|
|
Provide a read-only CLI operation that reports the effective input requirements
|
||
|
|
for a selected prompt and execution context. A proposed invocation is:
|
||
|
|
|
||
|
|
```text
|
||
|
|
scriptorium inspect requirements --prompt ID [--prompt-version VERSION]
|
||
|
|
[--profile ID] [--var name=value ...]
|
||
|
|
[--config PATH] [--config-overlay NAME]
|
||
|
|
[--promptkit-source PATH ...] [--schema-dir DIR]
|
||
|
|
[--format text|json]
|
||
|
|
```
|
||
|
|
|
||
|
|
Command naming is open for agreement. The required capability is distinct from
|
||
|
|
the currently documented [prompt definition inspection](../cli.md#scriptorium-inspect-prompt):
|
||
|
|
the result describes the caller-facing requirements of the selected execution
|
||
|
|
context. Extending an existing public operation is acceptable if this contract
|
||
|
|
remains explicit and machine-readable.
|
||
|
|
|
||
|
|
Selection must follow the same public configuration, source, prompt-version,
|
||
|
|
profile, and variable semantics as an equivalent execution request. Report the
|
||
|
|
resolved prompt ID/version and effective profile identity, including defaults
|
||
|
|
when the caller omits a selector. Unknown or ambiguous selections must produce
|
||
|
|
an actionable failure consistent with execution selection.
|
||
|
|
|
||
|
|
Input paths, input contents, and an output-artifact path are not required.
|
||
|
|
Inspection must work before upstream products are created. It must not call a
|
||
|
|
model, require generation credentials, reserve generation capacity, or create
|
||
|
|
execution artifacts. Reading configuration and prompt resources needed for
|
||
|
|
inspection is expected.
|
||
|
|
|
||
|
|
## Response contract
|
||
|
|
|
||
|
|
Offer a readable text view for operators and a versioned JSON response for
|
||
|
|
subprocess consumers. JSON stdout must contain one response document; incidental
|
||
|
|
diagnostics belong on stderr. Stable ordering should make repeated inspection
|
||
|
|
of unchanged effective inputs easy to compare.
|
||
|
|
|
||
|
|
The response should expose:
|
||
|
|
|
||
|
|
- Resolved prompt and profile identities.
|
||
|
|
- Every discoverable caller-supplied input name used by the selected prompt,
|
||
|
|
including template references rather than only explicit input declarations.
|
||
|
|
- Whether each input is required, optional, or conditional for this request.
|
||
|
|
- Any declared format or schema constraint that affects accepted input, using
|
||
|
|
public identifiers. Distinguish an unspecified constraint from a known one;
|
||
|
|
do not infer a format from an input's name.
|
||
|
|
- A description or requirement explanation where available, sufficient for an
|
||
|
|
operator to understand a missing-binding diagnostic.
|
||
|
|
- Caller-supplied variable requirements separately from file input requirements,
|
||
|
|
including whether supplied values or defaults satisfy them. Resolved variable
|
||
|
|
values and prompt contents need not be returned.
|
||
|
|
- Whether requirement discovery is complete for the supplied context, with
|
||
|
|
explicit reasons for unresolved requirements.
|
||
|
|
|
||
|
|
For example, this is an illustrative proposed JSON shape, not an existing
|
||
|
|
Scriptorium schema:
|
||
|
|
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"schema_version": "scriptorium.requirements.v1",
|
||
|
|
"prompt": {"id": "session_recap", "version": "1"},
|
||
|
|
"profile": {"id": "default"},
|
||
|
|
"complete": true,
|
||
|
|
"inputs": [
|
||
|
|
{"name": "glossary", "requirement": "optional", "format": null},
|
||
|
|
{"name": "transcript", "requirement": "required", "format": null}
|
||
|
|
],
|
||
|
|
"variables": [],
|
||
|
|
"unresolved": []
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
The final field names and schema identifier are open for agreement. Publish
|
||
|
|
their meanings and compatibility rules so Narratio can consume the result
|
||
|
|
without parsing human-readable text. Input names must retain the exact spelling
|
||
|
|
used by execution.
|
||
|
|
|
||
|
|
## Conditional and incomplete requirements
|
||
|
|
|
||
|
|
Use supplied variables and effective defaults when they determine whether an
|
||
|
|
input is needed. An input used only by an inactive conditional branch must not
|
||
|
|
be presented as unconditionally required.
|
||
|
|
|
||
|
|
When a requirement depends on unavailable input contents, an unspecified
|
||
|
|
variable, or a dynamically selected name, report that limitation explicitly.
|
||
|
|
Return any requirements that are known, but do not present a partial list as a
|
||
|
|
complete contract. Conditional entries should explain their condition where it
|
||
|
|
can be expressed through the public API; otherwise report an unresolved reason.
|
||
|
|
|
||
|
|
An empty, complete input list means the selected context needs no caller-supplied
|
||
|
|
file inputs. An empty, incomplete list does not establish that conclusion.
|
||
|
|
Successfully returning an incomplete inspection result may exit zero, provided
|
||
|
|
the response clearly identifies incompleteness. Narratio can then distinguish
|
||
|
|
inspection success from a completed dependency check.
|
||
|
|
|
||
|
|
## Errors and externally observable guarantees
|
||
|
|
|
||
|
|
Invalid selectors, unreadable required configuration/resources, and invalid
|
||
|
|
definitions that prevent inspection must return nonzero. Provide a structured
|
||
|
|
error with a stable category and safe human-readable message in JSON mode.
|
||
|
|
Known incomplete analysis should use the response's incompleteness contract
|
||
|
|
rather than masquerade as an unknown prompt or a successful complete result.
|
||
|
|
|
||
|
|
Inspection must not fail merely because an eventual input artifact is absent.
|
||
|
|
It must not expose credentials, input payloads, or rendered prompt content in
|
||
|
|
its output or errors.
|
||
|
|
|
||
|
|
The reported requirements must agree with execution for the same effective
|
||
|
|
configuration and selectors. This is a description of input requirements at
|
||
|
|
inspection time, not a guarantee of generation success or validation of future
|
||
|
|
input bytes. Changes to prompt resources or configuration after inspection may
|
||
|
|
change the requirements of a later execution.
|
||
|
|
|
||
|
|
## Consumer acceptance scenarios
|
||
|
|
|
||
|
|
1. A prompt requires `transcript`, but Narratio binds only `session_transcript`.
|
||
|
|
Inspection reports the exact required name, allowing Narratio to identify the
|
||
|
|
missing binding before producing either file.
|
||
|
|
2. A template references an undeclared input. Inspection reports the reference
|
||
|
|
or an actionable definition error; it does not silently omit it from a
|
||
|
|
supposedly complete result.
|
||
|
|
3. A prompt accepts an optional glossary. Inspection identifies its optional
|
||
|
|
status, allowing a pipeline without a glossary to remain valid.
|
||
|
|
4. Supplied variables resolve a conditional input requirement. Inspection
|
||
|
|
reflects the selected context; unresolved conditions remain visibly uncertain.
|
||
|
|
5. Selecting another prompt version or profile reports the corresponding
|
||
|
|
effective requirements and resolved identities.
|
||
|
|
6. Inspection succeeds with no generated input files, no model access, and no
|
||
|
|
generation credentials, assuming the selected configuration/resources are
|
||
|
|
otherwise valid and readable.
|
||
|
|
7. An unknown prompt, ambiguous version, or invalid profile yields a structured
|
||
|
|
failure suitable for a Narratio diagnostic naming the affected artifact.
|
||
|
|
|
||
|
|
## Scope boundary
|
||
|
|
|
||
|
|
Narratio owns its source catalog, stage ordering, source-to-input bindings,
|
||
|
|
artifact dependency graph, and session availability checks. Scriptorium only
|
||
|
|
needs to describe its prompt requirements through this public interface.
|
||
|
|
Narratio source IDs and pipeline concepts need not appear in the Scriptorium
|
||
|
|
API. A graph renderer, HTTP endpoint, batch operation, and changes to generation
|
||
|
|
behavior are outside this request.
|