From 0cd95f6e8e4d472cdf4ca59d23a4de54fc5450e8 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 4 Oct 2026 17:15:59 +0000 Subject: [PATCH] Request prompt requirement inspection for Narratio --- docs/roadmap/narratio-feature-request.md | 165 +++++++++++++++++++++++ 1 file changed, 165 insertions(+) create mode 100644 docs/roadmap/narratio-feature-request.md diff --git a/docs/roadmap/narratio-feature-request.md b/docs/roadmap/narratio-feature-request.md new file mode 100644 index 0000000..36cd342 --- /dev/null +++ b/docs/roadmap/narratio-feature-request.md @@ -0,0 +1,165 @@ +# Feature request: inspect prompt input requirements + +Status: proposed. This document requests an external Scriptorium API for +Narratio and other executable consumers. It does not describe implemented +behavior or prescribe implementation structure. + +## Consumer need + +Narratio produces transcripts, extraction outputs, and generated artifacts in +a staged pipeline. It binds named inputs to a Scriptorium prompt when generating +each analysis artifact. A Narratio source ID and a prompt input name are separate +identities: for example, the source +`narratio.transcript.final_trimmed_markdown` might be supplied under the prompt +input name `transcript`. + +Narratio needs to discover the selected prompt's expected input names before +running expensive upstream stages. It can then show operators how prompt inputs +map to pipeline products and report missing bindings before transcription or +generation begins. At that point, the eventual input files often do not exist. + +The requested API must expose requirements without requiring callers to read +prompt definitions or reproduce Scriptorium's interpretation of them. + +## Requested operation + +Provide a read-only CLI operation that reports the effective input requirements +for a selected prompt and execution context. A proposed invocation is: + +```text +scriptorium inspect requirements --prompt ID [--prompt-version VERSION] + [--profile ID] [--var name=value ...] + [--config PATH] [--config-overlay NAME] + [--promptkit-source PATH ...] [--schema-dir DIR] + [--format text|json] +``` + +Command naming is open for agreement. The required capability is distinct from +the currently documented [prompt definition inspection](../cli.md#scriptorium-inspect-prompt): +the result describes the caller-facing requirements of the selected execution +context. Extending an existing public operation is acceptable if this contract +remains explicit and machine-readable. + +Selection must follow the same public configuration, source, prompt-version, +profile, and variable semantics as an equivalent execution request. Report the +resolved prompt ID/version and effective profile identity, including defaults +when the caller omits a selector. Unknown or ambiguous selections must produce +an actionable failure consistent with execution selection. + +Input paths, input contents, and an output-artifact path are not required. +Inspection must work before upstream products are created. It must not call a +model, require generation credentials, reserve generation capacity, or create +execution artifacts. Reading configuration and prompt resources needed for +inspection is expected. + +## Response contract + +Offer a readable text view for operators and a versioned JSON response for +subprocess consumers. JSON stdout must contain one response document; incidental +diagnostics belong on stderr. Stable ordering should make repeated inspection +of unchanged effective inputs easy to compare. + +The response should expose: + +- Resolved prompt and profile identities. +- Every discoverable caller-supplied input name used by the selected prompt, + including template references rather than only explicit input declarations. +- Whether each input is required, optional, or conditional for this request. +- Any declared format or schema constraint that affects accepted input, using + public identifiers. Distinguish an unspecified constraint from a known one; + do not infer a format from an input's name. +- A description or requirement explanation where available, sufficient for an + operator to understand a missing-binding diagnostic. +- Caller-supplied variable requirements separately from file input requirements, + including whether supplied values or defaults satisfy them. Resolved variable + values and prompt contents need not be returned. +- Whether requirement discovery is complete for the supplied context, with + explicit reasons for unresolved requirements. + +For example, this is an illustrative proposed JSON shape, not an existing +Scriptorium schema: + +```json +{ + "schema_version": "scriptorium.requirements.v1", + "prompt": {"id": "session_recap", "version": "1"}, + "profile": {"id": "default"}, + "complete": true, + "inputs": [ + {"name": "glossary", "requirement": "optional", "format": null}, + {"name": "transcript", "requirement": "required", "format": null} + ], + "variables": [], + "unresolved": [] +} +``` + +The final field names and schema identifier are open for agreement. Publish +their meanings and compatibility rules so Narratio can consume the result +without parsing human-readable text. Input names must retain the exact spelling +used by execution. + +## Conditional and incomplete requirements + +Use supplied variables and effective defaults when they determine whether an +input is needed. An input used only by an inactive conditional branch must not +be presented as unconditionally required. + +When a requirement depends on unavailable input contents, an unspecified +variable, or a dynamically selected name, report that limitation explicitly. +Return any requirements that are known, but do not present a partial list as a +complete contract. Conditional entries should explain their condition where it +can be expressed through the public API; otherwise report an unresolved reason. + +An empty, complete input list means the selected context needs no caller-supplied +file inputs. An empty, incomplete list does not establish that conclusion. +Successfully returning an incomplete inspection result may exit zero, provided +the response clearly identifies incompleteness. Narratio can then distinguish +inspection success from a completed dependency check. + +## Errors and externally observable guarantees + +Invalid selectors, unreadable required configuration/resources, and invalid +definitions that prevent inspection must return nonzero. Provide a structured +error with a stable category and safe human-readable message in JSON mode. +Known incomplete analysis should use the response's incompleteness contract +rather than masquerade as an unknown prompt or a successful complete result. + +Inspection must not fail merely because an eventual input artifact is absent. +It must not expose credentials, input payloads, or rendered prompt content in +its output or errors. + +The reported requirements must agree with execution for the same effective +configuration and selectors. This is a description of input requirements at +inspection time, not a guarantee of generation success or validation of future +input bytes. Changes to prompt resources or configuration after inspection may +change the requirements of a later execution. + +## Consumer acceptance scenarios + +1. A prompt requires `transcript`, but Narratio binds only `session_transcript`. + Inspection reports the exact required name, allowing Narratio to identify the + missing binding before producing either file. +2. A template references an undeclared input. Inspection reports the reference + or an actionable definition error; it does not silently omit it from a + supposedly complete result. +3. A prompt accepts an optional glossary. Inspection identifies its optional + status, allowing a pipeline without a glossary to remain valid. +4. Supplied variables resolve a conditional input requirement. Inspection + reflects the selected context; unresolved conditions remain visibly uncertain. +5. Selecting another prompt version or profile reports the corresponding + effective requirements and resolved identities. +6. Inspection succeeds with no generated input files, no model access, and no + generation credentials, assuming the selected configuration/resources are + otherwise valid and readable. +7. An unknown prompt, ambiguous version, or invalid profile yields a structured + failure suitable for a Narratio diagnostic naming the affected artifact. + +## Scope boundary + +Narratio owns its source catalog, stage ordering, source-to-input bindings, +artifact dependency graph, and session availability checks. Scriptorium only +needs to describe its prompt requirements through this public interface. +Narratio source IDs and pipeline concepts need not appear in the Scriptorium +API. A graph renderer, HTTP endpoint, batch operation, and changes to generation +behavior are outside this request.