scriptorium/README.md

# scriptorium

Scriptorium is a generic prompt execution engine.

It takes:
- a prompt definition
- a selected or default execution profile
- named input artifacts
- template variables
- optional runtime overrides

It returns:
- generated artifact
- validation result
- metadata

## Prompt vs Profile

Scriptorium separates **what** to do (Prompt) from **how** to do it (Profile).

### Prompt Definition
Defines the task logic and output contract.
- Task description and version.
- Message templates (system, user, etc.).
- Required and optional input artifacts.
- Output format and validation rules.
- Repair settings for structured output.
- Optional `default_profile` for convenience.

### Execution Profile
Defines the runtime environment and model settings.
- LLM endpoint (URL).
- Model name.
- Generation parameters: `temperature`, `max_tokens`, `top_p`.
- Runtime settings: `timeout`, `reasoning_effort`.
- API key source via `api_key_env`.

Callers can explicitly provide a `profile_id` to override the prompt's `default_profile`.

## Precedence

When resolving runtime settings, Scriptorium follows this precedence model (highest to lowest):

1. **Runtime Overrides**: Provided via CLI flags or HTTP request `model` object.
2. **Execution Profile**: Settings defined in the selected profile.
3. **Application Defaults**: Built-in fallback values.

### Profile Selection Logic
The engine determines which profile to use in this order:
1. Explicit `profile_id` (via `--profile` or HTTP request).
2. The `default_profile` named in the Prompt Definition.
3. Error: If neither is provided and no default exists.

## API Key Policy

To ensure security, Scriptorium does not support raw API keys in configuration files, CLI arguments, or HTTP requests.

- **`api_key_env`**: Profiles and overrides specify the name of an environment variable (e.g., `SCRIPTORIUM_API_KEY`).
- **Runtime Resolution**: The value of the environment variable is read directly from the process environment at runtime.
- **Zero Leakage**: API key values are never included in metadata, logs, or response bodies.

## CLI Usage

### `scriptorium run`

Runs a single prompt execution.

**Required Flags:**
- `--prompt-dir`: Directory containing prompt YAML files.
- `--profile-dir`: Directory containing profile YAML files.
- `--prompt`: The prompt ID to execute.
- `--input`: Input mapping `name=path` (repeatable).

**Optional Flags:**
- `--profile`: Override the prompt's default profile.
- `--var`: Template variable `name=value` (repeatable).
- `--out`: Write output to a file instead of stdout.
- `--llm-base-url`: Override endpoint.
- `--model`: Override model name.
- `--api-key-env`: Override API key environment variable name.
- `--temperature`: Override temperature.
- `--max-tokens`: Override max tokens.
- `--top-p`: Override top_p.
- `--timeout`: Override request timeout (e.g., `30s`, `1m`).

**Examples:**

Using the prompt's `default_profile`:
```bash
export SCRIPTORIUM_API_KEY="sk-..."
scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md
```

Overriding the profile:
```bash
scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --profile local-quality \
  --input transcript=./examples/fixtures/transcript.md
```

Overriding model and runtime values:
```bash
scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --model gpt-4o \
  --temperature 0.7 \
  --input transcript=./examples/fixtures/transcript.md
```

Using a local OpenAI-compatible vLLM endpoint:
```bash
scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --llm-base-url http://localhost:8000/v1 \
  --model meta-llama-3-8b \
  --input transcript=./examples/fixtures/transcript.md
```

### `scriptorium serve`

Starts the HTTP API.

**Required Flags:**
- `--prompt-dir`: Directory containing prompt YAML files.
- `--profile-dir`: Directory containing profile YAML files.

**Optional Flags:**
- `--addr`: Listen address (default `:8080`).
- `--schema-dir`: Base directory for validation schemas.
- `--model`: Default model override.
- `--timeout`: Default request timeout.

## HTTP API

### `POST /v1/runs`

Executes a prompt. No built-in authentication is provided; deploy behind a trusted gateway.

**Request Body:**
```json
{
  "prompt_id": "generic.structured_events",
  "profile_id": "local-quality",
  "inputs": {
    "transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"}
  },
  "vars": {
    "session_date": "2026-05-04"
  },
  "model": {
    "endpoint": "http://localhost:8000/v1",
    "model": "gpt-4o-mini",
    "temperature": 0.0
  }
}
```

**Response:**
Returns a `200 OK` with the generated artifact, validation results, and metadata including the `prompt_id` and the `selected_profile_id`.

**Validation Failures:**
If the model output fails validation (e.g., invalid JSON), the API returns `200 OK` with `validation.status = "failed"`. The original `raw_model_output` is preserved in the response to allow debugging.

## Prompt Definition Authoring

Prompts are defined in YAML.

### Canonical Shape
```yaml
id: generic.structured_events
version: "1.0.0"
description: "Extracts structured events from a transcript"
default_profile: local-quality

inputs:
  - name: transcript
    required: true
    description: "The raw session transcript"
  - name: glossary
    required: false

templates:
  - role: system
    content: "You are a helpful assistant."
  - role: user
    content_file: messages/extract_events.tmpl

output_format: json
validation:
  validation_mode: json_schema
  schema_path: structured_events.schema.json
  repair_attempts: 2
```

**Key Features:**
- **Inline vs File**: Use `content` for short prompts or `content_file` for larger templates.
- **Inputs**: Mark inputs as `required` to ensure the runner fails early if they are missing.
- **Validation**: Support `none`, `basic`, `json`, and `json_schema`.
- **Repair**: `repair_attempts` enables bounded retries to fix structured output.

## Execution Profile Authoring

Profiles are defined in YAML.

### Canonical Shape
```yaml
id: local-quality
endpoint: http://localhost:8000/v1
model: gpt-4o
temperature: 0.0
max_tokens: 4096
top_p: 1.0
timeout_seconds: 300
reasoning_effort: high
api_key_env: SCRIPTORIUM_API_KEY
```

**Constraints:**
- **No Raw Keys**: Do not include actual API keys. Only specify the environment variable name in `api_key_env`.
- **Local Profiles**: For local endpoints that don't require auth, `api_key_env` can be omitted.

## Examples

- **Prompt Definitions**: `prompts/`
- **Execution Profiles**: `profiles/`
- **Schemas**: `schemas/`
- **Fixtures**: `examples/fixtures/`
- **Local Experimentation**: `local-test/`

## Build and Test

```bash
go build -o scriptorium ./cmd/scriptorium
go test ./...
```