scriptorium/docs/roadmap/implementation.md

# Prompt Cache Control Implementation Plan

## Scope

This plan implements the feature defined in `docs/roadmap/cache.md`.

The implementation target is message-level Anthropic/OpenRouter-style cache control for rendered prompt messages, plus cache usage observability. Do not implement deferred roadmap items such as `content_blocks`, top-level `cache_control`, or general `extra_params` serialization.

Follow the policy documents under `docs/policy/` while implementing:

- Keep prompt/profile/HTTP/config decoding strict.
- Keep adapters thin.
- Keep provider request serialization in `internal/llm`.
- Keep prompt loading in `internal/promptdef`.
- Update canonical non-roadmap docs only when the behavior is implemented.
- Do not accept or emit raw secret values.

## Stage 1: Domain Types And Prompt Loader

Objective: make prompt YAML accept and validate optional `messages[].cache_control`, and carry normalized cache-control metadata in domain prompt definitions.

Files to update:

- `internal/domain/domain.go`
- `internal/promptdef/filesystem_repository.go`
- `internal/promptdef/repository_test.go`
- `internal/promptdef/testdata/`

Domain decisions:

- Add a typed cache-control value in `internal/domain`.
- Use a pointer on message structs so absence is distinguishable from an empty object.
- Use `omitempty` on JSON tags for the new field so existing no-cache prompt and render hashes do not gain `null` fields.

Use this domain shape:

```go
type CacheControlType string

const (
    CacheControlEphemeral CacheControlType = "ephemeral"
)

type CacheControl struct {
    Type CacheControlType `yaml:"type" json:"type"`
    TTL  string           `yaml:"ttl,omitempty" json:"ttl,omitempty"`
}
```

Add cache control to:

```go
type PromptMessageTemplate struct {
    Role         string        `yaml:"role"`
    Content      string        `yaml:"content"`
    ContentFile  string        `yaml:"content_file"`
    CacheControl *CacheControl `yaml:"cache_control,omitempty" json:"cache_control,omitempty"`
}
```

Prompt loader changes:

- Add this field to `promptMessageFile`:

```go
CacheControl *cacheControlFile `yaml:"cache_control"`
```

- Add a private `cacheControlFile` struct with `type` and `ttl`.
- In `normalizePromptDefinition`, validate `cache_control` per message.
- Trim whitespace from `type` and `ttl`.
- Reject present `cache_control` when `type` is empty.
- Reject any `type` other than `ephemeral`.
- Accept omitted or empty `ttl`.
- Reject any non-empty `ttl` other than `1h`.
- Copy normalized values into `domain.PromptMessageTemplate.CacheControl`.
- Preserve the existing rule that each message must set exactly one of `content` or `content_file`.

Testing requirements:

- Add a valid fixture with `cache_control: {type: ephemeral, ttl: 1h}`.
- Add a valid fixture with `cache_control: {type: ephemeral}`.
- Add invalid fixtures for:
  - empty `cache_control.type`
  - unsupported `cache_control.type`
  - unsupported `cache_control.ttl`
  - unknown field under `cache_control`
- Assert valid fixtures load with the expected `CacheControl` values.
- Assert invalid fixtures fail with `promptdef.ErrInvalidPromptDefinition` or `promptdef.ErrInvalidYAML` as appropriate.
- Re-run:

```bash
go test ./internal/promptdef
```

Do not update renderer, formatter, LLM serialization, adapters, or user-facing docs in this stage.

## Stage 2: Rendering, Prepared Output, And Render Hashing

Objective: preserve cache-control metadata through rendering, expose it in prepared-run output, and make `rendered_prompt_hash` reflect outbound-affecting cache metadata.

Files to update:

- `internal/domain/domain.go`
- `internal/prompt/go_renderer.go`
- `internal/prompt/renderer_test.go`
- `internal/usecase/runner.go`
- `internal/usecase/runner_test.go`
- `internal/domain/prepared_run_test.go`
- `internal/format/prepared_run.go`
- `internal/format/prepared_run_test.go`

Domain changes:

- Add cache control to rendered messages:

```go
type RenderedMessage struct {
    Role         string        `json:"role"`
    Content      string        `json:"content"`
    CacheControl *CacheControl `json:"cache_control,omitempty"`
}
```

Renderer changes:

- When a `PromptMessageTemplate` has `CacheControl`, copy it to the returned `RenderedMessage`.
- Do not mutate the template's pointer in place.
- Add and use a small helper such as `cloneCacheControl` to avoid aliasing.
- Do not change template execution behavior for `{{input "name"}}` or `{{.var}}`.

Rendered hash decision:

- `rendered_prompt_hash` must change when cache-control metadata changes, because the outbound provider request changes.
- Preserve existing hashes for no-cache prompts by keeping the current role/content hash flow.
- Append normalized cache-control metadata only when present, for example:
  - `cache_control.type=<type>`
  - `cache_control.ttl=<ttl>` only when non-empty
- Include cache-control fields in a stable order.

Prepared-run formatting:

- JSON prepared-run output should include `cache_control` on messages only when present.
- Text prepared-run output should show cache-control metadata under the specific message, before `content`, for example:

```text
      cache_control: ephemeral ttl=1h
      content: |
```

- If `ttl` is empty, omit the `ttl=...` suffix.
- Do not print cache-control lines for messages without cache control.

Testing requirements:

- Renderer test: cache control is copied to rendered messages.
- Renderer test: changing returned rendered cache control does not mutate the source template.
- Runner test: `rendered_prompt_hash` changes when cache control differs and remains stable for identical rendered messages.
- Format JSON test: `cache_control` appears only when present.
- Format text test: cache control appears under the correct message.
- Existing render/format tests still pass.
- Re-run:

```bash
go test ./internal/prompt ./internal/usecase ./internal/domain ./internal/format
```

Do not update outbound LLM serialization or adapter response surfaces in this stage.

## Stage 3: Outbound Chat Serialization And Usage Parsing

Objective: serialize cache-controlled messages as one-item text content-block arrays and parse cache usage fields from compatible provider responses.

Files to update:

- `internal/llm/openai_compatible_client.go`
- `internal/llm/openai_compatible_client_test.go`
- `internal/domain/domain.go`

Request serialization decisions:

- Keep string content for messages without cache control.
- Use content blocks only when `RenderedMessage.CacheControl` is non-nil.
- Do not introduce `content_blocks` prompt syntax.
- Do not serialize top-level `cache_control` or `extra_params`.
- Split request and response wire message structs. The current `openAIChatMessage` is used in both request and response paths; changing request content to `any` should not make response decoding less strict than necessary.

Use request wire types equivalent to:

```go
type openAIChatRequestMessage struct {
    Role    string `json:"role"`
    Content any    `json:"content"`
}

type openAIChatTextContentBlock struct {
    Type         string               `json:"type"`
    Text         string               `json:"text"`
    CacheControl *openAICacheControl  `json:"cache_control,omitempty"`
}

type openAICacheControl struct {
    Type string `json:"type"`
    TTL  string `json:"ttl,omitempty"`
}
```

When cache control is present, content should serialize as:

```json
[
  {
    "type": "text",
    "text": "rendered text",
    "cache_control": {
      "type": "ephemeral",
      "ttl": "1h"
    }
  }
]
```

When `ttl` is empty, omit `ttl`.

Usage parsing decisions:

- Extend `domain.TokenUsage`:

```go
type TokenUsage struct {
    PromptTokens     int
    CompletionTokens int
    TotalTokens      int
    CachedTokens     int
    CacheWriteTokens int
}
```

- Parse `usage.prompt_tokens_details.cached_tokens` into `CachedTokens`.
- Parse `usage.cache_write_tokens` into `CacheWriteTokens`.
- Treat absent cache fields as zero.
- Do not fail response decoding if cache fields are absent.
- If a provider later returns a different cache-write field, add it in a later change after observing a real response.

Testing requirements:

- Existing request serialization test still sees string message content for no-cache messages.
- New request serialization test sees content-block array for cache-controlled messages.
- New request serialization test verifies empty `ttl` is omitted.
- Response parsing test verifies cached and cache-write tokens are mapped.
- Response parsing test verifies absent cache usage fields remain zero.
- Existing structured-output and service-tier behavior remains unchanged.
- Re-run:

```bash
go test ./internal/llm
```

Do not update CLI or HTTP observability in this stage unless required by compilation after extending `TokenUsage`.

## Stage 4: CLI And HTTP Cache Usage Observability

Objective: surface parsed cache usage through existing CLI and HTTP result paths without adding request knobs.

Files to update:

- `internal/adapter/cli/run.go`
- `internal/adapter/cli/run_test.go`
- `internal/adapter/http/dto.go`
- `internal/adapter/http/handler.go`
- `internal/adapter/http/handler_test.go`

CLI decisions:

- Do not add CLI flags.
- Keep success summaries concise.
- Preserve the existing summary output when cache usage is zero.
- When either cache field is non-zero, append:

```text
cached_tokens=<n> cache_write_tokens=<n>
```

HTTP decisions:

- Do not add request fields.
- Add numeric fields to `metadata.usage`:
  - `cached_tokens`
  - `cache_write_tokens`
- Always include these fields as numbers, matching the current non-omitempty numeric usage fields.
- Continue omitting raw model output unless `include_raw_output` is true.

Testing requirements:

- CLI summary omits cache fields when both are zero.
- CLI summary includes cache fields when either is non-zero.
- HTTP success response includes `metadata.usage.cached_tokens`.
- HTTP success response includes `metadata.usage.cache_write_tokens`.
- Existing HTTP request mapping is unchanged.
- Re-run:

```bash
go test ./internal/adapter/cli ./internal/adapter/http
```

## Stage 5: Canonical Documentation

Objective: update implemented-behavior docs after the feature works and tests pass.

Files to update:

- `docs/config.md`
- `docs/integrations/openai-compatible-chat.md`
- `docs/cli.md`
- `docs/troubleshooting.md`
- `docs/internal/runner.md`
- `docs/internal/adapters.md`, if adapter response behavior needs an explicit note
- `docs/roadmap/cache.md`, only if the accepted target changes during implementation

Documentation requirements:

- Keep all unimplemented ideas under `docs/roadmap/`.
- Document `messages[].cache_control` in the prompt definition reference in `docs/config.md`.
- Document the outbound message serialization rule in `docs/integrations/openai-compatible-chat.md`.
- Document parsed cache usage fields in `docs/integrations/openai-compatible-chat.md`.
- In `docs/cli.md`, note that prompt cache control is configured in prompt YAML, not with CLI flags.
- In `docs/troubleshooting.md`, add cache miss guidance:
  - stable context should appear before the cache breakpoint
  - dynamic input before the breakpoint changes the cache key
  - providers may impose minimum token thresholds and breakpoint limits
  - use cached/cache-write token counts to verify behavior
- Do not claim support for deferred roadmap items.

Testing and verification:

```bash
go test ./...
```

If documentation examples include new YAML snippets, ensure they match the strict prompt loader schema.

## Stage 6: Final Verification

Objective: verify the whole change set end to end.

Required commands:

```bash
go test ./...
go build ./cmd/scriptorium
```

Recommended smoke checks:

```bash
go run ./cmd/scriptorium render \
  --config ./examples/config.yml \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md \
  --input glossary=./examples/fixtures/glossary.yml \
  --format json
```

If adding a new example prompt with `cache_control`, also smoke-render that prompt with `--format json` and inspect that the rendered message includes `cache_control`.

Final review checklist:

- Existing prompt definitions without `cache_control` still load.
- Prompt definitions with `cache_control` render successfully.
- Unknown cache-control YAML fields are rejected.
- No-cache outbound messages still serialize with string `content`.
- Cache-controlled outbound messages serialize with text content-block arrays.
- `rendered_prompt_hash` changes when cache-control metadata changes.
- Cache usage fields are zero when providers omit them.
- CLI and HTTP request surfaces have no new knobs.
- CLI and HTTP response surfaces expose cache usage.
- Non-roadmap docs describe only implemented behavior.
- `docs/roadmap/cache.md` still defines target state and deferred work, not task steps.

## Stage Boundaries

Stages 1 through 4 can be implemented in separate prompts if needed. Stage 5 should happen only after behavior is implemented. Stage 6 should be performed after all implementation and documentation changes are complete.

If a stage fails because prior stage changes were not implemented, stop and complete the earlier stage first rather than papering over missing domain fields or DTO mappings.

## Open Questions

None required for the initial implementation.

The plan chooses message-level `cache_control`, string content for legacy messages, text-block array content only when cache control is present, and no new CLI/HTTP request knobs. These choices match `docs/roadmap/cache.md` and preserve existing prompt-definition compatibility.