Retire the legacy CLI and document native operator workflows

This commit is contained in:
2026-11-20 05:08:40 +00:00
parent 10387d4aea
commit bd7136ceeb
52 changed files with 846 additions and 7067 deletions

View File

@@ -1,243 +1,120 @@
# Operations Guide
## Scope And References
This runbook covers deployment, normal operation, capacity, sensitive data, and
recovery. Exact [CLI syntax/statuses](cli.md), [configuration](config.md), and
[HTTP wire behavior](api.md) remain in their canonical references.
See the [CLI migration guide](consumers/upgrading-cli.md) for changed jobs and
the [framework upgrade guide](consumers/upgrading-promptkit.md) for definitions.
This runbook covers deployment, normal operation, capacity planning, and safe
recovery for Scriptorium. It does not redefine invocation syntax, configuration
fields, or HTTP wire behavior.
## Deploy Sources And Process
- [CLI reference](cli.md): commands, output destinations, and exit codes.
- [Configuration reference](config.md): application settings, source
locations, defaults, and credential mapping.
- [Promptkit source upgrade guide](consumers/upgrading-promptkit.md): concise
source-selector and definition migration examples.
- [PromptKit v0.23.3 framework formats](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/804a4d32b23aa2e7973400bb0a322a7016574c9f/docs/formats.md):
prompt, profile, schema, execution-setting, and framework credential
contracts.
- [HTTP API reference](api.md): route, request/response schema, status codes,
limits, and HTTP artifact access.
- [Consumer integration overview](consumers/api.md): caller responsibilities.
Provide readable application configuration, its selected framework entry file,
and explicitly imported definitions/resources. Set cwd deliberately: application
path values are cwd-relative, while framework imports and message resources keep
their native source contexts. ConfigKit confines source imports/overlays to the
selected main's canonical root; Scriptorium does not widen it.
## Operational Model And State
Engine construction validates selected effective definitions and captures message
resources/template syntax, including definitions not used by a particular request.
Schema/artifact reads, credential resolution, and model work retain their native
preparation/runtime lifecycles. A server constructs one engine at startup and
does not hot-reload changed files; restart to capture updates. Separate CLI
invocations load afresh rather than sharing an immutable snapshot.
Scriptorium handles one prompt request for each CLI invocation or HTTP request.
It has no durable run store, archive, checkpoint, cache, or resume mechanism.
A failed or interrupted request is recovered by correcting its inputs,
configuration, or environment and submitting a new request.
For independent collections, install checkouts beneath the layout selected by
the [discovery entry](../examples/discovery/framework.yml). Each checkout declares
its namespace and explicit fragment imports; arbitrary YAML is not a framework
source. The maintained [discovery example](../examples/config.discovery.yml)
supports both entry-file spellings and multiple definitions. New checkouts or
updates become visible to later invocations or a restarted server.
Generated artifacts, rendered prompts, model output, and run metadata are
caller-owned data. Retention, encryption, backup, and deletion are deployment
responsibilities.
Optional [CLI execution reports](cli.md) also belong to the caller. They can
contain sensitive identifiers even though they omit prompt and response bodies.
Keep report paths separate from generated output and apply the same access and
retention controls. Output and report writes are not transactional; an older
report may remain when no terminal observation was produced.
HTTP callers may also request a [terminal execution report](api.md#execution-report).
It shares the same identifier sensitivity and should receive response access
and retention controls. An opted-in report increases the encoded response size;
an oversized success or report-bearing error becomes the compact 413 response
without a report.
Supply only needed credential environment variables through a supervisor or
secret-management mechanism. Confirm presence without printing their values.
Raw keys do not belong in arguments or checked-in configuration.
## Deploy The Filesystem And Process
## Normal Operator Workflow
Provide the process with readable selected Promptkit source files/directories
and schema sources that follow the tagged framework formats. At engine
construction, PromptKit v0.23.3 validates every selected effective definition,
including unused definitions, and captures referenced message-file contents
and static template syntax. `serve` constructs one engine at startup; it does
not reload changed configuration or message content. Restart the process to
validate and capture a new configuration. Schema and request-artifact reads,
credential resolution, and provider work remain request-preparation or runtime
concerns. For an HTTP deployment that accepts file artifacts, use a dedicated,
narrow artifact directory rather than a general-purpose or sensitive filesystem
tree.
1. Validate application configuration through the native application scope.
This does not acquire framework sources or prove provider readiness.
2. Inspect framework configuration/resources and discover requirements for the
selected prompt/profile/namespaces. Discovery can precede generated input
artifacts; completeness does not establish satisfied bindings.
3. Once artifacts exist, validate the concrete request or inspect prepared
content. These are provider-free preparation checks with native prerequisites.
4. Invoke a smoke test only when its provider effects are intended. Findings
can be unsupported or unsuccessful while the command itself succeeds.
Smoke tests are not an automatic startup or release prerequisite.
5. Generate with the same selections and bindings, then inspect the finalized
status and any returned validation result before accepting the output.
If application configuration uses explicit imports or a selected overlay,
deploy those files beneath the selected configuration file's canonical
directory. ConfigKit keeps imported and overlay files within that root;
Scriptorium does not widen it. Application path values inside those files still
resolve from the process working directory, so set that directory deliberately
in the service manager. This configuration-source boundary is distinct from
the HTTP artifact root and does not constrain prompt or schema directories.
Use the [maintained preview script](../examples/render-markdown-summary.sh)
for a provider-free example, and the [CLI contract](cli.md) for commands.
A complete discovery report can still list required inputs that are not bound;
a partial report can still be useful. Consumers own their policy for incomplete
findings and changed resources between invocations.
For independently maintained collections, place checkouts beneath the
directory selected by your framework discovery file. The maintained
[discovery example](../examples/config.discovery.yml) selects optional
`collection.yml` and `collection.yaml` entry files; each checkout owns its
namespace and explicit fragment imports. Update a checkout independently and
invoke the CLI again, or restart a running server to capture the update. Do
not treat arbitrary YAML in a checkout as a framework source. When upgrading
definitions, review exact profile IDs and replace retired catalog references
as described in the [migration guide](consumers/upgrading-promptkit.md).
## Sensitive Data And Retention
Run Scriptorium under an identity that can:
Artifacts, prompt messages, prepared content, native run reports, metadata,
diagnostics, and redirected streams are caller-owned potentially sensitive data.
The separate CLI report contains the full returned result, not metadata alone.
Control access, retention, encryption, backup, and deletion accordingly.
Framework effective/explanation views apply native classified-field redaction,
including history, but do not detect arbitrary secrets. Other content-bearing
reports are not automatically made safe for publication.
- read only the prompt, profile, schema, and allowed input-artifact paths it
needs;
- read the required credential environment variables without writing them to
files or logs; and
- write only caller-selected output locations when CLI output files are used.
Keep main output and report destinations separate. Delivery is nontransactional:
one destination or partial bytes can remain after failure. An existing report
can remain on a resultless error; check status and report identity, not just file
existence. A shell can truncate a redirected destination before the child starts.
Omit quiet if main stdout must be captured; the separate report effect remains
independent. Exact delivery behavior belongs to [CLI results](cli.md#results-and-separate-reports).
Do not make the HTTP artifact directory writable by untrusted users. The HTTP
artifact containment behavior is lexical and the operating system follows
symlinks; account for that when choosing ownership and mount boundaries. See
the [HTTP API reference](api.md) for the externally observable behavior.
HTTP's optional [execution report](api.md#execution-report) remains a distinct
terminal-observation projection. Protect its identifiers and any response data.
It increases encoded response size and remains subject to the configured limit.
## Supply Credentials And Protect Runtime Data
## HTTP Deployment And Shutdown
Set secret values in the process environment and configure only their
environment-variable names. Do not put raw keys in configuration, prompt or
profile files, process arguments, HTTP payloads, captured command lines, or
debug dumps.
The service has no built-in authentication/authorization. Deploy on a trusted
network or behind appropriate authenticated proxy/gateway controls. Use a narrow,
dedicated artifact root with controlled ownership and permissions when accepting
file inputs; this is lexical containment, not a symlink sandbox.
Treat stdout, stderr, prepared-run output, generated artifacts, and HTTP
responses as potentially sensitive. Send service logs to a controlled collector
and apply the same retention and access rules as for model input and output.
Use a supervisor for cwd, environment, restart policy, and stream capture.
Startup information uses native Info reporting, not stdout banners.
Interrupt/SIGTERM cancels the invocation and closes the listener and active
connections. Canceled shutdown is nonzero operational termination. There is no
graceful-drain deadline or promise that context-ignoring work has finished;
supervisors own forced termination when necessary.
## Run A Normal Workflow
## Capacity, Limits, And Recovery
Before changing production inputs, profiles, or schemas:
One server engine shares backend admission/queue state across concurrent requests.
That state is transient operational coordination, not a durable workflow or
queue of resumable runs. HTTP capacity exhaustion retains its
[API outcome](api.md); callers decide whether another model call is safe.
1. use `config validate`, `config show`, or `config sources` to check
Scriptorium's application configuration; these do not open Promptkit sources
or construct an engine;
2. use `inspect configuration validate` to construct the engine and validate
all selected framework definitions; this does not prepare a request, check
credentials, load request-time schemas/artifacts, or contact a provider;
3. confirm the deployed configuration selects the intended sources and
environment-variable names;
4. use [`render`](cli.md) with the same request inputs and variables to confirm
preparation without a model call;
5. use [`run`](cli.md) for generation; and
6. retain or discard validation-failed output according to the caller's
policy.
Plan for input/output sizes, schema complexity, provider latency, retries, and
repair costs. Native transient attempts and repair generations can add calls and
waiting; engine defaults and per-request overrides belong to
[configuration](config.md) and the [CLI](cli.md)/[HTTP](api.md) contracts.
Whole-operation budgets are distinct from per-attempt generation timeouts.
They remain cooperative and do not include configuration/setup or delivery.
Application configuration inspection can include paths and configured values.
Framework `inspect configuration show` and `explain` use Promptkit's native
redaction for classified credential-shaped provider fields, including
historical values, while leaving environment-reference names and ordinary
provider parameters visible. Neither report identifies arbitrary sensitive
content by value or name. Protect inspection output accordingly; Scriptorium
stores environment-variable names, not the resolved credential bytes.
Before increasing HTTP byte limits, measure representative requests/artifacts/
responses, confirm memory/network/provider capacity, retain authentication and
upstream request controls, and test a representative workload. Do not widen the
artifact root just to make an arbitrary host path accessible.
The [maintained render script](../examples/render-markdown-summary.sh) is a
copyable preflight example. The CLI reference owns its complete invocation and
exit semantics.
For failures, capture diagnostics without publishing sensitive data. Verify
selected configuration, resource identity, readable inputs, profile/credentials,
and schema deployment. Use native prepared preview to isolate pre-generation
problems. A completed content-validation failure is distinct from an operational
failure to validate; consumers choose whether failed content is usable.
Correct the cause, then issue a new request only when another provider call and
its cost are acceptable.
## Expose The HTTP Service
The HTTP service has no built-in authentication or authorization. Place it on a
trusted network or behind an authenticated reverse proxy, API gateway, or
equivalent access control. Restrict who can reach it and who can read the
artifact root.
Use a service manager or supervisor appropriate to the deployment to manage
process lifetime, restart policy, log capture, and environment injection. The
[HTTP API reference](api.md) owns client request shapes, status behavior, and
artifact-access outcomes.
## Plan Capacity And Limits
Capacity is primarily determined by concurrent model calls, input and output
sizes, schema complexity, provider latency, and network behavior. Size limits
protect request bodies, HTTP file artifacts, and encoded responses; configure
them through the [configuration reference](config.md) and rely on the
[HTTP API reference](api.md) for their response effects.
Configured backend concurrency and queue capacity are enforced per constructed
Promptkit engine. A `serve` process constructs one engine for its handler, so
concurrent HTTP requests share that transient admission state. Scriptorium does
not retain workflow state: capacity is neither durable nor a queue of resumable
runs. When admission is exhausted, HTTP returns `503 capacity_exceeded` without
retry timing; callers choose any retry policy that is safe for another model
call.
Promptkit's built-in client now allows two additional attempts for eligible
transient provider statuses by default. Account for added calls, latency, and
cost when planning capacity. Each repair generation has its own transient
budget, separate from the structural repair count. Configure the engine budget
with [`transient_retries`](config.md); `run` and `render` accept a request
override described in the [CLI contract](cli.md), and HTTP callers can select
one through the [API contract](api.md). Omission inherits the engine
setting and then Promptkit's default, while explicit zero disables additional
attempts. Existing generation timeouts apply
per attempt, while the caller context covers the complete operation. Provider
`Retry-After` timing can exceed Promptkit's ordinary backoff cap; consult the
[outbound integration contract](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/804a4d32b23aa2e7973400bb0a322a7016574c9f/docs/integrations/openai-compatible-chat.md)
for details.
CLI `run` can set a [whole-run deadline](cli.md) when total execution time must
be bounded. The existing generation timeout applies afresh to each attempt and
does not cap retry waits or the complete invocation. The whole-run interval
covers the engine call, including capacity waiting and repairs as supported by
context cancellation; it excludes parsing, engine setup, and output writes.
HTTP callers can use [`run_timeout_seconds`](api.md) to set the engine's
operation budget for the same interval. The API distinguishes a typed expired
execution deadline from caller cancellation and from a provider-attempt
timeout. A canceled capacity waiter releases its Promptkit admission state.
Before increasing a limit:
1. measure representative input, generated-output, and optional raw-output
sizes;
2. confirm memory, network, and upstream-provider capacity;
3. retain an upstream request-size and authentication boundary; and
4. test the intended workload in a non-production environment.
For large local inputs, prefer a controlled file-artifact directory over
placing arbitrary paths on the service host. Avoid disabling a limit unless an
equivalent trusted control exists elsewhere.
## Diagnose And Recover
### Preparation Or Configuration Failure
Capture the CLI diagnostic or HTTP error response, then verify the selected
configuration, prompt ID, profile selection, source readability, and input
mapping. Use `render` with the same request when it is unclear whether failure
occurs before model execution. Consult the [CLI reference](cli.md), the
[configuration reference](config.md), and the [HTTP API reference](api.md) for
the exact interface contract.
### Credential Or Provider Failure
Confirm that the process environment contains the configured credential name
without printing the secret. Check endpoint reachability and provider health
from the process network. If preparation succeeds but generation fails, inspect
the selected model settings in prepared output and the service's controlled
logs. Correct the deployment or provider issue, then submit a new request.
### Artifact Or Permission Failure
Verify that the process can read the intended local input. For HTTP file
artifacts, verify the deployment's artifact root, ownership, path layout, and
file size. Do not widen filesystem permissions or the allowed root merely to
make an arbitrary path work; move or copy the required artifact into the
controlled location instead.
### Validation Failure
A generated-content validation failure is distinct from a runtime failure.
CLI `run` reports the validation result and error count in its success summary;
it does not print the individual validation messages. For HTTP, inspect the
validation object in the response according to the [HTTP API reference](api.md).
Use rendered input and generated output to determine whether prompt instructions,
the selected model, or the schema needs correction. If schema loading or
compilation itself fails, correct the source deployment or schema document
before rerunning.
### HTTP Limit Or Request Failure
Compare the request, artifact, or expected response size with the deployed
configuration, and validate the request against the [HTTP API reference](api.md).
Reduce the payload, use an appropriate controlled artifact source, omit
unneeded raw output, or adjust the deployment limit after capacity review.
## Cleanup And Reruns
Because no run state is retained, cleanup concerns caller-owned output files,
logs, and artifacts only. Remove or rotate them using the deployment's normal
retention policy. After a correction, rerun the request from the beginning;
there is no safe resume point.
There is no durable run store, archive, checkpoint, or resume mechanism.
Cleanup concerns caller-owned files/logs only. Rotate/delete them under the
deployment's normal retention policy; recovery starts from a new request.