2026-05-23 04:51:16 +00:00
|
|
|
# Stage: publish
|
2026-05-18 22:03:59 -05:00
|
|
|
|
|
|
|
|
## Purpose
|
2026-08-09 21:50:21 +00:00
|
|
|
|
2026-05-23 12:57:59 +00:00
|
|
|
Upload run/session outputs to object storage and atomically advance remote current state.
|
|
|
|
|
|
|
|
|
|
## Inputs
|
2026-08-09 21:50:21 +00:00
|
|
|
|
|
|
|
|
- successful preceding stages from the [canonical stage set](overview.md#pipeline-stage-set)
|
|
|
|
|
- invocation-scoped run files
|
|
|
|
|
- resolved publish output rules
|
2026-08-10 20:17:54 +00:00
|
|
|
- effective publish locks (static + remote merged lock set), revalidated at the
|
|
|
|
|
remote commit boundary
|
2026-08-09 21:50:21 +00:00
|
|
|
- durable previous-session cache files when present
|
2026-05-23 12:57:59 +00:00
|
|
|
|
|
|
|
|
## Outputs
|
2026-08-09 21:50:21 +00:00
|
|
|
|
|
|
|
|
- uploaded invocation record and selected publish outputs;
|
|
|
|
|
- uploaded durable previous-session cache files when present;
|
2026-08-10 20:05:24 +00:00
|
|
|
- immutable run-scoped commit manifest; and
|
|
|
|
|
- current commit pointer, written last.
|
2026-08-09 21:50:21 +00:00
|
|
|
|
|
|
|
|
Exact remote placement and the operator workflow belong in
|
|
|
|
|
[Operations](../operations.md#publish-workflow).
|
2026-05-23 12:57:59 +00:00
|
|
|
|
|
|
|
|
## Key Behavior
|
2026-08-09 21:50:21 +00:00
|
|
|
|
2026-05-23 12:57:59 +00:00
|
|
|
- stage can self-skip when publish disabled or run upload disabled.
|
|
|
|
|
- validates prerequisite stage success and object-store availability.
|
2026-08-10 19:16:18 +00:00
|
|
|
- derives a deterministic run-archive allowlist from the validated run
|
|
|
|
|
`manifest.json`: declared run-local outputs, logs, generated configs, and the
|
|
|
|
|
manifest itself. Unlisted workspace files are not archive candidates.
|
|
|
|
|
- opens each archive candidate beneath its archive root without following
|
|
|
|
|
symlinked ancestors or leaf entries, verifies that it is a regular file and
|
|
|
|
|
checks a declared checksum when present, then streams the opened descriptor.
|
|
|
|
|
- derives the durable previous-cache archive from its validated manifest using
|
|
|
|
|
the same confinement and regular-file checks.
|
2026-05-23 12:57:59 +00:00
|
|
|
- resolves publish output sources through runtime artifact catalog and manifest-aware resolution.
|
2026-08-10 02:10:37 +00:00
|
|
|
- publishes extraction lanes only through explicit configured output rules;
|
|
|
|
|
neither run-local nor durable Notarius bundles are scanned or uploaded wholesale.
|
2026-05-23 12:57:59 +00:00
|
|
|
- selected artifact filter applies to configured artifact sources only.
|
|
|
|
|
- locked outputs are skipped intentionally (including required ones).
|
|
|
|
|
- optional missing outputs are skipped; required missing unlocked outputs fail.
|
2026-08-10 20:05:24 +00:00
|
|
|
- creates one complete immutable source-to-destination mapping before upload;
|
|
|
|
|
- uploads and verifies every declared immutable object and the commit manifest;
|
|
|
|
|
- updates `current/commit-pointer.json` exactly once, last; and
|
|
|
|
|
- does not write the legacy `current/manifest.json` or `current/run_id.txt` pair.
|
2026-08-10 20:17:54 +00:00
|
|
|
- rechecks remote lock state immediately before the pointer update. A newly
|
|
|
|
|
committed lock aborts selection, leaving any uploaded immutable attempt
|
|
|
|
|
unselected.
|
2026-05-23 12:57:59 +00:00
|
|
|
|
|
|
|
|
## Metadata Signals
|
2026-08-09 21:50:21 +00:00
|
|
|
|
2026-05-23 12:57:59 +00:00
|
|
|
Includes counts/lists for:
|
|
|
|
|
- run uploads
|
|
|
|
|
- published output uploads
|
|
|
|
|
- previous uploads
|
|
|
|
|
- skipped optional outputs
|
|
|
|
|
- skipped unselected outputs
|
|
|
|
|
- locked outputs
|
2026-08-10 20:05:24 +00:00
|
|
|
- remote commit and current-pointer key paths
|
2026-08-10 20:29:00 +00:00
|
|
|
- the run identifier selected by the commit
|
2026-05-23 12:57:59 +00:00
|
|
|
|
|
|
|
|
## Invariants
|
2026-08-09 21:50:21 +00:00
|
|
|
|
2026-08-10 20:05:24 +00:00
|
|
|
- `current/commit-pointer.json` is the remote commit marker and is written last.
|
|
|
|
|
- run files, selected outputs, previous-cache files, and the committed session
|
|
|
|
|
manifest are all declared by an immutable commit under the run prefix.
|
2026-08-10 19:16:18 +00:00
|
|
|
- run and previous uploads contain only manifest-declared regular files opened
|
|
|
|
|
from verified descriptors; symlinks, special files, replacement races, and
|
|
|
|
|
undeclared entries are rejected or ignored before uploads begin.
|
|
|
|
|
- run-local diagnostics, including Notarius receipt and stderr files, are
|
|
|
|
|
archived only when recorded by the run manifest.
|
2026-08-10 20:17:54 +00:00
|
|
|
- publish locks are not overridden by `--force`; remote locks are revalidated
|
|
|
|
|
immediately before current-state selection.
|
2026-08-10 20:29:00 +00:00
|
|
|
- post-commit local cleanup is authorized by the committed publish metadata and
|
|
|
|
|
is durably recorded by the application lifecycle before any local deletion.
|
2026-08-09 21:50:21 +00:00
|
|
|
|
|
|
|
|
The commit boundary and cleanup gate are normative architecture invariants; see
|
|
|
|
|
[Architecture](../policy/architecture.md#publish-commit-boundary).
|
|
|
|
|
|
|
|
|
|
## Related Contracts And Tests
|
|
|
|
|
|
|
|
|
|
- [Configuration](../config.md#publish-configuration-summary) owns output and
|
|
|
|
|
static-lock fields.
|
|
|
|
|
- [Operations](../operations.md#publish-locks) owns remote lock lifecycle and
|
|
|
|
|
physical remote state.
|
|
|
|
|
- [Artifact Internals](artifacts.md) explains source resolution and current-state
|
|
|
|
|
helpers.
|
|
|
|
|
- Implementation and tests: `internal/stage/publish.go`,
|
|
|
|
|
`internal/stage/publish_test.go`, `internal/app/operator_helpers_test.go`,
|
|
|
|
|
`internal/app/post_publish_cleanup_test.go`
|