69 lines
2.5 KiB
Markdown
69 lines
2.5 KiB
Markdown
|
# Stage: archive
|
||
|
|
|
||
|
|
## Purpose
|
||
|
|
Publish run records and promoted session artifacts to object storage, then atomically advance the remote current pointer.
|
||
|
|
|
||
|
|
## Inputs and Outputs
|
||
|
|
Inputs:
|
||
|
|
- session manifest and prerequisite stage records
|
||
|
|
- run root contents under `runs/{run_id}/`
|
||
|
|
- promotion sources from session root (`archive.promote_artifacts`)
|
||
|
|
|
||
|
|
Outputs:
|
||
|
|
- uploaded run files under `{session_prefix}/runs/{run_id}/...`
|
||
|
|
- uploaded promoted artifacts under `{session_prefix}/...`
|
||
|
|
- `{session_prefix}/current/manifest.json`
|
||
|
|
- `{session_prefix}/current/run_id.txt` written last
|
||
|
|
|
||
|
|
## Boundaries
|
||
|
|
Owns:
|
||
|
|
- Archive enable/disable gate behavior
|
||
|
|
- Prerequisite stage success enforcement
|
||
|
|
- Run file collection and upload (excluding `audio/`)
|
||
|
|
- Promotion rule resolution and upload
|
||
|
|
- Commit pointer publish order
|
||
|
|
|
||
|
|
Does not own:
|
||
|
|
- Stage execution before archive
|
||
|
|
- Post-archive local cleanup policy execution (handled by app cleanup logic)
|
||
|
|
|
||
|
|
## Config Fields Used
|
||
|
|
- `pipeline.archive.enabled`
|
||
|
|
- `pipeline.archive.upload_run`
|
||
|
|
- `pipeline.archive.promote_artifacts`
|
||
|
|
- `pipeline.storage.s3.bucket`
|
||
|
|
- `pipeline.storage.s3.root_prefix`
|
||
|
|
- `pipeline.workspace.root`
|
||
|
|
- `session.campaign`
|
||
|
|
- `session.session_id`
|
||
|
|
|
||
|
|
## External Adapters Used
|
||
|
|
- Object storage backend (`env.ObjectStore`) for upload/list primitives.
|
||
|
|
|
||
|
|
## State and Manifest Behavior
|
||
|
|
- Requires `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, and `analyze` status `succeeded`.
|
||
|
|
- Resolves bucket/prefix from manifest identity first, then config fallback.
|
||
|
|
- Writes metadata including:
|
||
|
|
- upload counts/paths
|
||
|
|
- `current_manifest_key`
|
||
|
|
- `current_run_id_key`
|
||
|
|
- `current_pointer_written`
|
||
|
|
- On skipped archive path, returns metadata with `skipped=true` and pointer not written.
|
||
|
|
|
||
|
|
## Skip and Resume Behavior
|
||
|
|
- Stage may self-skip (metadata skip) when archive disabled or run upload disabled.
|
||
|
|
- Runner-level skip also applies for previously succeeded stage unless forced.
|
||
|
|
|
||
|
|
## Failure Behavior
|
||
|
|
- Fails on missing prerequisite success, missing object store when required, missing run root, missing required promotion source, upload failures, or pointer write failures.
|
||
|
|
- Pointer semantics are fail-safe: `current/run_id.txt` is not written if prior required uploads fail.
|
||
|
|
|
||
|
|
## Tests to Inspect Before Changing
|
||
|
|
- `internal/stage/archive_test.go`
|
||
|
|
- `internal/app/post_archive_cleanup_test.go`
|
||
|
|
|
||
|
|
## Architectural Invariants
|
||
|
|
- Run upload excludes `audio/` subtree.
|
||
|
|
- `current/manifest.json` uploads before `current/run_id.txt`.
|
||
|
|
- `current/run_id.txt` is the remote publish commit marker.
|