Each entry lists the symptom, likely cause, diagnostic step, safe fix, and relevant reference link. Command syntax lives in [CLI](cli.md), configuration fields live in [Configuration](config.md), and operating procedures live in [Operations](operations.md).
## Config File Is Missing
Symptom: `load config ... no such file or directory`.
Likely cause: `--config` points to a missing file, or `--config` was omitted and `/usr/local/etc/distributor/config.yml` is not installed.
Symptom: `configured source mode requires --pipeline`, `does not accept a local path with --config, --pipeline, or --bundle`, `validate command requires a path`, or `inspect command requires a path`.
Safe fix: use either `distributor validate <path>` / `distributor inspect <path>`, or use `--config <path> --pipeline <id>` with optional `--bundle <path>`.
Symptom: `reconcile-state requires --config`, `requires --pipeline`, `requires --destination`, `pipeline "<id>" not found`, `destination <id> not found`, or `state owner is ... not ...`.
Likely cause: the command did not identify one configured destination root, or the selected root contains state for a different single-owner pipeline/destination.
Safe fix: pass the configured `--config`, `--pipeline`, and `--destination` values that identify the destination root containing the state file. For unrelated single-owner state, use the correct config selector or a separate destination root; `reconcile-state` will not take over mismatched state.
Symptom: `prune requires --config`, `requires --pipeline`, `requires --destination`, `requires exactly one of --dry-run or --apply`, `pipeline "<id>" not found`, `destination <id> not found`, or `state owner is ... not ...`.
Likely cause: the command did not identify one configured destination root, did not choose exactly one execution mode, or the selected root contains state for a different single-owner pipeline/destination.
Safe fix: pass the configured `--config`, `--pipeline`, and `--destination` values that identify the destination root containing the state file. Use `--dry-run` for read-only review or `--apply` for deletion, but not both. For unrelated single-owner state, use the correct config selector or a separate destination root; `prune` will not take over mismatched state.
Likely cause: the command failed before it could construct a result, such as invalid arguments, missing config, unreadable secrets, or source setup failure.
Likely cause: the source root, source-root-relative bundle path, S3 prefix, SSH path, or permissions do not expose a directory containing `manifest.json`.
Safe fix: correct the configured source backend root, permissions, prefix, or `--bundle` path. The selected bundle directory must contain `manifest.json`.
Safe fix: choose an empty destination path, move unrelated files aside, or preview `run --dry-run --force` only after confirming the reported bundle path is safe to replace.
Symptom: `fail_conflict`, `destination source id differs`, `same id and created time but different digest`, `pipeline id ... does not match`, or `destination id ... does not match`.
Safe fix: verify the source and destination are intended to match. Use a separate destination path for unrelated content. To replace the existing state, configure `transfer.on_conflict: replace`, preview with `--dry-run --force`, then publish with `--force`.
## Destination State References Missing Managed Outputs
Symptom: `reconcile-state --dry-run` reports `status=would_change` or JSON `missing_managed_outputs` entries.
Likely cause: files that were recorded as managed in `.distributor.json` were removed outside `distributor`, or a previous external cleanup removed destination files without updating state.
Diagnostic:
```sh
go run ./cmd/distributor reconcile-state \
--config <config-path> \
--pipeline <pipeline-id> \
--destination <destination-id> \
--dry-run
```
Safe fix: if the missing files should no longer be managed, rerun the same command without `--dry-run` to remove only the missing managed output records from `.distributor.json`. The command does not delete destination files or adopt unmanaged entries. If the files should exist, restore them from backup or republish instead.
Symptom: `prune --dry-run` reports `planned=0` or JSON `planned_outputs: []`.
Likely cause: pruning is disabled for the selected destination, every managed output is preserved by `keep_latest`, no managed output is older than `older_than`, or the selected shared-root owner has no eligible outputs.
Safe fix: verify the selected destination's retention policy and the managed output `updated_at` timestamps in `.distributor.json`. Adjust config if the policy is too conservative, then rerun `--dry-run` before using `--apply`.
Safe fix: keep the default skip behavior unless replacement is intentional. To replace newer state, configure `transfer.on_destination_newer: replace`, preview with `--dry-run --force`, then publish with `--force`.
Safe fix: inspect the pipeline id, destination id, backend, and bundle path. Proceed only if deleting everything inside that destination bundle path is intended.
Likely cause: publication would write two outputs to the same destination path, such as copying `report.html` while also generating `report.html` from `report.md`. For merge reconciliation, it can also mean a planned output path already exists in storage but is not recorded in `.distributor.json` as managed.
Safe fix: adjust source files or publish/transform policy so copied and generated outputs do not collide. For merge reconciliation, move unmanaged content aside, choose another destination path, or use replacement/forced replacement only when deleting the destination bundle path is intended.
Symptom: `fail_conflict` with a reason like `destination output path ... is owned by <pipeline>/<destination>`.
Likely cause: a `state.mode: shared_root` destination planned an output path already recorded in `.distributor.json` for another pipeline/destination owner.
Diagnostic:
```sh
cat <destination-path>/.distributor.json
go run ./cmd/distributor run --config <config-path> --dry-run
```
Safe fix: change one owner so it writes a different output path, use a separate destination root, or intentionally replace the whole destination root only after previewing with `--dry-run --force`.
Symptom: `fail_unmanaged` with a reason like `destination output path ... exists but is not managed by destination state`.
Likely cause: a `state.mode: shared_root` destination planned a new output path that already exists in storage but is not recorded as managed in `.distributor.json`.
Diagnostic:
```sh
find <destination-path> -maxdepth 2 -print
cat <destination-path>/.distributor.json
```
Safe fix: move the unmanaged file aside, change the planned output path, or use forced replacement only when deleting the configured destination root is intended.
Safe fix: inspect the destination bundle path printed in the error. `distributor` attempts to remove outputs from the failed attempt, but operators should verify the destination before retrying. In merge mode, previously managed retained or overwritten outputs may remain intentionally. Rerun `--dry-run` before publishing again.
Likely cause: `secrets.directory` points to a missing or unreadable directory, or it contains a regular file whose name is not a valid credential variable name.
Safe fix: mount or create the directory, adjust permissions for the service user, or rename/remove invalid secret files. Secret filenames must match `[A-Za-z_][A-Za-z0-9_]*`.
Symptom: `credential environment variable ... is not set`, `credential environment variable ... is empty`, or S3 authentication errors such as `AccessDenied`, `InvalidAccessKeyId`, or `SignatureDoesNotMatch`.
Likely cause: configured S3 credential variable names are not available through the process environment or `secrets.directory`, are empty, or do not authorize the requested bucket/prefix.
Safe fix: provide both configured S3 credential values, correct IAM/service permissions, or omit explicit credential fields to use the AWS SDK default credential chain.
Safe fix: pre-populate `known_hosts` for `strict`, configure a writable `known_hosts` path for `accept-new`, or verify the server identity before updating a changed key. Do not disable host key checking to bypass an unexpected changed key.
Safe fix: verify `endpoint`, `bucket`, `region`, `prefix`, and `force_path_style`. For S3-compatible services, keep `force_path_style: true` unless the service requires virtual-host addressing. Distributor does not provide an insecure TLS bypass setting.
Safe fix: provide one distinct non-empty token value per upload token record through the process environment or `secrets.directory`. Do not put literal tokens in YAML.
Likely cause: the request lacks `Authorization: Bearer <token>`, has an empty token, or uses a token that does not match any configured upload token record.
Safe fix: request the intended pipeline id, or update the token allowlist to include the configured `http_upload` pipeline that this producer may submit to.
Likely cause: the request path has an invalid pipeline id, included a `pipeline` or `pipeline_id` query, archive content is malformed, the body exceeds size limits, content type is unsupported, or the in-memory upload queue is full.
Safe fix: send one valid tar or tar.gz source bundle archive to `/v1/pipelines/<pipeline-id>/upload` with `Content-Type: application/x-tar`, `application/gzip`, or `application/x-gzip`; remove pipeline query parameters; reduce archive size or raise the configured limit; retry after queue pressure drops.
Likely cause: the request reused an `Idempotency-Key` for the same token id and pipeline id with a different source manifest, or another request with the same key is still being staged before its manifest is known.
Safe fix: if the response includes `"retryable":true`, retry the same upload later with the same key. Otherwise, inspect the producer operation and use the same key only for the same source bundle.
