This roadmap defines a documentation-only refresh plan for `distributor` after the local MVP and cleanup roadmap implementation.
The goal is to make current-behavior documentation concise, accurate, and compliant with `docs/policy/documentation.md` while keeping planned, aspirational, or unimplemented behavior under `docs/roadmap/`.
This file is written for an LLM coding agent that will implement the documentation refresh in stages. It does not itself rewrite current user, policy, internal, or example documentation.
## Repository Documentation Inventory
The current documentation and examples reviewed are:
| Path | Current status | Notes |
| --- | --- | --- |
| `README.md` | Keep and lightly update | Short, accurate orientation page with a runnable local example command. |
| `docs/cli.md` | Keep and update against CLI tests | Covers `version`, `run`, `validate`, and `inspect`; should be checked against command help and parser tests. |
| `docs/config.md` | Keep and tighten | Current config reference documents local execution and also accepted SSH/S3 config fields. It must clearly distinguish config validation from executable backend support. |
| `docs/operations.md` | Keep and expand slightly | Covers local workflow, destination state, retry behavior, cleanup, fan-out failure handling, and caveats. |
| `docs/policy/architecture.md` | Keep and clarify where needed | Development policy is broad and includes future adapter direction. Wording should not imply SSH/S3 adapters currently exist. |
| `docs/internal/local-backend.md` | Missing, optional | Local adapter behavior may remain in `docs/internal/storage.md`; create this only if local filesystem safety detail outgrows that doc. |
| `docs/roadmap/audit.md` | Historical roadmap/report | Keep under roadmap unless replaced by a new audit. |
| `docs/roadmap/cleanup.md` | Historical or completed roadmap | Keep under roadmap; optionally add completion status in the documentation refresh. |
| `docs/roadmap/config.md` | Roadmap | Keep as planning material; avoid linking to it as current config reference. |
| `docs/roadmap/contracts.md` | Roadmap | Keep as planning material; current implemented contracts should be summarized in `docs/internal/` and user docs as needed. |
| `docs/roadmap/implementation.md` | Roadmap | Keep as implementation history plus future stages; status should be clear. |
| `docs/roadmap/packages.md` | Roadmap | Keep as planning material; current package docs belong under `docs/internal/`. |
| `docs/roadmap/storage.md` | Roadmap | Keep as planning material; current storage contract belongs in `docs/internal/storage.md`. |
| `examples/local-to-local.yml` | Keep | Minimal local config; load-tested. |
| `examples/local-publish.yml` | Keep | Runnable local publication example used by README and CLI docs. |
| `examples/local-html.yml` | Keep | Runnable local HTML example. |
-`docs/policy/development.md` contains real workflow guidance.
- Non-roadmap docs are scoped to implemented behavior. SSH/S3 execution, force overwrite, and external notification adapters remain described as unavailable unless the corresponding code exists.
-`examples/fan-out.yml` is local-only and runnable.
No broad rewrite is needed for `README.md`, `docs/cli.md`, `docs/config.md`, or `docs/operations.md`. They are close to the implemented local MVP and should be tightened against code and tests.
## Target Documentation Set
### `README.md`
- Audience: users, administrators, operators.
- Purpose: short project orientation and fastest useful local command.
- Canonical scope: project purpose, elevator pitch, one local quickstart command, links to current docs.
- Recommended outline: title, one-sentence description, local example command, links.
- Source-of-truth repo areas to inspect: `internal/cli`, `internal/app/run.go`, `examples/local-publish.yml`, `docs/cli.md`.
- Acceptance criteria: command is executable in the current local MVP; README does not describe SSH, S3, force overwrite, notification adapters, or future roadmap behavior as available.
- Recommended outline: shortest useful command, command overview, flag reference, common workflows, diagnostics and recovery commands.
- Source-of-truth repo areas to inspect: `internal/cli/*.go`, `internal/cli/*_test.go`, `internal/app/validate.go`, `internal/app/inspect.go`, `internal/app/run.go`.
- Acceptance criteria: every documented command and flag exists; `validate` and `inspect` are documented as local path commands; `run --dry-run` output is described without over-specifying every line; unsupported remote execution is stated clearly.
- Canonical scope: config file path behavior, minimal local config, production-oriented local config, full schema, defaults, validation rules, secrets handling, links to examples.
- Recommended outline: config file location, minimal local config, production-oriented local config, reference, defaults, secrets, examples.
- Source-of-truth repo areas to inspect: `internal/config/config.go`, `internal/config/defaults.go`, `internal/config/load.go`, `internal/config/validate.go`, `internal/config/load_test.go`, `examples/*.yml`.
- Acceptance criteria: fields and defaults match code; `KnownFields(true)` behavior is noted where useful; SSH/S3 fields are described as accepted by config validation but not implemented for execution; `on_digest_mismatch: warn` and unmanaged overwrite are not documented as active options.
### `docs/operations.md`
- Audience: administrators, operators.
- Purpose: operating and recovery notes for the implemented local MVP.
- Recommended outline: normal workflow, filesystem layout, destination state, dry-run and planning, retry and replacement behavior, failure handling, cleanup behavior, caveats.
- Source-of-truth repo areas to inspect: `internal/app/run.go`, `internal/publish/plan.go`, `internal/publish/execute.go`, `internal/publish/reconcile.go`, `internal/publish/safety.go`, `internal/state`, `internal/adapters/local`.
- Acceptance criteria: describes only local-to-local operation; explains `.distributor.json` as the managed sentinel; distinguishes skip, replace, conflict, and unmanaged destination behavior; does not promise resume, remote storage, force overwrite, or external notifications.
### `docs/troubleshooting.md`
- Audience: administrators, operators.
- Purpose: symptom-oriented fixes for common local MVP failures.
- Recommended outline: one entry per symptom with symptom, likely cause, diagnostic step, safe fix, and relevant link.
- Source-of-truth repo areas to inspect: `internal/config/validate.go`, `internal/bundle/validate.go`, `internal/state/compare.go`, `internal/publish/plan.go`, `internal/publish/output.go`, CLI tests.
- Acceptance criteria: entries are actionable and do not suggest unsafe deletion; remote backend failures are described only as unsupported execution; all fixes link to `docs/cli.md`, `docs/config.md`, or `docs/operations.md` where useful.
### `docs/policy/architecture.md`
- Audience: developers, LLM coding agents.
- Purpose: development principles and architectural invariants.
- Source-of-truth repo areas to inspect: full package tree, implemented internal docs, roadmap files for explicitly future work.
- Acceptance criteria: still gives long-term architecture direction, but any unimplemented adapter packages or future capabilities are worded as planned/target architecture rather than implemented behavior.
### `docs/policy/development.md`
- Audience: developers, LLM coding agents.
- Purpose: contributor and agent workflow.
- Canonical scope: repo layout, build/test commands, coding conventions, dependency policy, how to add config fields, CLI flags, backends, transforms, examples, and docs.
- Source-of-truth repo areas to inspect: `go.mod`, `cmd/distributor`, `internal/*`, `examples`, tests, `docs/policy/architecture.md`, `docs/policy/documentation.md`.
- Acceptance criteria: no placeholder content remains; commands are real; workflow guidance protects current boundaries; examples and docs update rules match policy.
- Recommended outline: purpose, inputs and outputs, run flow, backend and transform registration, dry-run behavior, failure behavior, notification behavior, tests to inspect, invariants.
- Source-of-truth repo areas to inspect: `internal/app/*.go`, `internal/app/*_test.go`, `internal/cli/root_test.go`.
- Acceptance criteria: documents current local-only backend execution and the no-op default notifier; does not introduce a generic stage framework that does not exist.
### `docs/internal/config.md`
- Audience: developers, LLM coding agents.
- Purpose: internal config loading/default/validation reference.
- Recommended outline: purpose, inputs and outputs, loading flow, defaults, validation responsibilities, executable support boundary, tests to inspect, invariants.
- Source-of-truth repo areas to inspect: `internal/config/*.go`, `internal/config/*_test.go`, `docs/config.md`, `examples/*.yml`.
- Acceptance criteria: documents that SSH/S3 config validation exists while execution does not; keeps user-facing config reference canonical in `docs/config.md`.
### `docs/internal/bundle.md`
- Audience: developers, LLM coding agents.
- Purpose: implemented bundle contract and validation reference.
- Recommended outline: keep current outline and verify against code.
- Source-of-truth repo areas to inspect: `internal/state`, `internal/publish/reconcile.go`, `internal/publish/execute.go`.
- Acceptance criteria: state schema and comparison outcomes match implemented structs and tests; `distributor_version` is described as optional diagnostic metadata.
### `docs/internal/storage.md`
- Audience: developers, LLM coding agents.
- Purpose: storage interface and backend safety reference.
- Canonical scope: logical paths, IO methods, traversal, `HasAny`, typed errors, managed deletion, local and fake backend behavior.
- Recommended outline: keep current outline and add any missing implemented details that matter for callers.
- Source-of-truth repo areas to inspect: `internal/storage`, `internal/storage/fake`, `internal/adapters/local`.
- Acceptance criteria: matches actual `Backend` interface, `WriteOptions`, `DeleteOptions`, `ErrStopWalk`, and `storage.List` helper; does not describe raw recursive delete as available.
### `docs/internal/publish.md`
- Audience: developers, LLM coding agents.
- Purpose: publish planning and execution reference.
- Canonical scope: request inputs, output planning, destination inspection, transfer policy, actions, replacement safety, cleanup on failed writes.
- Recommended outline: keep current outline and verify against code.
- Source-of-truth repo areas to inspect: `internal/publish`, `internal/app/run.go`, `internal/config/defaults.go`.
- Acceptance criteria: action names match constants; transfer policy values match code; collision detection and managed deletion behavior are covered.
### `docs/internal/transform.md`
- Audience: developers, LLM coding agents.
- Purpose: transform registry and Markdown transform reference.
- Canonical scope: transform interface, registry, Markdown sidecar output, deterministic output metadata, raw HTML behavior.
- Recommended outline: keep current outline; link to integration notes if `docs/integrations/markdown.md` is created.
- Source-of-truth repo areas to inspect: `internal/transform`, `internal/transform/markdown`, markdown tests.
- Acceptance criteria: `.md` to `.html` sidecar naming, skipped non-Markdown files, digest metadata, and source immutability match implementation.
- Recommended outline: keep current outline and add tests to inspect if useful.
- Source-of-truth repo areas to inspect: `internal/notify`, `internal/app/run.go`, `internal/app/run_test.go`.
- Acceptance criteria: says no external notification adapters or user-facing notification config exist.
### `docs/integrations/markdown.md`
- Audience: developers, LLM coding agents.
- Purpose: concise external integration note for Markdown rendering.
- Canonical scope: Goldmark dependency, renderer defaults used by `markdown.New`, raw HTML behavior as observed in tests, deterministic wrapper template, supported output mode.
- Recommended outline: purpose, dependency, behavior used, behavior intentionally not customized, tests to inspect, update rules.
- Source-of-truth repo areas to inspect: `go.mod`, `internal/transform/markdown`, markdown tests.
- Acceptance criteria: documents only the Markdown renderer behavior actually used; does not claim full CommonMark compatibility beyond Goldmark defaults.
- Recommended outline: add status notes only where useful; avoid rewriting history unless it causes confusion.
- Source-of-truth repo areas to inspect: current implementation and each roadmap file.
- Acceptance criteria: future work remains under `docs/roadmap/`; completed historical plans are labeled clearly enough that agents do not re-run them blindly.
Revise `examples/fan-out.yml` in the documentation refresh:
- Preferred option: replace it with a local-only fan-out example using two local destinations, such as one source archive destination and one HTML destination under `workspace/`.
- Alternative option: move the SSH/S3 fan-out material under a roadmap file and remove it from `examples/`.
- Do not keep a non-roadmap example that appears copyable for execution but uses unsupported SSH/S3 execution.
Future example tests should continue loading every YAML file under `examples/`. If `examples/fan-out.yml` becomes local-only, add or update a CLI/app test that exercises local fan-out behavior or rely on existing fan-out tests if they cover equivalent behavior.
## Internal Documentation Plan
Update existing internal docs only after checking package code and tests.
Defer these internal docs unless the corresponding implementation grows:
-`docs/internal/local-backend.md`: create only if local adapter details become too long for `docs/internal/storage.md`.
-`docs/internal/logging.md`: defer until logging has behavior beyond placeholders.
-`docs/internal/testutil.md`: defer unless test fixture helpers become a stable contributor-facing contract.
Do not create docs for packages or directories that do not exist.
## Integration Documentation Plan
No external service integration docs should be created for SSH, S3, or notification services until those integrations are implemented.
Recommended current integration doc:
-`docs/integrations/markdown.md`
This should document Goldmark usage because Markdown rendering is an implemented external file-format integration with externally visible output. Keep it concise and limited to:
- dependency and version source: `go.mod`;
- renderer construction: `goldmark.New()`;
- sidecar output path behavior owned by `internal/transform/markdown`;
- raw HTML behavior covered by tests;
- wrapper template behavior;
- tests to inspect before changing renderer behavior.
Do not create separate YAML integration docs unless configuration parsing behavior outgrows `docs/config.md` and `docs/internal/config.md`.
## Recommended Implementation Sequence
### Stage 1: Establish Documentation Status and Contributor Policy
- Goal: remove the required policy placeholder and classify active versus historical documentation.
- Files to create/update/delete/move: update `docs/policy/development.md`; optionally add short status notes to `docs/roadmap/audit.md` and `docs/roadmap/cleanup.md`.
- Repo areas to inspect: `go.mod`, `cmd/distributor`, `internal/*`, `examples`, `docs/policy/architecture.md`, `docs/policy/documentation.md`.
- Acceptance criteria: `docs/policy/development.md` contains real workflow guidance; no active policy doc says "Not yet implemented"; roadmap status notes do not change current behavior docs.
- Repo areas to inspect: `internal/cli`, `internal/app`, `internal/config`, `internal/publish`, `internal/state`, `examples`.
- Acceptance criteria: user docs describe local execution, local validation/inspection, current config schema, current defaults, current state/retry behavior, and clear unsupported remote execution boundaries.
- Files to create/update/delete/move: update or replace `examples/fan-out.yml`; update links in `README.md`, `docs/cli.md`, and `docs/config.md` if needed; move remote fan-out material to a roadmap section if preserving it is useful.
- Repo areas to inspect: `internal/config/load_test.go`, `internal/app/run_test.go`, `examples`.
- Acceptance criteria: every example under `examples/` is valid current config and does not rely on unsupported remote execution; primary examples remain load-tested.
- Suggested validation commands: `go test ./internal/config`; optional `go run ./cmd/distributor run --config examples/local-publish.yml --dry-run`; optional `go run ./cmd/distributor run --config examples/local-html.yml --dry-run`.
- One prompt: yes.
### Stage 4: Complete Internal Component Docs
- Goal: give future agents a current-behavior internal map before remote backends are added.
- Files to create/update/delete/move: create `docs/internal/app.md` and `docs/internal/config.md`; update existing `docs/internal/*.md` as needed.
- Repo areas to inspect: `internal/app`, `internal/config`, `internal/bundle`, `internal/state`, `internal/storage`, `internal/storage/fake`, `internal/adapters/local`, `internal/publish`, `internal/transform`, `internal/notify`, package tests.
- Acceptance criteria: every major implemented component has a concise doc with purpose, inputs/outputs, boundaries, failure behavior, tests to inspect, and invariants; no internal doc describes absent SSH/S3 adapters as implemented.
