This document defines the development principles for `distributor`. It is inward-facing: developers and LLM coding agents should use it to preserve the project’s shape, boundaries, and invariants as the code evolves.
## Project Scope
`distributor` is a domain-agnostic report bundle distribution tool.
Producer applications generate manifested bundles. `distributor` discovers those bundles, validates them, optionally derives publication artifacts such as HTML, and publishes selected source and generated artifacts to one or more configured destinations.
`distributor` does not generate domain reports, interpret domain-specific report content, run producer pipelines, edit reports, or act as a CMS. Weather reports, D&D recaps, calendar summaries, email digests, and additional report types should all enter `distributor` through the same bundle contract.
Default to a small, explicit, dependency-light Go application. Keep the design modular enough to test and change safely, but do not add abstraction unless it protects a real boundary or enables a real extension point.
Business logic should live outside CLI, transport, and external-adapter packages. The core application should reason in terms of pipelines, bundles, destination state, transforms, and publish plans—not S3 SDK calls, SFTP sessions, shell commands, or filesystem details.
The source is discovered and validated once. Each destination has independent backend configuration, publication policy, transform policy, replacement behavior, state, and notification behavior.
Destination-specific behavior must not leak back into the source bundle contract. A producer should not need to know whether a bundle will be published to local storage, another storage backend, a static site, email, RSS, or another notification channel.
A source bundle is a directory containing `manifest.json`.
`manifest.json` is the sole producer-to-`distributor` contract. `distributor` must not rely on producer-specific work directory layouts, filenames, metadata, or conventions outside the configured source root and the source manifest.
The MVP source manifest schema is intentionally minimal:
-`created`: RFC3339 timestamp. UTC is preferred; explicit offsets are allowed.
-`files`: ordered file list.
-`files[].path`: relative path beneath the bundle root.
-`files[].sha256`: SHA-256 digest of the file contents.
-`files[].size`: file size in bytes.
Bundle paths must be relative, clean, and confined to the bundle root. Paths must not be absolute, empty, contain `..` path traversal, or otherwise escape the bundle root. Symlink handling must be explicit; unless documented otherwise, source bundle symlinks should be rejected.
Before processing a bundle, `distributor` must validate file existence, file size, each file SHA-256, and the bundle digest. Digest mismatch must fail in the MVP before any destination writes occur.
The source manifest should remain minimal. Routing, destination selection, publication format, credentials, static-site layout, notification recipients, and destination-specific metadata belong in `distributor` configuration and destination state, not in producer manifests.
## Destination State Contract
`manifest.json` from the source bundle is not copied to destinations as destination state.
Each destination bundle path is managed by `.distributor.json`. This file is both the destination sentinel and the destination state record. It records:
-`distributor` state schema version;
- pipeline id;
- destination id;
- publication timestamp;
- the normalized source manifest used for publication;
- metadata for copied source outputs;
- metadata for generated outputs, such as HTML files;
- any additional metadata required by `distributor`.
A representative destination state file is:
```json
{
"schema_version": 1,
"distributor_version": "0.1.0",
"pipeline_id": "weather-daily",
"destination_id": "static-html",
"published_at": "2026-05-30T11:12:00Z",
"source": {
"manifest": {
"schema_version": 1,
"id": "weather.daily.brentwood.2026-05-30",
"digest": "sha256:...",
"created": "2026-05-30T11:10:00Z",
"files": [
{
"path": "report.md",
"sha256": "sha256:...",
"size": 12345
}
]
}
},
"outputs": [
{
"path": "report.html",
"kind": "generated",
"source_path": "report.md",
"transform": "markdown_to_html",
"sha256": "sha256:...",
"size": 23456
}
]
}
```
Destination comparison rules are based on `.distributor.json`:
- No `.distributor.json`: publish normally only if the destination bundle path is empty.
- Existing state embeds the same normalized source manifest: skip as already published.
- Existing state has the same source id and an older source `created`: replace, subject to destructive-operation safety rules.
- Existing state has the same source id and a newer source `created`: skip because the destination is newer than the source.
- Existing state has the same source id and same `created` but different digest: fail as a conflict.
- Existing state has a different source id: fail as a conflict.
## Publication and Transform Policy
Source files are canonical bundle artifacts. Transform outputs are derived publication artifacts.
Transforms are configured per destination. A destination may receive source files, generated HTML files, or both.
The MVP supports only Markdown-to-HTML transformation. HTML generation must not mutate the source bundle. Generated outputs must be deterministic from the source bundle and destination transform configuration, and must be recorded in `.distributor.json`.
The application should distinguish:
- transform policy: how derived files are generated;
- publish policy: which source and generated files a destination receives.
For example, one destination may publish source files only as a long-term archive, while another publishes HTML only as a static site.
Sources and destinations use the same storage abstraction. Current runtime execution uses the local filesystem, SSH/SFTP, and S3-compatible backends. Additional storage backends should be peer implementations behind the same interface, and any backend-specific execution limitation must be documented.
Application logic must interact with storage through internal backend interfaces. Backend-specific behavior belongs in adapter packages. Pipeline, bundle, state, publish, and transform packages must not import service-specific or filesystem adapter implementation details.
Adapters should be thin. Backend adapters should implement storage operations and translate backend-specific errors, but should not make bundle comparison, transform, routing, or replacement decisions.
Remote file-transfer support should prefer native protocol implementations over shelling out, unless a later design document records a reason to differ.
Use external dependencies only when justified by correctness, security, interoperability, or substantial complexity reduction. Good reasons include YAML parsing, S3-compatible storage integration, SSH/SFTP integration, and Markdown rendering.
Avoid dependencies for small conveniences. Do not let external dependency types leak across internal package boundaries unless the dependency is itself the explicit public contract of that package.
Package-private implementation constants may live near the package that owns them, preferably in `constants.go` when useful.
## Configuration
Centralize configuration loading, processing, precedence, defaults, and validation in `internal/config`.
The goal is to make configuration discoverable and avoid implicit or hidden operational values. User-visible defaults and cross-package operational defaults should be defined in `internal/config/defaults.go`.
Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables, secret files, SSH agent usage, standard AWS credential mechanisms, or explicitly named environment variable references for secrets.
If users can select backends, transforms, notifiers, or renderers, selection should go through a registry or equivalent mechanism rather than scattered conditionals.
The orchestrator should be able to plan, dry-run, and execute configured pipelines. Dry-run behavior should be first-class because the application may delete, overwrite, or publish files.
Store embedded JSON schemas, Markdown templates, HTML templates, CSS, and similar assets as separate files, not inline string literals, unless there is a strong reason otherwise.
Errors should be actionable and preserve context. Wrap errors with operation, pipeline id, destination id, backend, path, bundle id, and resource context where useful. CLI code should convert internal errors into concise user-facing messages.
Use structured logging where practical. Logs should describe discovery, validation, planned actions, skipped transfers, conflicts, replacements, external calls, retries, and failure causes, but should not include large report contents by default.
Skip and no-op decisions should be logged at an appropriate level so operators can distinguish successful publication from intentional no-op behavior.
Long-running operations should accept `context.Context`. Storage operations, service requests, transforms, and multi-step workflows should respect cancellation and timeouts.
If the application writes durable state, writes should be atomic where practical. Multi-step workflows should preserve enough state to support inspection, retry, or resume after failure.
Code that deletes, moves, or overwrites files must use narrow, explicit paths. Avoid broad parent-directory operations. Cleanup that can cause data loss must be opt-in.
`distributor` must never perform broad deletion against a configured source root. Destination deletion must be bounded to the resolved destination bundle path for the current source bundle and backend root.
Normal destructive replacement may occur only when a valid `.distributor.json` confirms that the destination bundle path is distributor-managed. Explicit forced replacement is a per-run CLI workflow for supported conflict and unmanaged-content cases; it must be dry-runnable, clearly reported, and constrained to the destination bundle path.
Replacement must be narrow, reported, test-covered, and configurable. Prefer normal replacement that deletes files recorded in `.distributor.json` and known generated outputs. Forced replacement may delete a bounded destination bundle prefix only when the operator explicitly requests it. Backend implementations must guard against path traversal, prefix confusion, and accidental deletion above the configured backend root.
Where practical, publish operations should use staging paths or temporary objects and promote them into place only after validation and transform steps succeed.
Config examples should be load-tested. Important CLI workflows should have parser or command tests. Component contracts should have focused tests that do not require running the full application unless end-to-end coverage is intentional.
Documentation should follow the project documentation policy. Keep user docs focused on implemented behavior. Put future, planned, or aspirational work only under `docs/roadmap/`.
When changing architecture, config, CLI behavior, adapters, manifest/state contracts, transform behavior, publish behavior, or component contracts, update the relevant docs and examples in the same change.
The source manifest and destination `.distributor.json` schemas should have canonical documentation once implemented. Example configs should be valid and load-tested where practical.
## Non-Goals
`distributor` is not:
- a report generator;
- a domain-specific weather, D&D, calendar, or email summarization app;
