ansible-workspace-dnd/docs/roadmap/implementation.md

# Implementation Plan 0016: Deploy the D&D Pipeline to `pipeline-dnd`

Status: ready

Target executor: GPT-5.6-Terra

Feature roadmap: `docs/roadmap/features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md`

Active plan location: `docs/roadmap/implementation.md`

## Objective

Deploy the reviewed collection through a minimal consumer playbook, host
inventory, and Ansible Vault. Demonstrate clean repeated convergence and one
operator-selected session through real publication. The feature roadmap owns
scope, rationale, target state, and acceptance criteria.

## Execution Rules

- Execute stages in order. Before each stage, reread its Required Policy
  Review and check the owning repository's working tree. Preserve unrelated
  changes. Validate and commit each completed stage with its stated intent.
- Keep the playbook limited to hosts and roles. Use existing interfaces;
  add no wrapper, preflight framework, custom module, or test harness.
- Inherit suite storage, publication, caching, cleanup, version, and asset
  defaults. Set only the roadmap-selected host choices.
- Never print decrypted inventory, credential values, or Vault passwords in
  commands, logs, documentation, or Git. Do not use decrypted inventory dumps
  as validation evidence. The operator supplies secrets directly through
  Ansible Vault's encrypted-file workflow, using an established secure editor.
  Do not solicit secrets in chat or create placeholder credentials.
- Stop dependent work when operational inputs are missing or validation fails.
  Record the concrete blocker; local checks do not prove a live deployment.
- Retain the historical playbook and proof-of-concept role. Role, collection,
  application, and asset repositories remain read-only during this feature.
- Deployment and paid provider execution occur only during the approved sprint
  with the selected controller, target, credentials, and session. Do not
  substitute an arbitrary session or broaden the host limit.

## Starting State and Preconditions

- Feature 0015 passed independent review without findings. Its pipeline commit
  is `51566afcf69083cdbb5b5718f2544c86b079001c`; the reviewed public collection
  commit is `46913446765e86ecfb29ebad9955d579ef6a4c7e`.
- `rakestraw-ansible` starts clean on `main` at
  `d86ec29a9cbd821fead248736171b5cdd8440ad6`. Inspect and preserve any subsequent
  changes before adapting this plan.
- Root `requirements.yml` already has an unpinned Gitea `rakestraw.dnd` entry
  alongside unrelated collections. The historical playbook is
  `playbooks/apps/dnd.yml`; it invokes `pipeline_dnd` on `automation-dnd`.
- Host `pipeline-dnd` already connects as `eric` to
  `pipeline-dnd.rakestrawhome.com`. Preserve its connection settings.
- `inventory/host_vars/pipeline-dnd/narratio.yml` contains the legacy campaign
  and S3 variables to migrate.
- The selected collection still declares version `0.1.0`. An installed
  release with that number is not sufficient evidence of the selected content.
- Approve and commit the planning documents before execution. Resolve the
  operational inputs under Open Questions before the sprint, as required by
  the roadmap; do not invent controller topology, credentials, or session data.

## Repository Scope

- `rakestraw-ansible`: requirements, playbook, host configuration, encrypted
  credentials, and `docs/pipeline-dnd.md` runbook linked from README.
- Workspace: feature status, implementation progress, and review handoff.
- Other repositories: read-only compatibility and application references.

## Stage 1: Establish the Operational Context

### Objective and Owning Repository

Record deployment prerequisites in `rakestraw-ansible` and close the reviewed
Feature 0015 in the workspace.

### Required Policy Review

Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016
roadmap, and `rakestraw-ansible/README.md` before this stage.

### Preconditions

The plan is approved, repositories have been checked, and the operator has
supplied the missing operational inputs below.

### Required Changes

1. Create `rakestraw-ansible/docs/pipeline-dnd.md` and link it from README.
   Record the actual controller, checkout, effective Ansible configuration,
   inventory, role/collection paths, Vault selection method, and acceptance
   campaign/session/audio location. Record only non-secret information.
2. Confirm controller Ansible Core is at least 2.18 and the public suite
   repositories and Galaxy are reachable. Inspect effective configuration
   without dumping decrypted host variables. Existing repository defaults are
   `/etc/ansible/inventory`, `/etc/ansible/roles`, and `/etc/ansible/collections`;
   establish how the operational checkout supplies those paths.
3. Use ordinary read-only host checks to confirm Ubuntu 26.04 and the existing
   `eric` account's SSH and required sudo access. Record the outcome. Report
   discrepancies before deployment rather than changing platform/account policy.
4. Mark Feature 0015 `done` in its roadmap and workspace status, citing its
   reviewed commits and the successful independent standalone/collection
   integration runs. Leave historical remediation records unchanged. Mark
   Feature 0016 `in_progress` when execution begins.

### Validation and Completion Criteria

Operational inputs are concrete and target platform/access checks pass.
Run `git diff --check` in both changed repositories. No deployment or provider
request is necessary for this stage.

### Commit

Consumer: `Document the D&D deployment context`.
Workspace: `Close storage review and begin host deployment`.

## Stage 2: Prepare the Consumer Playbook and Inventory

### Objective and Owning Repository

Make `rakestraw-ansible` a complete consumer of the reviewed suite snapshot.

### Required Policy Review

Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016
roadmap, `ansible-collection-dnd/README.md`, its `requirements.yml`, and the
Stage 1 runbook before this stage.

### Preconditions

Stage 1 is complete. The operator can populate and unlock the encrypted host
file using the recorded Vault mechanism.

### Required Changes

1. Add `playbooks/pipeline/dnd.yml` in the roadmap's exact minimal shape:
   one play targeting `pipeline-dnd` with only `rakestraw.dnd.pipeline` in its
   roles list. No play variables, tasks, or play-level privilege escalation.
2. Replace the legacy contents of
   `inventory/host_vars/pipeline-dnd/narratio.yml` with the roadmap's public
   inventory example: operator `eric`, the confirmed WhisperX URL,
   `narratio_campaigns_default_campaign_id: dilfs`, and the four Vault-backed
   `dnd_pipeline_secrets` mappings. Remove obsolete `narratio_s3_*` and
   `narratio_default_campaign_id` settings. Preserve unrelated intervening edits
   and the separate connection file. Do not copy suite defaults into inventory.
3. Have the operator create `inventory/host_vars/pipeline-dnd/vault.yml`, fully
   Ansible Vault-encrypted, defining the four exact `vault_dnd_*` names from
   the roadmap. Commit ciphertext only.
4. Pin the existing `rakestraw.dnd` Git entry in root `requirements.yml` to
   `46913446765e86ecfb29ebad9955d579ef6a4c7e` with its existing public URL.
5. Copy the five `roles` entries from that collection's compatibility manifest,
   retaining role names, public Gitea URLs, `scm: git`, and `v0.1.0` tags.
   Reconcile native collection entries to `ansible.posix: 2.2.0` and
   `community.general: 13.3.0`, without duplicates. Preserve unrelated
   dependencies; report demonstrated conflicts rather than silently changing
   the reviewed suite pins.
6. Extend the runbook with the concrete commands for dependency installation,
   collection refresh, Vault selection, deployment, fresh login, and native
   validation used below. Link upstream Narratio v2.1.1 operator documentation.

### Validation

Parse edited plaintext YAML and run `git diff --check`. In isolated controller
dependency paths, install from public sources with
`ansible-galaxy install -r requirements.yml`. Configure role and collection
destinations through Ansible configuration/environment; avoid a role-only
custom path that would skip the collection section of the mixed manifest.
Confirm resolution with `ansible-doc -t role rakestraw.dnd.pipeline` and the
five `rakestraw.<app>` role names.

From the consumer checkout, using the established Vault options, run:

```text
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --list-hosts
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --syntax-check
```

Use the concrete inventory path from Stage 1 if different. Do not impose
whole-repository lint changes on the established minimal-playbook style.

### Completion Criteria

Only the intended host is selected. Public dependencies resolve without local
sibling checkouts, syntax checking succeeds with encrypted inventory, and the
diff contains no plaintext secrets.

### Commit

`Add the D&D pipeline deployment playbook`.

## Stage 3: Deploy and Verify Clean Convergence

### Objective and Owning Repository

Deploy from the operational controller; record results in the consumer runbook.

### Required Policy Review

Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016
roadmap, and `rakestraw-ansible/docs/pipeline-dnd.md` before this stage.

### Preconditions

Stage 2 passed. The operational checkout contains its committed changes and
Vault is usable. Deliver changes through the established checkout workflow;
do not overwrite a dirty operational checkout.

### Required Changes and Validation

1. Install root requirements at the confirmed controller paths. Explicitly
   refresh only the D&D collection from its selected commit, even when `0.1.0`
   is already installed:

   ```text
   ansible-galaxy collection install git+https://gitea.maximumdirect.net/eric/ansible-collection-dnd.git,46913446765e86ecfb29ebad9955d579ef6a4c7e --force --no-deps
   ```

   Use the configured collection path; dependencies come from the manifest.
   Do not force-update unrelated unpinned collections. Confirm role resolution
   and playbook syntax on this operational controller as in Stage 2.
2. Run `ansible-playbook playbooks/pipeline/dnd.yml --limit pipeline-dnd`
   using the recorded inventory, Vault, and any required sudo-password options.
   Immediately repeat the identical command after success. Require
   `changed=0`, `failed=0`, and no unreachable hosts on the repeat.
3. On the target, execute
   `sudo -u narratio /usr/local/bin/narratio config validate`, then validate
   with `--profile testing`. The managed default campaign should make an
   explicit campaign selector unnecessary.
4. Start a fresh login as `eric` and execute the same default native validation
   without switching user, demonstrating usable supplementary group access.
5. Record consumer commit, target, date, repeat recap, and native validation
   results. Diagnose failures at the owning boundary; report required changes
   to roles/applications separately rather than expanding this feature.

### Completion Criteria

The actual host converges cleanly and both runtime identities validate the
configuration. Run `git diff --check` on the evidence/runbook changes.

### Commit

`Record successful D&D host deployment`.

## Stage 4: Exercise One Real Session

### Objective and Owning Repository

Demonstrate production processing and publication; record acceptance in
`rakestraw-ansible/docs/pipeline-dnd.md`.

### Required Policy Review

Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/testing.md`, `docs/planning.md`, the Feature 0016 roadmap, the runbook,
and Narratio v2.1.1's `docs/config.md`, `docs/cli.md`, and
`docs/operations.md` before this stage. Follow the sibling application's
`AGENTS.md` and development reading guide when consulting its repository.

### Preconditions

Stage 3 passed. Actual credentials, the operator-selected campaign/session,
session file, and audio are accessible. The exercise must demonstrate real
transcription and publication, not merely resume an already complete run.
Do not erase existing state to create a fixture.

### Required Changes and Validation

1. As `eric` in a fresh authorized login, run:

   ```text
   narratio session validate SESSION_ID --campaign CAMPAIGN --session SESSION_FILE
   narratio run SESSION_ID --campaign CAMPAIGN --session SESSION_FILE
   narratio session artifacts SESSION_ID --remote --campaign CAMPAIGN --session SESSION_FILE
   ```

   Substitute the operator-recorded values and use the production profile.
   The session document remains operational input; no session-management task
   belongs in the playbook. Observe transcription, processing, and successful
   publication. Do not force reprocessing or reduce the selected artifact set
   merely to bypass a failure.
2. Observe a representative published transcript or recap at the configured
   destination through the existing storage/operator interface. Successful
   cleanup may remove local workspace content; local survival is not required.
3. Record identity, input location, runtime user, completion outcome, and
   non-secret publication evidence. Avoid committing raw provider logs or
   credential-bearing URLs. Do not repeat paid work just to duplicate evidence
   or expand acceptance into a restore/locking test matrix.

### Completion Criteria

A real session completes and its published output is observable. Record any
provider/application blocker and leave acceptance incomplete if it cannot
finish. Run `git diff --check` on the runbook changes.

### Commit

`Record the first published D&D session`.

## Stage 5: Prepare the Independent Review Handoff

### Objective and Owning Repository

Record completed implementation evidence in the workspace.

### Required Policy Review

Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/testing.md`, `docs/planning.md`, and the Feature 0016 roadmap before
this stage.

### Preconditions

Stages 1–4 completed with committed changes and actual deployment/session
evidence. A blocked live exercise does not satisfy this stage.

### Required Changes and Validation

Update `docs/status.md` and this plan's progress notes with owning commits,
runbook link, validation outcomes, and deviations. Keep Feature 0016
`in_progress` pending independent review. Report any additional complexity
and its authorizing requirement. Run `git diff --check` and confirm changed
repositories are clean after committing. Do not rerun role integration
matrices or paid sessions without a new change or failure requiring it.

### Completion Criteria

Evidence is recorded and implementation is ready for independent review.

### Commit

`Record the D&D deployment implementation`.

## Final Validation and Handoff

Report dependency revision, consumer commits, controller/target, public
dependency and syntax results, immediate clean convergence, native validation
for both identities, and session publication evidence. Distinguish completed
work from operator-dependent blockers. Do not mark Feature 0016 `done` before
independent review.

## Progress Notes

Plan approved; no Feature 0016 implementation or deployment has begun.
Architecture is settled; the operational inputs below remain to be supplied.

## Open Questions

1. Which controller/checkout executes deployment and how do committed changes
   reach it? Recommended: use the existing operational controller and its
   established `/etc/ansible/` layout, keeping one operational inventory.
   Alternative: a dedicated checkout with explicit inventory and dependency
   paths, if the operator prefers isolation and records those paths consistently.
2. Which Vault identity/password mechanism is available, and can the operator
   populate all four values? Recommended: reuse the controller's established
   Vault workflow and populate the encrypted file directly. Alternative: an
   interactive Vault password prompt for this file if no identity is established.
   Both keep passwords outside Git; neither requires secrets in chat.
3. Which campaign/session/audio supplies acceptance? Recommended: an
   operator-selected unprocessed `dilfs` session with accessible audio, matching
   the retained default campaign and avoiding forced reprocessing. Alternative:
   a selected session from another deployed campaign with an explicit selector.
   Record the session file and audio location before execution; do not invent
   sample production input.