weatherreporter/docs/roadmap/implementation.md

# Domain-Specific Prompt Profiles Implementation Plan

Status: Stages 1–7 completed; remediation Stage 8 ready.

## Purpose And Authority

This document records the implementation and post-implementation remediation
of the
[domain-specific prompt profiles roadmap](domain-profiles.md). The roadmap is
authoritative for scope, user intent, policy choices, and the intended end
state. This plan records implementation sequence, verification, audit findings,
and exit gates.

This plan follows the repository's
[architecture](../policy/architecture.md),
[documentation](../policy/documentation.md), and
[testing](../policy/testing.md) policies.

## Completed Prerequisite

Weatherreporter is already pinned to Promptkit v0.5.0. That release provides
the public `WithFallbackProfileFS` option and the required precedence across
inspection, preparation, and execution. The dependency upgrade passed
Weatherreporter's full offline test suite, race-enabled suite, CLI help check,
and an operator `generate hourly` smoke test. Do not repeat or replace the
dependency upgrade as part of these stages.

## Locked Product Decisions

Implement these exact Weatherreporter-owned profiles:

| Profile ID | Backend | Model | Reasoning effort | Timeout | Service tier |
| --- | --- | --- | --- | --- | --- |
| `weather-light` | `openrouter` | `deepseek/deepseek-v4-flash` | Omitted | 180 seconds | `flex` |
| `weather-balanced` | `openrouter` | `~google/gemini-flash-latest` | `high` | 240 seconds | `flex` |
| `weather-deep` | `openrouter` | `~anthropic/claude-sonnet-latest` | `high` | 240 seconds | `flex` |

Assign Hourly to `weather-light`; assign Daily, Today, and Tomorrow to
`weather-balanced`; assign no report to `weather-deep` initially. Advance all
four prompt definitions and matching report-registry entries from `1.0.1` to
`1.1.0` when their defaults change.

The leading `~` in the Gemini and Claude model IDs is required and denotes an
OpenRouter rolling alias. Do not substitute the unavailable non-tilde IDs or a
dated model version. Do not add temperature, `top_p`, maximum-token, endpoint,
or credential fields to the embedded definitions.

Prompt preparation and execution artifacts written at `1.0.1` are not required
to remain readable after the transition to `1.1.0`. Do not add a migration,
compatibility shim, or weaker historical-artifact validation for this feature.

Definition lookup must remain:

1. explicit Promptkit in-memory profiles used by tests or an embedding
   consumer;
2. Weatherreporter's configured `profile_file` or `profile_dir` source;
3. Weatherreporter's embedded fallback profiles; and
4. Promptkit's built-in catalog.

Selection remains a separate concern: a nonblank global `promptkit.profile`
selects the profile for every report in the invocation; otherwise the exact
prompt definition's `default_profile` selects it. A malformed matching
higher-precedence profile is an error and never falls through.

## Continuing Invariants

- Keep all Promptkit types and mechanics inside
  `internal/adapters/promptkit`, its focused tests, and asset contract tests.
- Keep prompt inspection before weather collection and provider work.
- Keep one Promptkit engine per command action and one shared engine across a
  sequential batch.
- Preserve logical profile ID and effective backend/model information through
  active inspection and execution where the project-owned contract already
  exposes it. Do not add new durable-provenance fields or compatibility
  guarantees.
- Leave existing workspace persistence behavior otherwise unchanged. The
  accepted [ephemeral-state roadmap](ephemeral-state.md) owns its future
  removal and must not be partially implemented here.
- Do not expose endpoints, credentials, rendered messages, schemas, request
  bodies, response bodies, or complete parameter maps through ordinary errors,
  logs, summaries, or state.
- Keep the default suite deterministic, offline, and credential-free.
- Do not add endpoint discovery, health probing, provider failover, retries at
  a more expensive tier, profile merging, per-report configuration fields, or
  severity-driven model selection.
- Update canonical current-state documentation only in the stage where the
  corresponding behavior becomes implemented.
- Run `git diff --check` before completing every stage.

## Stage 1: Add The Embedded Weather Profile Catalog

### Goal

Create one repository-owned, embedded profile source containing exactly the
three locked logical profiles.

### Work

1. Add strict YAML profile assets beneath `internal/promptassets` using the
   exact IDs and definitions in this plan.
2. Extend `internal/promptassets` with a narrowly named accessor that returns
   the embedded profile `fs.FS`. Follow the existing prompt and schema asset
   pattern without exposing Promptkit types from the package.
3. Keep profile filenames and embed layout simple and deterministic. Do not
   duplicate Promptkit's built-in directory taxonomy unless the application
   assets require it.
4. Validate the assets through Promptkit's public engine/profile inspection
   surface rather than adding a second YAML parser or a Weatherreporter-owned
   profile representation.

### Tests

- Extend the asset contract tests to assert exactly the three logical IDs,
  their exact effective model IDs, and the intentional parameters.
- Prove all three profiles inspect successfully offline when supplied as a
  fallback source and no operator source is present.
- Assert that the catalog contains no endpoints, credentials, temperature,
  `top_p`, or maximum-token settings.
- Run:

  ```sh
  go test ./internal/promptassets
  git diff --check
  ```

### Exit Gate

The embedded catalog is complete, strictly valid, safe, and independently
inspectable through Promptkit v0.5.0's public API.

## Stage 2: Wire Fallback Resolution And Protect Precedence

### Goal

Supply the embedded catalog through Promptkit's application fallback layer
without changing existing operator configuration or application boundaries.

### Work

1. Add `promptkit.WithFallbackProfileFS(promptassets.ProfileFS(), ".")` to
   normal adapter engine construction.
2. Preserve existing `profile_file`, `profile_dir`, configured local backend,
   timeout, prompt filesystem, schema filesystem, and test-option behavior.
3. Ensure ordinary production construction and the adapter's test
   construction path exercise the same fallback wiring. Test-only explicit
   profiles may retain Promptkit's documented highest precedence.
4. Keep all fallback resolution in Promptkit. Do not add filesystem overlays,
   existence checks, YAML parsing, or merge behavior to Weatherreporter.

### Tests

- At the adapter boundary, prove fallback-only inspection of all three
  Weatherreporter profiles.
- Prove same-ID overrides through both configured `profile_file` and
  `profile_dir`, including resolution of the override's effective backend and
  model.
- Prove an absent operator match falls through, while a malformed matching
  operator definition fails without using the embedded profile.
- Prove a selected Promptkit built-in that is absent from both higher layers
  still resolves.
- Prove an explicit in-memory test profile retains highest precedence.
- Cover both local override forms required by the roadmap: an endpoint-only
  OpenAI-compatible `weather-light` profile and a `backend: local` profile
  using the configured local endpoint. No test may contact either endpoint.
- Run:

  ```sh
  go test ./internal/adapters/promptkit
  git diff --check
  ```

### Exit Gate

Inspection and prepared execution use Promptkit's exact four-layer precedence,
operator errors remain visible, and local overrides require no prompt or code
changes.

## Stage 3: Adopt Logical Defaults And Prompt Version 1.1.0

### Goal

Move operational prompts from provider-oriented defaults to the three-tier
Weatherreporter policy with an exact, synchronized version transition.

### Work

1. Change Hourly's `default_profile` to `weather-light`.
2. Change Daily, Today, and Tomorrow to `weather-balanced`.
3. Advance the exact version in all four prompt YAML assets from `1.0.1` to
   `1.1.0` without changing prompt text or generated-text schemas solely for
   this feature.
4. Advance the four matching report-registry prompt versions to `1.1.0` in the
   same change. Keep prompt IDs, report IDs, modules, periods, templates, and
   output contracts unchanged.
5. Update fixtures and expectations that intentionally assert the current
   prompt contract. Do not rewrite historical fixture versions or weaken tests
   that protect actual compatibility.

### Tests

- Update asset and report-registry contract tests to require exact version
  `1.1.0` and the report-to-profile assignments locked in this plan.
- Inspect every exact prompt version through the real embedded prompt, schema,
  and fallback-profile filesystems.
- Prove Hourly resolves DeepSeek V4 Flash, the three day-scale reports resolve
  Gemini Flash Latest, and `weather-deep` remains inspectable but unassigned.
- Run:

  ```sh
  go test ./internal/promptassets ./internal/report ./internal/adapters/promptkit
  git diff --check
  ```

### Exit Gate

Every operational prompt and registry definition agrees on exact version
`1.1.0`, selects its intended logical tier, and resolves its expected effective
model offline.

## Stage 4: Verify Application Selection And Batch Reuse

### Goal

Protect the assembled application behavior created by the new defaults and
confirm that logical identity is not lost during active effective-model
resolution.

### Work

1. Preserve the current pre-collection inspection order and fail-fast behavior
   for missing credentials, unknown profiles, malformed profiles, and unusable
   backends.
2. Preserve the global `promptkit.profile` all-report override. Do not add a
   second override mechanism or report-specific configuration fields.
3. Preserve batch preflight deduplication by selected effective profile ID:
   Today and Tomorrow in the same batch should inspect their shared
   `weather-balanced` selection once.
4. Preserve the selected logical profile ID and resolved backend/model through
   active inspection, preparation, and execution using the existing
   project-owned contract. Do not add state fields, expand persisted parameter
   detail, or create a new historical compatibility guarantee.

### Tests

- Add or update representative app tests for default Hourly and day-scale
  selection, a global-profile override, and a morning/evening batch sharing
  `weather-balanced`.
- Assert inspection completes before weather collection and provider
  generation, including malformed same-ID operator overrides.
- Assert active inspection and execution expose the logical profile ID and
  effective model for both embedded and overridden profiles.
- Assert endpoints and credentials remain absent from errors, summaries,
  normal logs, and ordinary state.
- Use project-owned executor fakes or Promptkit provider fakes; do not make live
  provider calls.
- Run:

  ```sh
  go test ./internal/app ./internal/cli
  git diff --check
  ```

### Exit Gate

Single-report and batch workflows select the intended tier, retain existing
override and preflight behavior, deduplicate shared batch inspection, and
preserve safe logical and effective model information during active execution
without adding a durable-provenance contract.

## Stage 5: Publish Canonical Operator And Maintainer Documentation

### Goal

Document the implemented feature once in each canonical owner and provide one
maintained, copyable local override example.

### Work

1. Update `docs/config.md` to explain global profile selection versus
   `profile_file`/`profile_dir` definition lookup and link to the maintained
   example. Keep the field reference in this canonical document.
2. Update the Promptkit integration document with the logical profile catalog,
   source precedence, exact prompt-version relationship, and safe active
   inspection and execution contract. Avoid restating complete configuration
   syntax or presenting transitional persistence as the target architecture.
3. Update the report-registry, Promptkit adapter, app-orchestration, and state
   internal documents only where their implemented contracts changed.
4. Update `docs/operations.md` with the normal local-override workflow and
   `docs/troubleshooting.md` with malformed override, unavailable local
   endpoint, missing credential, and unexpected effective-model diagnostics.
5. Add or update one secret-free file under `examples/` showing a
   `weather-light` override for a local OpenAI-compatible endpoint. Choose one
   supported form as the complete example and mention the other form only in
   its canonical reference.
6. Update the architecture policy only if implementation changed a normative
   boundary or invariant. Do not add future behavior to current-state docs.
7. Keep the feature roadmap and this plan in their pre-implementation statuses
   until the final repository gate passes. Do not create release notes before
   a release version is chosen.

### Tests

- Verify every changed repository-relative link and every profile/model ID.
- Validate maintained YAML examples through the same strict configuration or
  Promptkit profile path used by production where practical.
- Run the focused tests that own any executable examples, followed by:

  ```sh
  git diff --check
  ```

### Exit Gate

Users, operators, and maintainers can discover the tier defaults, precedence,
global override, local override, and failure behavior without duplicated or
future-state documentation.

## Stage 6: Complete Repository Verification And Roadmap Handoff

### Goal

Demonstrate that the complete feature is coherent, offline-testable, and ready
for review and a later release decision.

### Work

1. Review the complete diff against the roadmap, this plan, and all three
   policy documents. Remove stale identifiers, temporary helpers, redundant
   tests, and documentation duplication.
2. Confirm `go.mod` and `go.sum` retain tagged Promptkit v0.5.0 without a local
   replacement or dependency drift.
3. Confirm only the four supported report products exist and no retired report
   surfaces were reintroduced.
4. Confirm the roadmap's completion criteria one by one. Change its status to
   implemented and this plan's status to completed only after every criterion
   and command below passes.
5. Do not require a live provider for completion. If credentials and network
   access are deliberately supplied by an operator, record live smoke results
   separately as release-candidate evidence rather than adding them to the
   default suite.

### Verification

Run `gofmt -w` on every changed Go file, then run:

```sh
go test ./...
go test -race ./...
go run ./cmd/weatherreporter --help
git diff --check
git status --short
```

Also inspect all three logical profiles through the application's normal
preflight path using offline provider doubles, including one same-ID local
override and one explicit global override.

### Exit Gate

All roadmap completion criteria are satisfied, all verification commands pass,
the working tree contains only intentional changes, and the canonical
documentation describes the implemented state. The feature is ready for code
review and release preparation.

## Post-Implementation Review

Stages 1–6 implemented the intended production behavior and passed their
offline verification gates. A subsequent review found no high-severity runtime
defect, but identified three test-quality issues and one remaining validation
obligation:

- one app test asserted durable preparation and execution artifact provenance,
  contrary to the active-execution boundary and accepted ephemeral-state
  direction;
- an adapter-package test depended upward on app orchestration and duplicated
  test ownership;
- embedded fallback profiles were inspected but not exercised through one
  prepared execution with a provider fake; and
- the roadmap's representative model-evaluation policy had no recorded
  evidence.

Stages 7 and 8 address those findings without changing the profile catalog,
selection precedence, report assignments, prompt content, generated-text
schemas, or default offline test contract.

## Stage 7: Correct Test Ownership And Fallback Execution Coverage

### Goal

Remove accidental durable-state and cross-layer test commitments while adding
one focused offline execution test for the embedded fallback path.

### Work

1. Rewrite `TestGenerateDetailedPreservesSelectedProfileThroughExecution` so
   it protects active workflow behavior only:

   - retain the Hourly default, day-scale default, and global-override cases;
   - assert the profile ID sent in `promptexec.ExecuteRequest`;
   - have the executor fake record the preparation and execution values it
     emits, then assert their logical profile ID and effective backend/model;
   - do not load preparation, execution, or metadata files to establish a
     durable profile-provenance contract; and
   - remove artifact-content scans whose fake inputs cannot contain an endpoint
     or credential.

2. Preserve meaningful safety coverage at the boundary that can expose the
   sensitive value:

   - retain adapter mapping coverage proving an endpoint from a real Promptkit
     profile does not enter `promptexec.ProfileInspection`;
   - retain app error coverage proving dependency errors containing an endpoint
     or credential are replaced by a bounded classified error; and
   - do not add profile endpoints or credentials to project-owned execution
     types merely to make a leakage test possible.

3. Remove `internal/app`, app configuration, and report-registry dependencies
   from `internal/adapters/promptkit/adapter_test.go`. Move the assembled
   application-preflight test to a new app-owned external integration test,
   such as `internal/app/prompt_profile_integration_test.go` with package
   `app_test`:

   - construct the real Promptkit adapter through its public `New` function;
   - call the public app prompt-inspection operation;
   - supply a deterministic credential lookup rather than reading the process
     environment; and
   - cover Hourly, one representative day-scale default, the explicit
     `weather-deep` global override, and a same-ID endpoint-only
     `weather-light` override. The asset contract tests already own the exact
     mapping for all four prompts, so the integration test need not repeat all
     four.

4. Add one adapter-owned, offline fake-client execution test using the real
   embedded Hourly prompt at `1.1.0` and selected profile `weather-light`.
   Execute through the normal prepared adapter path and assert:

   - the preparation callback runs before the fake provider;
   - preparation and execution report logical profile `weather-light`, backend
     `openrouter`, and model `deepseek/deepseek-v4-flash`;
   - the fake provider request targets `deepseek/deepseek-v4-flash`; and
   - schema validation completes without contacting a live service.

   One execution case is sufficient because Promptkit owns uniform source
   precedence and the adapter's inspection tests already cover fallback,
   operator file, operator directory, built-in, and explicit in-memory layers.

5. Reconcile the profile-related current-state documentation:

   - it may accurately describe fields present in current preparation and
     execution receipts;
   - it must not promise cross-version readability or characterize those
     receipts as the profile feature's durable target architecture; and
   - troubleshooting should prefer active command errors and explicit secure
     debug capture, mentioning current-version receipts only as transitional
     state if they remain useful before the ephemeral-state refactor.

6. Do not change production profile resolution, prompt definitions, state
   schemas, artifact validators, or the ephemeral-state roadmap in this stage.

### Tests

Run:

```sh
go test -count=1 ./internal/promptassets ./internal/adapters/promptkit ./internal/app ./internal/cli
go test -count=1 -race ./internal/promptassets ./internal/adapters/promptkit ./internal/app ./internal/cli
go test -count=1 ./...
go vet ./...
go run ./cmd/weatherreporter --help
git diff --check
```

Review the changed tests against the testing policy and confirm that adapter
tests own adapter behavior, app tests own orchestration, and state tests remain
the sole owner of durable artifact format and validation details.

### Exit Gate

Active profile selection and effective-model propagation remain protected
without adding a durable-provenance commitment; the adapter test package no
longer imports the app layer; one embedded fallback profile completes prepared
execution through a provider fake; and every required check passes offline.

## Stage 8: Evaluate The Initial Model Ladder

### Goal

Produce explicit release-candidate evidence that the selected models are
acceptable for their intended report tiers and that a representative local
override provides the promised operator experience.

This is an opt-in evaluation stage, not an ordinary automated-test stage. It
requires operator-approved provider credentials, network access, and a local
OpenAI-compatible endpoint. Do not mark it complete when those prerequisites
are unavailable; report the missing prerequisite instead.

### Corpus

Use four representative, secret-free YAML data packages: one each for Daily,
Today, Tomorrow, and Hourly. The set must include at least one package with
precipitation windows and at least one with none. Remove precise private
location identifiers or other operationally sensitive values without changing
the meteorological relationships being evaluated.

Record a SHA-256 hash and a short, non-sensitive description for each package.
Do not commit full packages or generated prose unless the user separately
approves them as repository fixtures.

### Execution Matrix

Run these six evaluations from the exact package bytes:

| Case | Package | Profile |
| --- | --- | --- |
| Hourly default | Hourly | `weather-light` |
| Daily default | Daily | `weather-balanced` |
| Today default | Today | `weather-balanced` |
| Tomorrow default | Tomorrow | `weather-balanced` |
| Deep comparison | The same Daily package used above | `weather-deep` |
| Local override | The same Hourly package used above | operator-defined `weather-light` endpoint profile |

After the successful local-override case, stop or deliberately address an
unavailable test endpoint and repeat it as a negative control. Confirm that the
request fails visibly and does not call or select an embedded remote profile.
This negative control is not an additional quality-evaluation case.

Use a temporary, untracked evaluation harness beneath the module when exact
package replay is needed. It should call the existing Promptkit adapter and
project-owned execution contract rather than duplicate prompt loading,
rendering, or schema validation. Remove the harness and all unapproved raw
outputs before completing the stage. Never print or record credentials.

### Evaluation Record

Add a concise `## Evaluation Record` section to
`docs/roadmap/domain-profiles.md`. For every case, record:

- evaluation date, logical profile, effective backend, and exact model
  reported by execution;
- corpus hash, validation outcome, latency, prompt/completion/total token use,
  and provider-reported or contemporaneously calculated cost;
- whether every generated claim is supported by the deterministic package;
- whether hazards, periods, uncertainty, and precipitation timing are used
  correctly;
- whether `precipitation_timing` is exactly an empty string for the no-window
  case;
- a short usefulness assessment for summary and forecast discussion; and
- any provider, alias, or local-endpoint caveat observed.

Do not include credentials, endpoints, complete effective parameter maps,
full data packages, rendered prompts, or full generated responses in the
record. The secure debug directory may be used temporarily for operator review
and remains operator-managed.

### Acceptance Rules

- Every case must complete strict JSON Schema validation without repair.
- Generated prose must contain no material unsupported weather claim or
  contradiction of deterministic hazards, periods, or uncertainty.
- Precipitation timing must agree with the deterministic windows and use the
  required empty-string representation when no window exists.
- The local override must select the operator model without modifying a prompt
  or application code and must not fall back to a remote profile when the local
  endpoint is unavailable.
- Latency, tokens, and cost must be recorded, but this initial evaluation does
  not impose an invented numeric threshold. The operator decides whether the
  observed tradeoff remains acceptable for the named tier.
- If a default case fails schema or factual acceptance, do not weaken the
  schema or prompt to accommodate the model. Reopen the concrete model or
  profile-setting decision in the feature roadmap and leave this stage
  incomplete.

### Verification

After removing temporary evaluation material, run:

```sh
go test -count=1 ./...
git diff --check
git status --short
```

Confirm that the only intended repository change from this stage is the
concise evaluation record and any roadmap status correction required by its
result. Do not add live credentials, provider-dependent tests, a permanent
benchmark framework, or release notes before a release version is selected.

### Exit Gate

All six cases satisfy the acceptance rules, the roadmap contains concise and
safe evaluation evidence, no temporary corpus or response material remains in
the repository, and the default suite remains offline. Set this plan back to
`Status: Completed` only after both Stages 7 and 8 have passed.

## Open Questions

None. The model identifiers, profile settings, report assignments, version
transition, precedence, compatibility behavior, test boundaries, and
documentation ownership are decision-complete.