diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md index 55a5d4d..81c82ff 100644 --- a/docs/roadmap/remediation.md +++ b/docs/roadmap/remediation.md @@ -6,6 +6,11 @@ compared with the [adoption roadmap](library-adoption.md) and pre-sprint revision `40420a2`. **Status:** Open. +The follow-up integration audit compared the two binaries against a loopback +provider. A normal grammar correction produced the same transcript and +completed both validators on each revision. Differences at the output, +credential, and glossary boundaries are recorded in R10–R12. + ## Architecture Assessment ConfigKit selects and composes the v3 application configuration, including @@ -25,11 +30,11 @@ validation, source publication, and asynchronous Linux delivery. | Stages | Reviewed behavior | | --- | --- | | 1 and 15 | Dependency and toolchain pins, release profile, launcher, tag verification, and candidate evidence. A disposable ReleaseKit check passed with Go 1.27.3, including race tests, vet, Markdown and module checks, both Linux builds, and version output. The plan records the annotated-tag delivery rehearsal. | -| 2–4 | Native application load, main-relative engine inputs, prompt resources and source accounting, prepared role clients, retries and repairs, and redaction. See R2 and R4. | +| 2–4 | Native application load, main-relative engine inputs, prompt resources and source accounting, prepared role clients, retries and repairs, and redaction. See R2, R4, and R10. | | 5–6 | Native app and PromptKit scopes, typed routing, preview, validation, and explicit smoke, including controlled transport tests. See R5. | -| 7–8 | Destination checks, deferred device/pipe delivery, transcript replacement, ordinary reports, optional debug, and reporting modes. See R1, R7, and R9. | +| 7–8 | Destination checks, deferred device/pipe delivery, transcript replacement, ordinary reports, optional debug, and reporting modes. See R1, R7, R9, and R11. | | 9–10 | Native identified glossary composition, typed extensions and selections, recorded defaults, and explicit clearing after CLI assignments. | -| 11–12 | Stage-specific glossary views, late-stage readiness, fixed chunks, ordered mutation, and partial success. See R3 and R9. | +| 11–12 | Stage-specific glossary views, late-stage readiness, fixed chunks, ordered mutation, and partial success. See R3, R9, and R12. | | 13–14 | Stage attribution, report schemas, canonical docs, migration, examples, ADR, and attribution. See R1, R6, R7, and R8. | ## R1 — P2: Keep Failure Records and Counts Consistent @@ -238,6 +243,9 @@ implements directory selection separately. The former only test callers. The runner still collects validator timing for the latter. The single `GlossaryPath` survives through report data and prints an empty `Glossary input:` line after the standalone glossary loader was removed. +The debug invocation record still emits `modules: null` on a run with active +stages. The CLI writes resolved stages to the effective configuration artifact +instead. [Invocation metadata](../../internal/cli/process_inputs.go#L104) **Recommended change** @@ -264,6 +272,10 @@ Documentation divergence in Stage 14. “Audita v3”, although Audita 3.x already shipped with application schema v2. It also omits the change that default-mode processing hides completion counts, including when ordinary reports are disabled. +It leaves out the rename from legacy glossary `summary` to `description`. +A copied `summary` is retained as an extension, so migration can appear +successful while the existing prompt builder loses that context. +[Legacy format](../migrations/v2.md) · [Glossary projection](../../internal/core/glossary/selection.go#L29) [Subprocess guidance](../integrations/subprocess.md#result-handling) says failed reports can describe invalid input after initialization. Transcript parsing now occurs before report preparation. @@ -275,12 +287,14 @@ but still links the old format description to the current configuration guide. Distinguish application schema v3 from the executable release version. Explain how to see completion counts at the cutover, link historical format guidance to its applicable revision, and align subprocess -guidance with early transcript validation. +guidance with early transcript validation. Include the glossary field mapping +so operators can preserve their prompt context. **Acceptance** A v3.1.x operator can identify which schema is changing and -how to observe completed runs. Current guidance explains which failures can +how to observe completed runs. A migrated glossary entry retains the intended +description in prompt context. Current guidance explains which failures can leave reports. Historical references continue to describe their own versions. ## R9 — P2: Honor Cancellation in Normalization-Only Processing @@ -310,3 +324,96 @@ A controlled cancellation after preparation and before successful delivery of an empty-pipeline run leaves the final transcript untouched and attempts failed-status reports. An uncanceled empty pipeline still produces normalized output without provider calls. + +## R10 — P2: Mask the Credential Value PromptKit Actually Sends + +**Category** + +Credential normalization and diagnostic redaction disagree in Stage 4. + +**Evidence** + +PromptKit trims surrounding whitespace from its environment credential before +setting the Authorization header. Audita supplies the literal environment +value to its existing exact-value redactor. With a synthetic credential padded +by spaces and a tab, a loopback provider observed the trimmed value and echoed +it in a schema-invalid response. The run completed with a recoverable model +error and wrote the trimmed credential to its proposal response artifact. +The same response with an unpadded credential was masked. +[Credential collection](../../internal/framework/llm/secrets.go#L40) · +[Proposal debug output](../../internal/framework/proposal_generation/generate.go#L155) + +**Recommended change** + +Supply both raw and normalized nonempty values for referenced credential +variables to the existing redactor. Keep credential selection in PromptKit. + +**Acceptance** + +A controlled provider that echoes the effective value of a padded synthetic +credential leaves neither representation in retained diagnostics, ordinary +reports, or surfaced errors. Credential availability and processing outcomes +keep their native behavior. + +## R11 — P2: Preserve Pipes That Appear After Output Preparation + +**Category** + +The new Stage 7 preparation changes the existing special-file delivery behavior. + +**Evidence** + +Preparation classifies a missing destination as a regular-file output. Final +delivery then renames the staged transcript over that path without checking +its current type. In a controlled run, the provider fixture created a named +pipe at the selected path after the first proposal request arrived. The +previous binary wrote the transcript to the pipe and left it in place. The +new binary replaced it with a regular file, and both reported success. +[Output preparation](../../internal/core/diagnostics/output_session.go#L140) · +[Transcript publication](../../internal/core/diagnostics/output_session.go#L309) + +**Recommended change** + +Recheck the destination type at delivery using the existing filesystem +behavior. Deliver to a current pipe or device directly, or return a clear +failure if its change makes the prepared destination unusable. Keep the +ordinary per-file replacement and temporary-file cleanup. + +**Acceptance** + +A FIFO created after preparation survives delivery. An existing FIFO remains +unopened during preflight. Ordinary destinations still publish from the +prepared sibling file, and failed delivery cleans up that file. + +## R12 — P2: Preserve Longest-Match Context for Glossary Exemptions + +**Category** + +The new Stage 11 exemption changes how the existing matcher identifies +overlapping terms. + +**Evidence** + +The projection removes active spellings before building the protected-term +matcher. That lets a shorter protected spelling match inside an exempt active +name. With `Greater Mage Armor` and `Greater Magus Armor` active and +`Mage Armor` protected from another collection, the previous glossary stage +accepted the full-name correction under controlled validation. The new stage +rejected it before either model validator ran. It treated the suffix of the +active name as a separate protected occurrence. +[Vocabulary projection](../../internal/core/glossary/vocabulary.go#L19) · +[Longest-first matcher](../../internal/framework/validators/protected_terms.go#L29) · +[Deterministic validator](../../internal/framework/validators/deterministic.go#L139) + +**Recommended change** + +Keep active spellings available as match boundaries and exempt the matched +active occurrence from protection. Apply the same interpretation in +deterministic validation and when evaluating the edited segment at apply time. + +**Acceptance** + +An exempt full name can be corrected when a shorter protected term occurs +inside it. A standalone occurrence of the shorter term and a distinct longer +protected name retain protection. Other module stages keep their independently +selected vocabulary.