Record the library integration audit findings
This commit is contained in:
@@ -6,6 +6,11 @@ compared with the [adoption roadmap](library-adoption.md) and pre-sprint
|
||||
revision `40420a2`.
|
||||
**Status:** Open.
|
||||
|
||||
The follow-up integration audit compared the two binaries against a loopback
|
||||
provider. A normal grammar correction produced the same transcript and
|
||||
completed both validators on each revision. Differences at the output,
|
||||
credential, and glossary boundaries are recorded in R10–R12.
|
||||
|
||||
## Architecture Assessment
|
||||
|
||||
ConfigKit selects and composes the v3 application configuration, including
|
||||
@@ -25,11 +30,11 @@ validation, source publication, and asynchronous Linux delivery.
|
||||
| Stages | Reviewed behavior |
|
||||
| --- | --- |
|
||||
| 1 and 15 | Dependency and toolchain pins, release profile, launcher, tag verification, and candidate evidence. A disposable ReleaseKit check passed with Go 1.27.3, including race tests, vet, Markdown and module checks, both Linux builds, and version output. The plan records the annotated-tag delivery rehearsal. |
|
||||
| 2–4 | Native application load, main-relative engine inputs, prompt resources and source accounting, prepared role clients, retries and repairs, and redaction. See R2 and R4. |
|
||||
| 2–4 | Native application load, main-relative engine inputs, prompt resources and source accounting, prepared role clients, retries and repairs, and redaction. See R2, R4, and R10. |
|
||||
| 5–6 | Native app and PromptKit scopes, typed routing, preview, validation, and explicit smoke, including controlled transport tests. See R5. |
|
||||
| 7–8 | Destination checks, deferred device/pipe delivery, transcript replacement, ordinary reports, optional debug, and reporting modes. See R1, R7, and R9. |
|
||||
| 7–8 | Destination checks, deferred device/pipe delivery, transcript replacement, ordinary reports, optional debug, and reporting modes. See R1, R7, R9, and R11. |
|
||||
| 9–10 | Native identified glossary composition, typed extensions and selections, recorded defaults, and explicit clearing after CLI assignments. |
|
||||
| 11–12 | Stage-specific glossary views, late-stage readiness, fixed chunks, ordered mutation, and partial success. See R3 and R9. |
|
||||
| 11–12 | Stage-specific glossary views, late-stage readiness, fixed chunks, ordered mutation, and partial success. See R3, R9, and R12. |
|
||||
| 13–14 | Stage attribution, report schemas, canonical docs, migration, examples, ADR, and attribution. See R1, R6, R7, and R8. |
|
||||
|
||||
## R1 — P2: Keep Failure Records and Counts Consistent
|
||||
@@ -238,6 +243,9 @@ implements directory selection separately. The former
|
||||
only test callers. The runner still collects validator timing for the latter.
|
||||
The single `GlossaryPath` survives through report data and prints an empty
|
||||
`Glossary input:` line after the standalone glossary loader was removed.
|
||||
The debug invocation record still emits `modules: null` on a run with active
|
||||
stages. The CLI writes resolved stages to the effective configuration artifact
|
||||
instead. [Invocation metadata](../../internal/cli/process_inputs.go#L104)
|
||||
|
||||
**Recommended change**
|
||||
|
||||
@@ -264,6 +272,10 @@ Documentation divergence in Stage 14.
|
||||
“Audita v3”, although Audita 3.x already shipped with application schema v2.
|
||||
It also omits the change that default-mode processing hides completion
|
||||
counts, including when ordinary reports are disabled.
|
||||
It leaves out the rename from legacy glossary `summary` to `description`.
|
||||
A copied `summary` is retained as an extension, so migration can appear
|
||||
successful while the existing prompt builder loses that context.
|
||||
[Legacy format](../migrations/v2.md) · [Glossary projection](../../internal/core/glossary/selection.go#L29)
|
||||
[Subprocess guidance](../integrations/subprocess.md#result-handling) says
|
||||
failed reports can describe invalid input after initialization. Transcript
|
||||
parsing now occurs before report preparation.
|
||||
@@ -275,12 +287,14 @@ but still links the old format description to the current configuration guide.
|
||||
Distinguish application schema v3 from the executable
|
||||
release version. Explain how to see completion counts at the cutover, link
|
||||
historical format guidance to its applicable revision, and align subprocess
|
||||
guidance with early transcript validation.
|
||||
guidance with early transcript validation. Include the glossary field mapping
|
||||
so operators can preserve their prompt context.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
A v3.1.x operator can identify which schema is changing and
|
||||
how to observe completed runs. Current guidance explains which failures can
|
||||
how to observe completed runs. A migrated glossary entry retains the intended
|
||||
description in prompt context. Current guidance explains which failures can
|
||||
leave reports. Historical references continue to describe their own versions.
|
||||
|
||||
## R9 — P2: Honor Cancellation in Normalization-Only Processing
|
||||
@@ -310,3 +324,96 @@ A controlled cancellation after preparation and before
|
||||
successful delivery of an empty-pipeline run leaves the final transcript
|
||||
untouched and attempts failed-status reports. An uncanceled empty pipeline
|
||||
still produces normalized output without provider calls.
|
||||
|
||||
## R10 — P2: Mask the Credential Value PromptKit Actually Sends
|
||||
|
||||
**Category**
|
||||
|
||||
Credential normalization and diagnostic redaction disagree in Stage 4.
|
||||
|
||||
**Evidence**
|
||||
|
||||
PromptKit trims surrounding whitespace from its environment credential before
|
||||
setting the Authorization header. Audita supplies the literal environment
|
||||
value to its existing exact-value redactor. With a synthetic credential padded
|
||||
by spaces and a tab, a loopback provider observed the trimmed value and echoed
|
||||
it in a schema-invalid response. The run completed with a recoverable model
|
||||
error and wrote the trimmed credential to its proposal response artifact.
|
||||
The same response with an unpadded credential was masked.
|
||||
[Credential collection](../../internal/framework/llm/secrets.go#L40) ·
|
||||
[Proposal debug output](../../internal/framework/proposal_generation/generate.go#L155)
|
||||
|
||||
**Recommended change**
|
||||
|
||||
Supply both raw and normalized nonempty values for referenced credential
|
||||
variables to the existing redactor. Keep credential selection in PromptKit.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
A controlled provider that echoes the effective value of a padded synthetic
|
||||
credential leaves neither representation in retained diagnostics, ordinary
|
||||
reports, or surfaced errors. Credential availability and processing outcomes
|
||||
keep their native behavior.
|
||||
|
||||
## R11 — P2: Preserve Pipes That Appear After Output Preparation
|
||||
|
||||
**Category**
|
||||
|
||||
The new Stage 7 preparation changes the existing special-file delivery behavior.
|
||||
|
||||
**Evidence**
|
||||
|
||||
Preparation classifies a missing destination as a regular-file output. Final
|
||||
delivery then renames the staged transcript over that path without checking
|
||||
its current type. In a controlled run, the provider fixture created a named
|
||||
pipe at the selected path after the first proposal request arrived. The
|
||||
previous binary wrote the transcript to the pipe and left it in place. The
|
||||
new binary replaced it with a regular file, and both reported success.
|
||||
[Output preparation](../../internal/core/diagnostics/output_session.go#L140) ·
|
||||
[Transcript publication](../../internal/core/diagnostics/output_session.go#L309)
|
||||
|
||||
**Recommended change**
|
||||
|
||||
Recheck the destination type at delivery using the existing filesystem
|
||||
behavior. Deliver to a current pipe or device directly, or return a clear
|
||||
failure if its change makes the prepared destination unusable. Keep the
|
||||
ordinary per-file replacement and temporary-file cleanup.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
A FIFO created after preparation survives delivery. An existing FIFO remains
|
||||
unopened during preflight. Ordinary destinations still publish from the
|
||||
prepared sibling file, and failed delivery cleans up that file.
|
||||
|
||||
## R12 — P2: Preserve Longest-Match Context for Glossary Exemptions
|
||||
|
||||
**Category**
|
||||
|
||||
The new Stage 11 exemption changes how the existing matcher identifies
|
||||
overlapping terms.
|
||||
|
||||
**Evidence**
|
||||
|
||||
The projection removes active spellings before building the protected-term
|
||||
matcher. That lets a shorter protected spelling match inside an exempt active
|
||||
name. With `Greater Mage Armor` and `Greater Magus Armor` active and
|
||||
`Mage Armor` protected from another collection, the previous glossary stage
|
||||
accepted the full-name correction under controlled validation. The new stage
|
||||
rejected it before either model validator ran. It treated the suffix of the
|
||||
active name as a separate protected occurrence.
|
||||
[Vocabulary projection](../../internal/core/glossary/vocabulary.go#L19) ·
|
||||
[Longest-first matcher](../../internal/framework/validators/protected_terms.go#L29) ·
|
||||
[Deterministic validator](../../internal/framework/validators/deterministic.go#L139)
|
||||
|
||||
**Recommended change**
|
||||
|
||||
Keep active spellings available as match boundaries and exempt the matched
|
||||
active occurrence from protection. Apply the same interpretation in
|
||||
deterministic validation and when evaluating the edited segment at apply time.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
An exempt full name can be corrected when a shorter protected term occurs
|
||||
inside it. A standalone occurrence of the shorter term and a distinct longer
|
||||
protected name retain protection. Other module stages keep their independently
|
||||
selected vocabulary.
|
||||
|
||||
Reference in New Issue
Block a user