Record test coverage and effectiveness audit findings
This commit is contained in:
355
docs/roadmap/audit/08-testing.md
Normal file
355
docs/roadmap/audit/08-testing.md
Normal file
@@ -0,0 +1,355 @@
|
||||
# Test coverage and effectiveness audit
|
||||
|
||||
Reviewed 2026-11-15 at `416cdea`. The audit used the canonical testing policy to
|
||||
assess protection against consequential failures, test ownership, fixture
|
||||
fidelity, and maintenance cost. The findings are follow-up options.
|
||||
[Testing policy](../../policy/testing.md)
|
||||
|
||||
## Architecture and measured coverage
|
||||
|
||||
Audita is a single-process Go CLI. CommandKit handles invocation, ConfigKit
|
||||
loads application settings, and core packages parse, normalize, and chunk the
|
||||
transcript. The runner coordinates correction proposals, validators, repairs,
|
||||
and edit application. The role adapter uses a shared PromptKit engine, and the
|
||||
CLI publishes the transcript and reports through an output session. Tests use
|
||||
pure domain inputs, temporary files, an injected HTTP transport or loopback
|
||||
provider, and subprocesses. Release tests in this repository inspect the local
|
||||
profile, while ReleaseKit owns the underlying delivery implementation.
|
||||
[Package map](../../internal/overview.md) · [Pipeline](../../internal/pipeline.md)
|
||||
· [LLM runtime](../../internal/llm-runtime.md)
|
||||
· [Release procedure](../../release.md#binary-delivery)
|
||||
|
||||
On Go 1.27.3, the full `go test -count=1 -race -covermode=atomic -coverpkg=./...`
|
||||
run passed with `-coverprofile` and `-json` output directed to temporary files.
|
||||
Go's cover tool reported **89.3%**
|
||||
statement coverage. The JSON run log recorded 441 passing top-level tests and
|
||||
210 passing named subtests. The package percentages below merge repeated
|
||||
profile rows by source block before counting statements.
|
||||
|
||||
| Package or boundary | Statement coverage | Audit observation |
|
||||
| --- | ---: | --- |
|
||||
| `cmd/audita` | 0.0% | The test executable handles signals in a helper. |
|
||||
| `internal/framework/preflight` | 73.8% | `Prepare` is at 63.9%, including gaps around cancellation and report recovery. |
|
||||
| `internal/core/normalization` | 78.6% | Several merge cases exercise ordinary timestamps. |
|
||||
| `internal/core/diagnostics` | 81.4% | Tests exercise publication and cleanup, with gaps in failure sequencing. |
|
||||
| `internal/core/schema` | 83.3% | Source decoding has shape and validation cases, with a precision gap. |
|
||||
| `internal/cli` | 87.6% | Command invocation is covered more broadly than failure classification. |
|
||||
| `internal/framework/runner` | 91.3% | Partial outcomes, repair, and cancellation are exercised. |
|
||||
| `internal/framework/llm` | 91.4% | Adapter failures and shared capacity are covered with conditions noted below. |
|
||||
| `internal/core/chunking` | 94.5% | The explicit-target builder is at 85.2%. |
|
||||
|
||||
The percentages describe statements reached in this run. The audit assigned
|
||||
test value by the assertions and failure scenario. In particular, low coverage
|
||||
in test support or thin forwarding wrappers does not justify standalone tests.
|
||||
Existing higher-value checks include partial validation with surviving edits,
|
||||
repair revalidation, typed failure causes, stable user-visible identities, and
|
||||
the presence of a masked credential echo in an actual retained artifact.
|
||||
[Coverage policy](../../policy/testing.md#coverage)
|
||||
· [Partial validation](../../../cmd/audita/main_integration_test.go#L1091)
|
||||
· [Repair integration](../../../cmd/audita/correction_repair_integration_test.go)
|
||||
· [Failure identity](../../../internal/framework/structuredoutput/failure_test.go#L10)
|
||||
· [Credential echo](../../../cmd/audita/main_integration_test.go#L965)
|
||||
|
||||
## Behavior that needs stronger verification
|
||||
|
||||
### 1. Subprocess tests recreate the production executable boundary
|
||||
|
||||
The subprocess suite launches the test executable. Its helper registers
|
||||
SIGINT and SIGTERM, passes arguments and streams to `cli.Run`, and exits with
|
||||
the returned code. The production `run` and `main` contain their own copy of
|
||||
that wiring and recorded zero hits. A regression in production-only signal
|
||||
registration, argument slicing, or exit propagation can pass the subprocess
|
||||
suite, including the cancellation case. The existing tests still exercise
|
||||
real process boundaries and the assembled CLI.
|
||||
[Production entry](../../../cmd/audita/main.go#L12)
|
||||
· [Subprocess helper](../../../cmd/audita/main_integration_test.go#L28)
|
||||
· [Signal test](../../../cmd/audita/main_integration_test.go#L1059)
|
||||
|
||||
Use the actual built executable for one representative startup and signal or
|
||||
exit test, or have the helper invoke the same production startup function.
|
||||
Keep the smaller existing seams for the rest of the CLI suite.
|
||||
|
||||
### 2. Inspection failure tests accept the wrong exit classification
|
||||
|
||||
`TestResourceCommandsClassifyOperatorPromptErrorsAsInputErrors` and
|
||||
`TestResourceCommandKeepsCallerCancellationAsFailure` reject only exit zero.
|
||||
Both can pass if the integration maps invalid input and cancellation to the
|
||||
same nonzero status. A temporary build given the same malformed operator
|
||||
prompt file returned `1` via `promptkit list prompts` and `2` via `process`.
|
||||
The CLI contract distinguishes input failure (`2`) from runtime failure (`1`).
|
||||
The input test also sets a synthetic credential that its malformed templates,
|
||||
profiles, and paths never reference, so the absence assertion provides no
|
||||
evidence that a configured secret was found and redacted.
|
||||
[Tests and fixtures](../../../internal/cli/runtime_errors_test.go#L24)
|
||||
· [CLI exit contract](../../cli.md#inspection-and-outcomes)
|
||||
· [CLI wiring](../../../internal/cli/app_wiring.go#L80)
|
||||
|
||||
Assert the selected status and a stable diagnostic fragment or typed cause.
|
||||
Clarify the intended delegated-command classification if it differs from the
|
||||
CLI contract. Test redaction where a known value reaches the relevant channel,
|
||||
as the existing credential echo test does, and remove the unused secret
|
||||
assertion from these cases.
|
||||
[Nonvacuous redaction test](../../../cmd/audita/main_integration_test.go#L965)
|
||||
|
||||
### 3. Output tests primarily check the final artifact state
|
||||
|
||||
The short-write CLI test asserts final YAML status, counts, and a reason code.
|
||||
The replacement test checks the result after a failed-report pass. Both can
|
||||
pass when finalization briefly writes `completed` reports after it already
|
||||
knows transcript delivery failed. The short-write test also passes if the
|
||||
required run-error phase is missing. Collision tests cover aliases present
|
||||
during selection, while the finalization path resolves report destinations
|
||||
again. Existing tests that change the transcript's file type exercise a
|
||||
different path. Prior audits documented both the transient report state and
|
||||
the changed report referent.
|
||||
[CLI short-write test](../../../internal/cli/process_output_test.go#L79)
|
||||
· [Final report pass test](../../../internal/core/diagnostics/output_session_test.go#L207)
|
||||
· [Collision test](../../../internal/core/diagnostics/output_session_test.go#L14)
|
||||
· [Changed transcript type test](../../../internal/core/diagnostics/output_session_test.go#L435)
|
||||
· [Report publication finding](06-error-handling.md#1-finalization-publishes-completed-reports-after-it-knows-delivery-failed)
|
||||
· [Report path finding](04-concurrency.md#6-report-paths-can-change-referent-between-initialization-and-finalization)
|
||||
|
||||
At `OutputSession`, observe the first report state after a known delivery
|
||||
failure and verify that a changed report destination cannot defeat the checked
|
||||
identity. At the CLI boundary, assert the exit and serialized run phase. The
|
||||
failure-injection seam is useful because it observes publication state at an
|
||||
externally consequential boundary. Preserve the documented nontransactional
|
||||
behavior when a report error occurs after a transcript has been published.
|
||||
[Output lifecycle](../../operations.md#outputs-and-lifecycle)
|
||||
· [Run error fields](../../integrations/output-files.md#error-and-recovery-records)
|
||||
|
||||
The stream tests open a FIFO peer before final delivery and use a payload
|
||||
that fits in the pipe buffer. The cancellation tests exercise a caller
|
||||
canceled before delivery or while the provider is active. A supported stream
|
||||
blocked in delivery therefore falls outside their assertions. Verify bounded
|
||||
subprocess cancellation and failed-status finalization after the supported
|
||||
stream semantics are decided. Keep the existing small delivery tests for the
|
||||
successful stream contract.
|
||||
[FIFO fixture](../../../internal/core/diagnostics/output_stream_test.go#L13)
|
||||
· [Pre-delivery cancellation test](../../../internal/cli/process_output_test.go#L131)
|
||||
· [Blocked-stream finding](04-concurrency.md#1-blocked-output-delivery-can-ignore-cancellation-and-report-success)
|
||||
|
||||
Coverage recorded no hits in `Prepare`'s cancellation handling after acquiring
|
||||
the transcript staging file or initializing the reports, and no hits in `Close`
|
||||
after a temporary path removal error. The existing cleanup tests cover a write
|
||||
or rename failure followed by successful cleanup. A well-placed preflight test
|
||||
can observe output and report state when cancellation follows initialization.
|
||||
At the output-session boundary, verify which cleanup errors are reported and
|
||||
whether a temporary removal failure can be retried under the selected contract.
|
||||
[Preflight cancellation](../../../internal/framework/preflight/preflight.go#L61)
|
||||
· [Existing cleanup test](../../../internal/core/diagnostics/output_session_test.go#L394)
|
||||
· [Earlier cleanup finding](05-resources.md#1-failed-output-cleanup-loses-ownership-of-temporary-paths)
|
||||
|
||||
### 4. The timeout tests cover two of the three relevant deadline owners
|
||||
|
||||
The adapter test distinguishes a shorter Audita operation budget from an
|
||||
already canceled parent. Other adapter cases check provider statuses and
|
||||
transport failures. They do not exercise a profile's HTTP attempt deadline
|
||||
expiring under a still-live parent and a longer operation budget. The earlier
|
||||
CLI audit observed that case labeled `operation_timeout`, even though
|
||||
PromptKit reserves its operation sentinel for the logical budget.
|
||||
[Timeout and cancellation test](../../../internal/framework/llm/adapter_test.go#L406)
|
||||
· [Provider status test](../../../internal/framework/llm/adapter_test.go#L462)
|
||||
· [Earlier timeout finding](06-error-handling.md#2-a-provider-request-timeout-is-reported-as-the-logical-operation-deadline)
|
||||
|
||||
Put the deadline distinction at the adapter boundary using a controlled
|
||||
transport and deliberately separated deadlines. Assert both recoverability
|
||||
and the selected cause or reason. The CLI already has report mapping tests
|
||||
and only needs representative coverage of the adapter-to-report connection.
|
||||
[Timeout contract](../../integrations/openai-compatible-llm.md)
|
||||
|
||||
### 5. The validator batch fixture misses the real numeric decoding boundary
|
||||
|
||||
The subprocess mixed-response test and the validator reconciliation tests
|
||||
protect sibling decisions when ordinary integer references are missing,
|
||||
duplicated, or unexpected. The adapter has a separate overflow test with a
|
||||
single out-of-range reference and no usable sibling. The gap is a response
|
||||
with both a usable requested decision and an unexpected integer too large for
|
||||
the Go index type. Native schema validation can accept that integer, but the
|
||||
typed adapter decode can reject the whole batch before per-entry reconciliation.
|
||||
The existing overflow test provides evidence for failure classification and
|
||||
cannot establish sibling preservation.
|
||||
[Mixed-response integration](../../../cmd/audita/main_integration_test.go#L567)
|
||||
· [Adapter overflow case](../../../internal/framework/llm/adapter_test.go#L429)
|
||||
· [Earlier reference decoding finding](01-correctness.md#7-an-extra-numeric-validator-reference-can-discard-valid-sibling-decisions)
|
||||
|
||||
Exercise a mixed response through the actual provider envelope, schema, and
|
||||
adapter decode. Assert the usable sibling's decision and the appropriate
|
||||
disposition for the extra entry. A stub returning a pretyped integer cannot
|
||||
cover this case. Keep smaller reconciliation tests for the ordinary identity
|
||||
rules.
|
||||
[Validator reconciliation contract](../../internal/validators.md#llm-validator-batching)
|
||||
|
||||
### 6. Concurrency fixtures leave some schedules unproved
|
||||
|
||||
The runner's proposal-order test waits for all proposals to start, then
|
||||
releases three gates back-to-back in reverse section order. That establishes a
|
||||
release order, while workers may deliver their results in a different order.
|
||||
The test checks applied-change order by target segment and final text, without
|
||||
asserting the proposal indices promised by its name. Order by segment can
|
||||
survive a change to proposal index assignment because corrections are applied
|
||||
within their sections. The runner has other fixed-chunk and application-order
|
||||
checks, so the missing evidence is deterministic indexing under a demonstrably
|
||||
different completion order.
|
||||
[Parallel runner test](../../../internal/framework/runner/runner_test.go#L423)
|
||||
· [Result assertions](../../../internal/framework/runner/runner_test.go#L473)
|
||||
· [Application and recombination](../../../internal/framework/runner/runner.go#L300)
|
||||
· [Deterministic indexing policy](../../policy/architecture.md#correction-and-concurrency-invariants)
|
||||
|
||||
The shared-backend test holds the first HTTP call, then gives the second call
|
||||
a 30 ms deadline. It does not verify that the second call reached admission
|
||||
or the wait before it timed out. On a delayed execution, an earlier timeout
|
||||
can satisfy the same assertions. The runner cancellation test observes native
|
||||
`ExecutionStarted` for both peers and one dispatch. That event precedes
|
||||
admission. Twenty race-enabled repetitions of the selected capacity,
|
||||
cancellation, and proposal-order tests passed. The repetitions exercise the
|
||||
present fixtures, while the state-observation gaps remain.
|
||||
[Shared capacity test](../../../internal/framework/llm/adapter_capacity_test.go#L42)
|
||||
· [Peer cancellation test](../../../internal/framework/runner/cancellation_test.go#L67)
|
||||
· [Distinct backend control](../../../internal/framework/llm/adapter_capacity_test.go#L96)
|
||||
|
||||
Hold the earlier result until later completion is observable and assert the
|
||||
actual assigned indices and section identities. At the adapter boundary, prove
|
||||
that work reached the contested state before canceling it. Observe peak active
|
||||
dispatches relative to a test-controlled limit and check that waiting work can
|
||||
proceed or time out according to the configured contract. Keep wall-clock
|
||||
deadlines as bounds for the test itself where possible.
|
||||
[Concurrency testing policy](../../policy/testing.md#separate-mechanism-from-policy)
|
||||
|
||||
### 7. Representative transcript fixtures leave data-integrity boundaries open
|
||||
|
||||
| Behavior | Current protection and gap | Follow-up |
|
||||
| --- | --- | --- |
|
||||
| Accepted source IDs remain distinct | Source parser tests cover both shapes and ordinary duplicate IDs. They miss adjacent integers within native `int` range that collapse in the intermediate generic JSON decode. | Assert exact typed IDs and duplicate detection at the precision boundary in the source parser, with a word-size-appropriate fixture. [Parser cases](../../../internal/core/schema/transcript_test.go#L133) · [Earlier fidelity finding](01-correctness.md#6-transcript-shape-detection-loses-integer-and-unknown-field-fidelity) |
|
||||
| A merged interval covers all source speech | Merge tests use increasing end times. The chronological test checks the outer start and end for disjoint inputs. A nested same-speaker interval can shorten the result without failing them. | Assert the applicable merged time envelope for overlapping or nested segments, along with the text or category invariants already owned by normalization. [Existing time assertions](../../../internal/core/normalization/normalize_test.go#L182) · [Earlier timestamp finding](01-correctness.md#3-merging-nested-segments-truncates-the-earlier-timestamp-range) |
|
||||
| An achievable explicit chunk target succeeds | The success case uses equal weights and ample capacity. The impossible cases check obvious upper and lower bounds. These can pass when a skewed but feasible partition is incorrectly rejected. | Use heterogeneous weights and a case near the limit. For broader confidence, a small exhaustive partition oracle can check feasibility, contiguity, order, and coverage without pinning the optimizer. [Target cases](../../../internal/core/chunking/sections_test.go#L279) · [Earlier feasibility finding](01-correctness.md#4-balanced-chunking-rejects-feasible-explicit-targets) |
|
||||
|
||||
### 8. Debug redaction assertions mostly inspect the outer serialization
|
||||
|
||||
The subprocess test confirms that it actually captured a provider echo, then
|
||||
checks command streams and retained files for the clear credential value. The
|
||||
redactor unit test has a useful nested `final_content` case built with Go's
|
||||
own JSON encoder. Neither asserts the decoded content of that nested field
|
||||
when the provider uses another legal JSON representation. A literal-byte
|
||||
absence check can pass while the credential survives there. This matters to
|
||||
the documented diagnostic redaction boundary, even when a private debug
|
||||
directory limits access on disk.
|
||||
[Subprocess echo assertions](../../../cmd/audita/main_integration_test.go#L965)
|
||||
· [Nested redaction test](../../../internal/core/redaction/redaction_test.go#L43)
|
||||
· [Earlier debug finding](07-security.md#4-raw-structured-debug-responses-can-retain-an-encoded-credential)
|
||||
|
||||
Use a synthetic credential and verify the semantic content after parsing the
|
||||
retained artifact and any structured response field it contains. Own detailed
|
||||
string representation cases at the diagnostic writer. Keep one real provider
|
||||
test to prove that credential discovery and artifact redaction connect.
|
||||
[Diagnostic trust policy](../../policy/architecture.md#trust-boundaries-and-threat-scope)
|
||||
|
||||
## Candidates to remove or consolidate
|
||||
|
||||
### 9. Delete proposal-order assertions on the test-only composition harness
|
||||
|
||||
`TestDeterministicProposalIndexAssignment` runs `testChunkProposalHarness`
|
||||
twice and checks indices that the harness itself assigns with `nextIndex++`.
|
||||
It cannot detect an indexing regression in `Runner.Run`. The harness also
|
||||
passes the full transcript into a fake module and delegates range filtering
|
||||
to that fake. Production builds section-local requests. Related harness cases
|
||||
verify a composition that the test constructs itself. Remove the index test
|
||||
once the real runner test checks indices under controlled completion. Migrate
|
||||
any remaining distinct input-preservation or stale-application assertion to
|
||||
the existing runner or proposal API coverage before removing its harness case.
|
||||
[Test-only orchestration](../../../internal/framework/contracts/chunk_proposal_composition_test.go#L14)
|
||||
· [Test-only index assertion](../../../internal/framework/contracts/chunk_proposal_composition_test.go#L143)
|
||||
· [Real runner section contract](../../../internal/framework/runner/runner_test.go#L356)
|
||||
· [Stable application test](../../../internal/framework/proposals/apply_test.go#L36)
|
||||
|
||||
### 10. Consolidate validator registry and asset metadata assertions
|
||||
|
||||
`TestExecutionClassResolvableByStableKeyAndByValidatorInstance` repeats the
|
||||
class comparison in `TestRegistryBuildsClassifiedValidators`. The second test
|
||||
also requires the concrete validator to implement `ClassifiedValidator` even
|
||||
though `ClassOf` supports a stable-key fallback. A behavior-preserving change
|
||||
to that representation would fail the type assertion. Keep one table that
|
||||
builds registered keys and checks effective classes against documented roles.
|
||||
Preserve protective checks of concrete domain behavior and stable selector keys.
|
||||
[Duplicate registry checks](../../../internal/validators/registry_test.go#L80)
|
||||
· [Explicit class examples](../../../internal/validators/registry_test.go#L48)
|
||||
· [Fallback behavior](../../../internal/validators/metadata/metadata_test.go#L54)
|
||||
|
||||
`TestRegisteredSchemasLoadReadableEmbeddedJSON` repeats JSON validity already
|
||||
enforced at registry construction. The native preparation tests give stronger
|
||||
evidence that PromptKit can use the schemas. `TestRegistryMetadataMatchesEmbeddedDefinitions`
|
||||
checks an exact count of 11 prompts. That count detects additions while
|
||||
allowing a same-size substitution. Prefer a check for each required prompt ID
|
||||
and its preparability. Retain the tests for canonical schema asset agreement,
|
||||
stable diagnostics data, and protective copy behavior.
|
||||
[Repeated asset validity check](../../../internal/framework/responseschema/registry_test.go#L71)
|
||||
· [Production construction](../../../internal/framework/responseschema/registry.go#L129)
|
||||
· [Prompt metadata count](../../../internal/prompts/registry_test.go#L79)
|
||||
· [Native preparation](../../../internal/prompts/registry_test.go#L13)
|
||||
|
||||
### 11. Remove the tests of the release test's own set helper
|
||||
|
||||
`TestReleaseKitTargetMembership` calls only `sameMembers`, a helper defined
|
||||
in the same test file. It cannot detect a malformed release profile or
|
||||
bootstrap behavior. A standard comparison over the decoded profile can retain
|
||||
the documented target set, product and destination checks, and the credential
|
||||
exclusion without a test of the comparison's implementation. If adding release
|
||||
coverage, give it to the actual bootstrap and its exit handling. The known
|
||||
Git-status failure is a representative offline shell boundary, and
|
||||
ReleaseKit already checks the prepared checkout on its own boundary.
|
||||
[Test of the helper](../../../scripts/release_test.go#L63)
|
||||
· [Real profile assertions](../../../scripts/release_test.go#L10)
|
||||
· [Delivery contract](../../release.md#binary-delivery)
|
||||
· [Bootstrap failure](06-error-handling.md#7-ci-bootstrap-treats-a-failed-git-status-command-as-clean-if-stdout-is-empty)
|
||||
|
||||
### 12. Consolidate normalization fixtures at the domain and parser owners
|
||||
|
||||
`TestNormalizationNoOp` and `TestNormalizationNoMergeDifferentSpeakers`
|
||||
exercise the same rejection with different gap values that remain below the
|
||||
configured maximum. One case can assert the unchanged transcript and the
|
||||
meaningful merge counts together. Seven normalization golden tests also
|
||||
repeat the conversion from `SourceTranscript` to `Transcript` in test code.
|
||||
That conversion mirrors the CLI and cannot catch a regression in the CLI's
|
||||
copy. The fixtures compare decoded values and contain useful domain assertions,
|
||||
including exact text and categories. Keep the distinct domain cases, remove
|
||||
duplicate fixtures after comparing assertions, and let the parser and one
|
||||
assembled CLI case own source shapes and wiring. No automatic golden update
|
||||
was found in these tests.
|
||||
[Overlapping cases](../../../internal/core/normalization/normalize_test.go#L11)
|
||||
· [Different-speaker case](../../../internal/core/normalization/normalize_test.go#L101)
|
||||
· [Copied conversion and semantic comparison](../../../internal/core/normalization/normalize_test.go#L329)
|
||||
· [Production conversion](../../../internal/cli/run.go#L12)
|
||||
|
||||
### 13. Narrow integration assertions to their distinct contracts
|
||||
|
||||
`TestProcessingUsesPerEntryProfilesInOrder` checks the escaped byte spelling
|
||||
of stage IDs in two JSON debug artifacts. A JSON encoder that preserves the
|
||||
same decoded ID with different escaping would fail the test. Decode the
|
||||
artifacts and assert stage, role, and evidence association. Keep end-to-end
|
||||
checks for per-entry profiles, successive transcript state, and output
|
||||
selection, while the reporting package owns Markdown layout and escaping.
|
||||
[Integration artifact assertions](../../../internal/cli/process_ordered_integration_test.go#L145)
|
||||
· [Dedicated report coverage](../../../internal/core/reporting/markdown_test.go#L101)
|
||||
|
||||
`TestSubprocessReportingAndDebugSelections` runs the repair/filter/later-failure
|
||||
scenario in all four flag combinations. `TestSubprocessRecoveredFilteringSurvivesLaterFailure`
|
||||
uses the same model script and also checks the distinct attribution in both
|
||||
YAML documents. The selection matrix took about 6.9 seconds in the
|
||||
instrumented run. Keep one detailed scenario with the attribution assertions
|
||||
and use a smaller fixture for any duplicate combination once it still proves
|
||||
that reporting and debug selections leave decisions and counts intact. The
|
||||
shared fixture also writes a legacy glossary file and stores its path without
|
||||
using it. Remove that unused fixture input separately from behavioral checks.
|
||||
[Selection matrix](../../../cmd/audita/main_integration_test.go#L879)
|
||||
· [Detailed attribution](../../../cmd/audita/main_integration_test.go#L1013)
|
||||
· [Unused fixture input](../../../cmd/audita/main_integration_test.go#L283)
|
||||
|
||||
## Verification scope
|
||||
|
||||
The full race-and-coverage run, 20 repetitions of the selected concurrency
|
||||
tests, and the offline CLI classification probe completed successfully. The
|
||||
audit used source inspection to identify false passes and refactor-sensitive
|
||||
assertions. The full suite uses synthetic model responses, consistent with the
|
||||
offline policy. Live-model editorial quality is a separate, deferred evaluation.
|
||||
[Execution policy](../../policy/testing.md#test-execution-requirements)
|
||||
· [Deferred evaluation](../future.md#live-model-editorial-evaluation)
|
||||
Reference in New Issue
Block a user