73 lines
4.5 KiB
Markdown
73 lines
4.5 KiB
Markdown
# Diagnostics and Reporting Internals
|
|
|
|
The [output-file contract](../integrations/output-files.md) owns public
|
|
schemas and the [operations guide](../operations.md) owns physical placement.
|
|
|
|
`internal/framework/outcomes` contains processing-owned operation references,
|
|
counts, failure reasons, filtering details, and work states. It imports no
|
|
reporting, CLI, or provider package. The runner records those facts per initial
|
|
section and module instance. `internal/framework/processreport` maps them once
|
|
into `internal/core/reporting.Snapshot`, including nullable counts, grouped
|
|
work-unit errors, separate run errors, and one recovery per successful logical
|
|
operation. Reports order stages, sections, validators, and batches by semantic
|
|
identity, never concurrent completion time. The core reporting package owns
|
|
Audita's versioned YAML DTOs, fixed Markdown rendering, and brief CLI
|
|
completion formatter. Reports encode the typed documents directly to YAML. Ordinary reports omit raw causes,
|
|
transcript text, complete decisions, and token/timing payloads.
|
|
Validation operations with unavailable decisions report only affected requested
|
|
indices, even when other candidates in the batch continue. Ignored extra
|
|
validator entries yield `validation_decisions_filtered` only for a successful
|
|
operation; they do not count as filtered correction proposals. Failed operations
|
|
retain their execution counts and never produce a recovery record.
|
|
|
|
`internal/core/diagnostics.OutputSession` owns destination selection, report
|
|
initialization, and publication. Transcript and report files share a staging
|
|
primitive that writes beside the resolved destination. Streams are opened for
|
|
delivery. Transcript input is protected from early report writes but
|
|
may be the final transcript destination. The LLM runtime records consumed PromptKit sources and effective host message
|
|
files and supplies those source paths to CLI;
|
|
no second filename-based discovery rule is used.
|
|
CLI composes it after configuration, transcript acquisition, and preflight
|
|
readiness and before dispatching model work. It does not create ordinary report parents.
|
|
Final report rendering uses the delivery and cleanup errors already observed.
|
|
If a report write first fails, the remaining writes use the failed bundle and
|
|
one pass attempts to update the reports still missing failed evidence. A
|
|
published transcript remains available after a later report failure.
|
|
|
|
Debug is independent. The shared `--debug` flag enables a unique retained child
|
|
under `diagnostics.work_dir`. The CLI creates one interaction sink rooted in
|
|
that directory and passes it to proposal generation, validation, and repair.
|
|
With debug enabled, it saves the source bytes as acquired, separate from the parsed and normalized
|
|
transcript snapshots. It also captures invocation metadata, effective settings
|
|
rendered by ConfigKit, the correction ledger, LLM execution observations, and
|
|
model interactions. Candidate capture includes proposals later
|
|
rejected or interrupted. Repair and revalidation artifacts use section, proposal,
|
|
attempt, validator, and batch coordinates as applicable. Request metadata records
|
|
these coordinates with the interaction role. Each interaction retains its available
|
|
prepared request, response, and failure evidence. Ledger mapping preserves empty
|
|
proposed and applied replacements and distinguishes unavailable candidates from
|
|
interrupted work.
|
|
|
|
The sink groups concurrent write failures by role, artifact operation, and cause.
|
|
Warnings report the count and an affected path. The writer still attempts other
|
|
artifacts and returns the paths it wrote, and the CLI presents warnings separately
|
|
from processing outcomes and required delivery. Debug artifacts can contain sensitive transcript and
|
|
prompt content. Execution
|
|
position and optional operator ID are carried separately from generated safe
|
|
artifact path coordinates.
|
|
|
|
Execution diagnostics include a native-group invocation summary and per-call
|
|
summaries. Terminal coverage is separate from usage completeness; unknown usage
|
|
is distinct from known zero. Overlapping summaries must not be added together.
|
|
The collector snapshots after work joins and retains domain labels and stable
|
|
presentation order. Prepared evidence comes from `OnPrepared`, so preparation or
|
|
admission failures may have no request snapshot. Coordinator execution failures
|
|
still finalize the attempted logical call and its available evidence.
|
|
|
|
## Key Tests
|
|
|
|
Useful tests: `internal/core/reporting/*_test.go`,
|
|
`internal/core/diagnostics/*_test.go`,
|
|
`internal/framework/processreport/*_test.go`, and
|
|
`cmd/audita/main_integration_test.go`.
|