Files
audita/docs/internal/diagnostics-reporting.md

88 lines
5.4 KiB
Markdown
Raw Normal View History

# Diagnostics and Reporting Internals
The [output-file contract](../integrations/output-files.md) owns public
schemas and the [operations guide](../operations.md) owns physical placement.
`internal/framework/outcomes` contains processing-owned operation references,
counts, failure reasons, filtering details, and work states. It imports no
reporting, CLI, or provider package. The runner records those facts per initial
section and module instance. `internal/framework/processreport` maps them once
into `internal/core/reporting.Snapshot`, including nullable counts, grouped
work-unit errors, separate run errors, and one recovery per successful logical
operation. Reports order stages, sections, validators, and batches by semantic
identity, never concurrent completion time. The core reporting package owns
Audita's versioned YAML DTOs, fixed Markdown rendering, and brief CLI
completion formatter. Reports encode the typed documents directly to YAML. Ordinary reports omit raw causes,
transcript text, complete decisions, and token/timing payloads.
Validation operations with unavailable decisions report only affected requested
indices, even when other candidates in the batch continue. Ignored extra
validator entries yield `validation_decisions_filtered` only for a successful
operation; they do not count as filtered correction proposals. Failed operations
retain their execution counts and never produce a recovery record.
`internal/core/diagnostics.OutputSession` owns destination selection, report
initialization, and publication. Transcript and report files share a staging
primitive that writes beside the resolved destination. Streams are opened for
delivery. Transcript input is protected from early report writes but
may be the final transcript destination. The LLM runtime records consumed PromptKit sources and effective host message
files and supplies those source paths to CLI;
no second filename-based discovery rule is used.
CLI composes it after configuration, transcript acquisition, and preflight
readiness and before dispatching model work. It does not create ordinary report parents.
Final report rendering uses the delivery and cleanup errors already observed.
If a report write first fails, the remaining writes use the failed bundle and
one pass attempts to update the reports still missing failed evidence. A
published transcript remains available after a later report failure.
Debug is independent. The shared `--debug` flag enables a unique retained child
under `diagnostics.work_dir`. The CLI creates one interaction sink rooted in
that directory and passes it to proposal generation, validation, and repair.
With debug enabled, it saves the source bytes as acquired, separate from the parsed and normalized
transcript snapshots. It also captures invocation metadata, effective settings
rendered by ConfigKit, the correction ledger, LLM execution observations, and
model interactions. Candidate capture includes proposals later
rejected or interrupted. Repair and revalidation artifacts use section, proposal,
attempt, validator, and batch coordinates as applicable. Request metadata records
these coordinates with the interaction role. Proposal request metadata carries
the zero-based response origin (`start_index`) and the ordered `segment_ids`
used to interpret response references. Each interaction retains its available
prepared request, response, and failure evidence. Ledger mapping preserves empty
proposed and applied replacements and distinguishes unavailable candidates from
interrupted work.
The sink groups concurrent write failures by role, artifact operation, and cause.
Warnings report the count and an affected path. The writer still attempts other
artifacts and returns the paths it wrote, and the CLI presents warnings separately
from processing outcomes and required delivery. Debug artifacts can contain sensitive transcript and
prompt content. Execution
position and optional operator ID are carried separately from generated safe
artifact path coordinates.
Execution diagnostics include a native-group invocation summary and per-call
summaries. Terminal coverage is separate from usage completeness; unknown usage
is distinct from known zero. Overlapping summaries must not be added together.
The collector snapshots after work joins and retains domain labels and stable
presentation order. Prepared evidence comes from `OnPrepared`, so preparation or
admission failures may have no request snapshot. Coordinator execution failures
still finalize the attempted logical call and its available evidence.
## Key Tests
Useful tests: `internal/core/reporting/*_test.go`,
`internal/core/diagnostics/*_test.go`,
`internal/framework/processreport/*_test.go`, and
`cmd/audita/main_integration_test.go`.
Report projection uses typed operation and delivery results. Recoverable
operations count toward work-unit errors. A fatal processing cause retains its
own phase and available operation coordinates, and peer interruptions and later
file failures preserve their own attribution. The correction ledger joins one
candidate row by module instance and proposal index, preserving repair lineage
and the recorded validator execution class.
The invocation selects retained evidence once. Summary-only processing
accumulates counters as completed sections leave the work window. Ordinary
reports retain the full work matrix and operation facts. Debug retains
candidate and decision history, repair lineage, and native observations.
Without debug, the runtime skips prepared message snapshots.