4.6 KiB
Diagnostics and Reporting Internals
The output-file contract owns public schemas and the operations guide owns physical placement.
internal/framework/outcomes contains processing-owned operation references,
counts, failure reasons, filtering details, and work states. It imports no
reporting, CLI, or provider package. The runner records those facts per initial
section and module instance. internal/framework/processreport maps them once
into internal/core/reporting.Snapshot, including nullable counts, grouped
work-unit errors, separate run errors, and one recovery per successful logical
operation. Reports order stages, sections, validators, and batches by semantic
identity, never concurrent completion time. The core reporting package owns
Audita's versioned YAML DTOs, fixed Markdown rendering, and brief CLI
completion formatter. Reports encode the typed documents directly to YAML. Ordinary reports omit raw causes,
transcript text, complete decisions, and token/timing payloads.
Validation operations with unavailable decisions report only affected requested
indices, even when other candidates in the batch continue. Ignored extra
validator entries yield validation_decisions_filtered only for a successful
operation; they do not count as filtered correction proposals. Failed operations
retain their execution counts and never produce a recovery record.
internal/core/diagnostics.OutputSession owns destination selection, report
initialization, and publication. Transcript and report files share a staging
primitive that writes beside the resolved destination. Streams are opened for
delivery. Transcript input is protected from early report writes but
may be the final transcript destination. The LLM runtime records consumed PromptKit sources and effective host message
files and supplies those source paths to CLI;
no second filename-based discovery rule is used.
CLI composes it after configuration, transcript acquisition, and preflight
readiness and before dispatching model work. It does not create ordinary report parents.
Final report rendering uses the delivery and cleanup errors already observed.
If a report write first fails, the remaining writes use the failed bundle and
one pass attempts to update the reports still missing failed evidence. A
published transcript remains available after a later report failure.
Debug is independent. The shared --debug flag enables a unique retained child
under diagnostics.work_dir. The CLI creates one interaction sink rooted in
that directory and passes it to proposal generation, validation, and repair.
With debug enabled, it saves the source bytes as acquired, separate from the parsed and normalized
transcript snapshots. It also captures invocation metadata, effective settings
rendered by ConfigKit, the correction ledger, LLM execution observations, and
model interactions. Candidate capture includes proposals later
rejected or interrupted. Repair and revalidation artifacts use section, proposal,
attempt, validator, and batch coordinates as applicable. Request metadata records
these coordinates with the interaction role. Proposal request metadata carries
the zero-based response origin (start_index) and the ordered segment_ids
used to interpret response references. Each interaction retains its available
prepared request, response, and failure evidence. Ledger mapping preserves empty
proposed and applied replacements and distinguishes unavailable candidates from
interrupted work.
The sink groups concurrent write failures by role, artifact operation, and cause. Warnings report the count and an affected path. The writer still attempts other artifacts and returns the paths it wrote, and the CLI presents warnings separately from processing outcomes and required delivery. Debug artifacts can contain sensitive transcript and prompt content. Execution position and optional operator ID are carried separately from generated safe artifact path coordinates.
Execution diagnostics include a native-group invocation summary and per-call
summaries. Terminal coverage is separate from usage completeness; unknown usage
is distinct from known zero. Overlapping summaries must not be added together.
The collector snapshots after work joins and retains domain labels and stable
presentation order. Prepared evidence comes from OnPrepared, so preparation or
admission failures may have no request snapshot. Coordinator execution failures
still finalize the attempted logical call and its available evidence.
Key Tests
Useful tests: internal/core/reporting/*_test.go,
internal/core/diagnostics/*_test.go,
internal/framework/processreport/*_test.go, and
cmd/audita/main_integration_test.go.