Files
audita/docs/roadmap/implementation.md

653 lines
39 KiB
Markdown
Raw Normal View History

# Reliability and Maintainability Implementation Plan
Implement the [target roadmap](reliability-and-maintainability.md) in one
end-to-end sprint, starting from its application baseline `c4ccd78`. The
[assessment](audit/assessment.md) governs the accepted scope and decisions
D1–D7. Numbers under each stage refer to the [audit catalog](audit/catalog.md).
Use the original audits for reproductions and source references. Where an old
remedy differs from a later decision, follow the assessment and target roadmap.
Work through the numbered stages in order. Each stage is sized for one Luna
implementation prompt and contributes directly to the final design. Intermediate
source may be incomplete or fail to build while adjacent changes are in flight.
Use the next stage to finish those connections. A stage needs neither a
standalone commit nor a passing full suite. Replace obsolete paths directly;
compatibility layers and temporary production adapters are unnecessary.
Use the [development guide](../development.md) and applicable adopted policies,
especially [agents](../policy/agents.md), [architecture](../policy/architecture.md),
[testing](../policy/testing.md), and [documentation](../policy/documentation.md).
Add or adapt focused tests at the behavior owner and update canonical docs when
their implemented behavior changes. Run affected tests when the relevant path
builds, then close the sprint with Stage 20. Stage checks establish behavior
without fixing private names, scheduler sizes, event order, or prompt wording.
The target roadmap defines retained contracts, selected changes, and exclusions.
## Stage 1 — Establish the delegated inspection dependency
Audit #24, delegated part.
1. Use a temporary CLI build to reproduce invalid operator configuration with
`promptkit list prompts` and `process`. Reuse the malformed companion fixture
in `internal/cli/runtime_errors_test.go` and include a separate cancellation
case. The baseline reported status 1 for the malformed companion via
inspection and status 2 via processing. Both commands must remain offline.
2. Trace `internal/cli/app_wiring.go` into the pinned `ckprompt` and `ckconfig`
commands. For the known catalog case, `ckprompt.runCatalogList` converts the
loader error to `definitionFailure` even when a typed input cause is
available. Check relevant configuration-scope acquisition paths separately.
Preserve typed causes in Audita's loader. Classify only input failures at
the responsible boundary, leaving cancellation and runtime failures distinct.
3. Check for a released upstream fix for each demonstrated delegated case. If a
suitable release exists, adopt the smallest compatible update using the
[OSS policy](../policy/oss.md) and verify classification at the Audita CLI.
Otherwise record the affected command, reproduction, owner, and needed
behavior for an upstream handoff. Continue the independent stages and use
Stage 20 to integrate and verify a supporting release if available. Keep the
native declarations and routing throughout.
A fix is demonstrated by status 2 and actionable diagnostics for the invalid
input, and status 1 for cancellation. While the supporting release is pending,
record the mismatch explicitly. A test that asserts only nonzero status does
not verify this fix.
**Complete.** The upstream handoff is recorded below. The exit-status fix
remains outstanding for the final verification pass.
Build a temporary executable with `go build -o /tmp/audita-inspection ./cmd/audita`.
In a scratch directory, put `version: 3` in `config.yml`, `profiles: [` in the
adjacent `prompts.yml`, and `[]` in `transcript.json`. End each file with a newline.
| Invocation with `/tmp/audita-inspection` | Status | Diagnostic |
| --- | --- | --- |
| `promptkit list prompts --config config.yml` | 1 | `yaml.invalid` in `prompts.yml` |
| `process transcript.json --config config.yml` | 2 | `yaml.invalid` in `prompts.yml` |
| `config show --scope promptkit --config config.yml` | 1 | `yaml.invalid` in `prompts.yml` |
`config sources`, `config validate`, `config explain --list`, and
`config diff --against-config config.yml` also returned 1 for the `promptkit`
scope with the same diagnostic. All five `app` scope operations succeeded
with the malformed companion and returned 1 with `yaml.invalid` for a
separately selected main file containing `version: [`. These failures occur
during configuration loading, before provider work. The cancellation test
passes an already canceled context through the application entry point and
verifies status 1, `context.Canceled`, and empty stdout for catalog listing,
configuration show, and processing.
Both native adapters receive the typed ConfigKit load cause from Audita's
existing loaders. In `ckprompt`, `runCatalogList` assigns the operational
outcome through `definitionFailure`. In `ckconfig`, the scope handlers
assign it through `configFailure`, with `show` calling it through
`configAcquisitionFailure`. The upstream owners are the respective CommandKit
integrations. They need to assign status 2 for typed operator-input failures
at those boundaries, preserving diagnostics, available inspection evidence,
and the cause. Cancellation and other operational failures need status 1.
The upstream tag list contained releases through `v0.7.0` for each
integration, which are already pinned. Recheck both in the final stage.
## Stage 2 — Remove application credential substitution
Audit #3, #9, #31 (source bytes), #63–65. Use the
[focused removal inventory](audit/14-redaction.md) for affected sites.
1. Switch ordinary reports and debug artifacts to direct typed JSON/YAML
encoding. Keep encoded effective configuration from ConfigKit. In Markdown,
retain escaping and remove substitution. Keep malformed `final_content` as
the captured string. Write the acquired transcript bytes to the debug source
artifact and keep the parsed and normalized views separate.
2. Send domain `repairFindings` directly for initial feedback and subsequent
findings. Preserve the proposal association and typed unavailable assessment.
Present native errors, warnings, and actual paths through CommandKit. This
step removes substitution. Stage 12 exposes the native setup detail that is
currently hidden by generic failure presentation.
3. Remove writer/renderer secret arguments, `DebugInput.Secrets`, discovery
assertions and closures, repair role-list combination, cache and role
tracking, runtime secret discovery, and `internal/core/redaction`. Keep the
prepared and terminal callbacks, source-path tracking, native effective
rendering, and selected provider authentication.
4. Delete masking-only assertions, scanners, fixtures, and stale redaction
descriptions identified in the inventory. Move independent assertions to
writer or caller tests. Check decoded schema and scalar types, literal source
bytes, malformed response evidence, actual paths, and native auth using
synthetic data. Check feedback by decoding a production repair request.
**Complete.** Reports and debug artifacts encode typed data directly, source
snapshots preserve the acquired bytes, and repair requests carry the original
domain findings. Removed application credential discovery and substitution.
The CLI build and full offline suite passed with Go 1.27.3, including checks
for native authentication, malformed response capture, and destination paths.
## Stage 3 — Unify optional interaction capture
Audit #33 and #43.
1. Define one small optional interaction sink at the shared execution port.
Compose the existing filesystem writer from the run directory in CLI wiring.
Pass the sink through proposal, validation, and repair. Each role supplies
its own payload and stage/chunk/operation coordinates, including proposal,
batch, validator, and repair turn where applicable. Keep returned paths and
write errors available for ledger association and warnings.
2. Reuse the write path and artifact envelope in
`internal/framework/llm/diagnostics.go`. Remove private writer adapters and
repeated construction. Keep artifacts self-contained, including revalidation
and a final unusable or undelivered response when available. Keep debug
directory placement, naming safety, and creation modes.
3. Collect optional write warnings safely across workers, grouping equivalent
failures by operation and cause with counts and affected artifact information.
Continue later capture after a write error. Keep the warning path outside
correction decisions and required file delivery.
Verify role, chunk, batch, proposal, and attempt associations with decoded
artifacts. Exercise a failed optional write followed by later capture. Update
the [diagnostics internals](../internal/diagnostics-reporting.md) for the sink.
**Complete.** The CLI supplies one optional sink for proposal, validation,
repair, and revalidation artifacts. The writer records successful paths even
after a partial write and groups failures by role, artifact operation, and
cause. Decoded-artifact and recovery checks passed, along with the full Go
suite and targeted race checks.
## Stage 4 — Bind publication to checked destinations
Audit #1, #6 (cooperative boundaries), #7, #8, #16, #26, #42.
1. In `internal/core/diagnostics/output_session.go` and its CLI callers, resolve
checked transcript and report destinations with filesystem semantics.
Carry those destination and parent choices through staging and publication.
Keep protected-source checks, intentional in-place replacement, ordinary
symlink and stream behavior, and disabled reporting's inactive destination.
Add no directory creation or locking for required destinations.
2. Give the required-output owner a shared regular-file staging primitive for
transcript and reports. Keep optional debug setup separate from that owner.
Keep temporary paths until removal succeeds or absence is established.
Surface cleanup errors alongside the initiating error, including after
partial preparation, publication failure, or cancellation.
3. Make report rendering return an explicit error through initialization and
finalization. Publish the transcript first when processing succeeded.
Select a failed bundle before the first final report write when a required
delivery or cleanup failure is already known. If a report write first
establishes failure, switch the remaining writes to failed status and make
one best-effort pass over reports that still need failed evidence. If the
failed bundle cannot be rendered, keep initialized or published evidence
and return the rendering error.
4. Check context when transcript acquisition returns, at natural preparation
and publication boundaries, and after stream delivery returns. A cancellation
observed after bytes escape still fails the invocation. Join cancellation
and I/O/cleanup errors where available and keep the initiating failure
distinguishable. Use ordinary blocking I/O and retain completed transcript
publication if a later report write fails.
5. Escape input/output paths as literal Markdown text in `core/reporting`,
preserving raw values in structured artifacts. Update
[operations](../operations.md) for caller-owned destination isolation,
retention, and the resulting publication lifecycle.
Exercise symlink/path resolution, in-place and stream output, cancellation
after I/O, cleanup errors, initialized evidence when rendering fails, and
write behavior after known failure. Observe publication attempts as well as
final files. Leave processing fact and ledger projection to Stage 17.
**Complete.** Required outputs use the destinations resolved during selection
and share regular-file staging. Publication checks cancellation, retains and
reports cleanup failures, and uses failed-status reports when required delivery
cannot complete. Tests verified publication attempts, retained evidence when
rendering failed, symlink destinations, stream cancellation, and Markdown path
escaping. The CLI build and full offline Go suite passed.
## Stage 5 — Consolidate typed transcript data
Audit #13, #31, #51, #52, #54, #62 (file helper).
1. In `core/schema`, detect the two accepted root shapes while keeping the
segment array as raw JSON. Decode typed segments directly. Preserve source
ID presence, native integer range, duplicate checks, ignored unknown fields,
and existing format and error behavior. Keep one live source-to-canonical
conversion and retire obsolete parser wrappers after moving meaningful tests.
2. Centralize transcript cloning in the model. Use it at the runner and other
copy boundary, preserving independence and nil versus explicit empty
mutable slices. Share one typed segment projection across both encoders in
`core/schema` and `core/outputschema`. Preserve the envelopes, version,
numeric fidelity, category and omission rules, and encoding error behavior.
3. Release raw input after its final parse or source-snapshot write. Drop the
parsed source after normalization and its debug consumers. Avoid retaining
it via CLI finalization. Remove the audited unused file helper after checking
callers. Keep both file and stream inputs and current output delivery
semantics. Use a representative large fixture to check that the removed
representations no longer overlap needlessly. A fully streaming encoder is
optional if supported by the measurements and existing error semantics.
Test both envelopes, adjacent accepted IDs beyond exact float precision where
supported by native `int`, clone independence with independent expected data,
and the distinction between raw and typed debug snapshots. Update the
[input contract](../integrations/transcript-glossary-files.md) and
[encoder internals](../internal/output-schemas.md) where affected.
**Complete.** Source segments decode directly from both accepted JSON shapes.
Exact IDs, clone independence, encoder values, and debug snapshot fidelity were
verified. Processing releases raw bytes after parsing or source capture and
releases the parsed view before normalization. On a 100,000-segment object
fixture (Go 1.27.3, five iterations), parser allocations fell from 240.6 MB to
84.1 MB per operation. The CLI build and full offline suite passed.
## Stage 6 — Correct and simplify normalization
Audit #17, #21, #35, #53, #59, #62 (merge helper).
1. In `core/normalization`, sort stably by the existing time key and retain
input order for ties. Compute the prospective merge end as the later end
and use that interval for the duration limit, merged segment, and next gap.
Keep overlap acceptance and the existing speaker, gap, and token rules.
2. Use one implementation of the existing token estimate across normalization,
chunking, and its consumers. Maintain the components needed for that exact
estimate as a merge grows. Accumulate text and category membership/order
incrementally with the existing joiners. Avoid rescanning or copying a
growing prefix on each append. Retire the unreachable merge helper.
3. Consolidate overlapping nonmerge tests. Keep independent fixtures for
nested intervals, ties, joiners, category ordering, and exact estimates.
Compare allocations or growth on a representative run of same-speaker merges.
**Complete.** Verified stable start-time ties, nested intervals, exact joiner
and token behavior, and category order. The merge uses incremental text,
categories, and the shared token estimator. On synthetic same-speaker runs
(Go 1.27.3, three iterations each), allocations fell from 238.1 MB to 0.42 MB
per operation for 2,000 segments and from 960.1 MB to 0.85 MB for 4,000. The
CLI build and full offline suite passed.
## Stage 7 — Make feasible explicit section targets succeed
Audit #20 and the infeasibility portion of #24.
1. In `core/chunking/sections.go`, use indivisible, ordered segments as the
partition units. Preserve the empty-transcript behavior and the singleton
exception for segments individually above the token maximum. Keep
`min_section_tokens` validation and compatibility without adding a new
partition constraint.
2. For an explicit target, compute the farthest legal end for each start
with a rolling token sum, honoring the oversized singleton exception. Work
backward to compute each suffix's minimum section count as one plus the
count at its farthest end. In the forward builder, reserve enough segments
for the remaining nonempty sections and use the suffix count to reject cuts
that leave too little capacity. Prefer the current average-token target
among the feasible cuts. Count sections by segment presence, including
zero-token segments. Preserve implicit-target behavior.
3. Return a typed infeasibility error for impossible requested targets. Carry
it through the runner and CLI so the existing input status applies while
internal and cancellation failures retain their classes.
Verify the audited weights `[2,5,1]` with target 2 and maximum 6, impossible
targets, and the oversized singleton case. Check contiguity, order, coverage,
and feasibility independently of the chosen cut. Keep a single chunking pass.
**Complete.** Explicit targets use rolling legal ends and suffix feasibility to
admit exactly the requested count when possible. An independent enumeration
checked 1,024 weight sequences for every possible count, including zero-token
and oversized segments. The real CLI returned status 2 for infeasibility
without dispatching model work. The CLI build and full offline suite passed.
## Stage 8 — Share edit location and protection evidence
Audit #12, #18 (punctuation portion), #19, #38, #47.
1. In `framework/proposals`, search for a second literal starting offset for
require-unique replacements, including overlaps. Use the same applicability
rule in previews and apply. Preserve replace-all's nonoverlapping behavior
and the existing blank-result check.
2. Share the positional segment lookup for initial proposals and repair
revisions. Carry raw invalid model references to evidence, keep segment
positions distinct from candidate and validator response indices, and
revalidate a repair that retargets to another segment.
3. Move protected-name comparison to the glossary owner. Compare usable
full-segment before/after text and repeat the check against the current
segment at apply time. A missing or ambiguous preview cannot support a
protection verdict. An available empty preview may support the verdict
independently of blank-result rejection. Preserve active-glossary
exemptions, case and longest-match rules, and spoken-word repetition cleanup.
Use standard literal-edge checks for punctuation-edged spellings while
retaining Go regexp word-boundary semantics.
4. Reuse the glossary scan for context identity membership without expanding
each occurrence/contributor pair. Keep occurrence evidence for protection
and preserve the order of selected context.
Test overlapping occurrences, full-segment protection, punctuation edges,
ambiguous previews, spoken-word cleanup, and stale apply-time changes at the
owning boundaries. Check the initial and repair lookup against independent
expected references, and compare work on a repeated-glossary fixture.
**Complete.** Proposal preview and application agree on overlapping spans.
Initial and repair responses share positional lookup and retain original model
references in diagnostic evidence. Protection runs on usable full-segment
previews and checks the current text at application. The 2,000-mention
fixture with 128 shared contributors measured 81.0 MB to 0.27 MB allocated
and 77.8 ms to 2.1 ms per operation (Go 1.27.3, ten iterations). The CLI build
and full offline suite passed.
## Stage 9 — Consolidate validator decisions and reconciliation
Audit #10, #11, #27 (runtime class), #40, #49 (validator port).
1. Use one shared owner for the validator interface, applicability, decisions,
and cardinality helpers. Remove duplicate ports and unused validation-only
envelopes after tracing live consumers. Keep concrete validator chains,
built-in prompt/rule ownership, and reusable LLM batching.
2. Resolve a single authoritative disposition before a candidate advances.
Reject contradictory internal results as fatal contract failures, and derive
redundant serialized approval from the validated disposition. Preserve full
normal cardinality and checked partial results on fatal returns. Preserve
zero-budget shortlisting, independent repair feedback, unavailable
assessments, and deterministic validator ordering.
3. Record the effective execution class selected by the runner on both initial
and revalidation decisions, including supported instance overrides. Keep
the existing default for validators without an explicit class. Document
concurrent-call and borrowed-input behavior at the port and allow
synchronized stateful implementations.
4. Preserve exact numeric references through structured-response decoding
using `json.Number` or raw numeric tokens. Reconcile with the requested
batch-local range before converting to local `int`. Recognize equivalent
integral decimal and exponent forms exactly and ignore extras that cannot
name a requested candidate, including values beyond native integer range.
Keep supplied numeric references in diagnostic evidence. Continue whole
response schema validation and structural repair upstream. One usable
decision keeps its outcome. Omission or duplication affects only its
requested candidate, and ignored extras on a successful batch retain their
existing recovery classification.
Test with actual response decoding on a mixed batch and one legitimate decision
beside an out-of-range reference. Cover equivalent numeric spellings,
contradictory internal results, and class override from execution through the
record supplied to Stage 17. Update [validator internals](../internal/validators.md).
**Complete.** Validators share one port and applicability owner. The runner
rejects contradictory decisions and records the effective class on initial and
repair validation evidence. Controlled provider responses verified exact
numeric decoding, equivalent spellings, duplicate and missing decisions, and
a usable decision alongside a reference above the native integer range. The
CLI build and full offline suite passed.
## Stage 10 — Reduce validation context work
Audit #36.
1. In `framework/validators`, consolidate budget and optional-context policy
across the live validation batch builder and trimmer. Measure the required
payload first and stop testing optional material beyond the usable
ordered prefix. Reuse computations where the exact serialized estimate
permits it. Keep context priority, numbering, effective budget, oversized
single-input behavior, and independently measured wire size.
2. Retain a test oracle built from serialized payloads rather than the shared
estimator. Use a representative long optional-context fixture to demonstrate
reduced measurements or allocation. Retain request-shape and available
sibling expectations under over-budget input.
**Complete.** Validation budgets measure required data first and stop at the
first optional entry that does not fit, with glossary entries preceding
neighbors. An independent serialized-payload oracle checked fit and batching.
With 5,000 optional entries and room for one, allocations fell from 3.12 MB to
2.13 KB per operation (Go 1.27.3, twenty iterations). The CLI build and full
offline suite passed.
## Stage 11 — Hand checked selections to execution
Audit #24 (readiness), #37, #49 (module port), #50, #55, #56.
1. Consolidate Audita-owned root and stage validation/fallback helpers in
`core/config` after ConfigKit composition. Preserve presence/null,
explicit zero, inherited values, diagnostic paths and provenance, and
stage/role-dependent defaults. ConfigKit retains its load and override paths.
2. Replace duplicate derivation in `framework/preflight` and
`internal/cli/process_pipeline.go` with one checked processing plan. Store
resolved specs, validated role choices, and effective glossary/protection
views. Cache repeated profile/backend inspections and immutable glossary
views only within the captured runtime. Include ordered selection and
exemptions in reuse keys. Preserve readiness check order, first failure
attribution, protected source paths, no model calls during preflight, and
preflight-before-publication. Return typed status-2 failures for invalid
selections; keep runtime/canceled outcomes distinct.
3. Move built-in construction to composition beside the concrete modules or
in CLI wiring. Keep the injected runner factory and dependency-light key
catalog. Document concurrent calls and borrowed data at the module port.
Copy owned inputs where mutation is required.
4. In processing and inspection request construction, merge shared variables
once per call while preserving each path's precedence, the protected-term
default, and operator replacements for bundled prompts.
Check representative effective selections and provenance, first-failure
attribution, provider-free inspection, constructible catalog entries, and both
request precedence paths. Update [CLI/config internals](../internal/cli-config.md),
[module internals](../internal/modules.md), and configuration docs where affected.
## Stage 12 — Unify structured-call completion and failure facts
Audit #23, #25, #41, #44.
1. In `framework/llm` and `framework/structuredoutput`, share once-only
completion between ordinary execution and coordinated turns. Keep
prepared snapshots, call IDs, terminal synchronization, native group and
call counts, coverage and unknown usage, and diagnostic-only content from
undelivered responses. Remove obsolete response/usage copies and callback
payloads after checking report and debug consumers.
2. Carry typed class, reason and cause from adapter through operation records.
Use the internal-failure reason when a reason is absent, retaining the
original class and cause. Apply parent cancellation precedence, identify
native logical-operation deadline expiry, then native transport/attempt
timeout before treating a generic deadline as logical timeout. Use the
existing transport-failure reason for HTTP-only timeouts.
3. Make setup diagnostics actionable at the CLI. Surface the native missing
credential requirement and available role/profile context with the typed
cause intact. Keep ordinary reports on their compact typed projections.
Exercise distinct parent, operation, and attempt timeouts with independently
controlled budgets. Check setup class/cause and useful native information,
plus ordinary/coordinator terminal and undelivered-evidence paths. Update
[LLM runtime internals](../internal/llm-runtime.md).
## Stage 13 — Use one proposal handoff
Audit #46.
1. In `framework/proposal_generation` and `framework/runner`, return one
set of decoded proposal results with raw local references, filtering
evidence, and optional native exchange. Remove the generator's enriched
projection discarded by the live wrapper. Keep runner ownership of
sequential proposal indices within a stage, ordered by chunk and response.
2. Check emitted interaction metadata such as `start_index` separately from
private references. Preserve its meaningful serialized association and
retire fields confirmed to have no surviving purpose.
3. Move proposal-order assertions from the test-only composition harness to
real runner results. Observe out-of-order completion in the fixture and
verify indices and original references. Keep model response and reference
coverage in the generator's owning tests.
## Stage 14 — Own each correction repair branch explicitly
Audit #34, #45, #48, #64 (branch integration).
1. In `framework/runner/repair.go`, give one proposal branch a clear
execution input and result. Make repair capability explicit at setup. Keep
branch state and terminal outcome together. Merge outcomes into the section
after each branch, without mutating unrelated approvals or decisions.
2. Run repairs through PromptKit coordination with full exchange/history and
configured per-correction limits. Preserve initial and subsequent direct
domain feedback, sibling independence, withdrawal, unavailable assessment,
retargeting, and revalidation of the final allowed revision. Move useful
structural repair and isolation tests onto this path before deleting the
direct-continuation-only test branch.
3. Use controlled observations or temporary instrumentation to measure
cumulative turns, available token/byte costs, revalidation calls, and debug
work on a representative multi-correction fixture. Remove unnecessary local
copies. Keep those measurements out of the ordinary report contract.
Check independent approved siblings, attempts/turn counts, actual branch
history and feedback, final-turn revalidation, and associated artifacts in
production coordination.
## Stage 15 — Give stage execution one commit boundary
Audit #14, #15 (evidence), #22 and #41 (failure facts).
1. In `framework/runner`, check context at entry, including empty pipelines,
and before committing a stage. Use one stage completion path to record
outcomes and evidence. Only a successful active stage advances the working
transcript. Preserve previously committed text for partial results.
2. Carry initiating cause, phase and available stage/chunk/operation reference
from the owner. Cancel and join peers on fatal failure or parent cancellation;
keep peer interruption and secondary errors attributable without promoting
them over the initiating failure. Continue recoverable work under existing
partial-success semantics.
3. Collect completed proposal evidence independently of dispatch to validators.
After workers join, include usable out-of-order completions in interrupted
diagnostics with deterministic indices and associations. Keep raw evidence
from undelivered responses separate from usable proposals. Retain the ordered
indexing/validation boundary until Stage 16 evaluates it.
Test canceled entry and precommit, a later completed proposal behind a slow
earlier chunk, and a fatal stage after earlier committed text. Assert causal
origin and preserved evidence as well as returned failure. Leave report/ledger
projection to Stage 17.
## Stage 16 — Bound local preparation and actively drain results
Audit #15 (dispatch) and #30, under D3.
1. Replace the per-chunk goroutines and chunk-count-sized pending channels in
`collectSectionProposals` and `runModulePipeline` with a private rolling
stage window. Admit chunks to the window in input order. Charge a slot for
each chunk still in preparation, model work, or awaiting downstream work.
Release it when the chunk is finished and its final facts have been drained.
Start with a small fixed local window and keep it private. Drain outcomes
while workers run and on cancellation, then join and finish. Keep compact
or requested completed evidence apart from remaining work. Collect results
while the workers can still send them.
2. Measure staged overlap and peak pending work with many sections, a
controlled slow first proposal, and shared PromptKit capacity. Where active
draining and ordered indexing yield adequate progress, keep that simpler
boundary. If ordered indexing still leaves repeatable idle capacity with
validation-ready chunks, use a chunk index and response position to identify
candidates during work. At ordered collection, assign the existing stage
indices and translate every projected proposal reference, including failure
indices, repair lineage, and interaction-to-ledger mappings. Keep the
identity-sensitive artifact data in the charged window until the index is
assigned; use the Stage 3 writer for the final artifacts. Keep batch-local
model references and raw response evidence intact. Check the same mappings
after a cancellation joins the workers.
3. Keep the module barrier and deterministic application. PromptKit retains
admission, queueing, limits, retries and recoverable rejection. Include
nested validation/repair in the local resource and cancellation checks
without duplicating backend admission or holding its permits across a
nested call. Tune the window using the measured workload and contention;
no public window, throughput minimum, or eventual-admission guarantee follows.
Test actual observed overlap, finite native contention, cancellation with work
pending, and identical index ordering for the same usable results under
different completion orders. Report representative pending-work measurements
without pinning a timing or worker-count threshold in the default suite.
## Stage 17 — Project typed report and ledger facts
Audit #22, #27 (ledger), #39, #41 (projection).
1. In `framework/processreport`, build the snapshot from the Stage 12 and 15
facts and Stage 4 delivery outcomes. Preserve phase and available origin for
setup, processing and publication errors. Keep recovery and work-unit counts
and known versus unknown values. Distinguish the primary run error,
interrupted peers, and secondary output failures without counting a fatal
run failure again as a recoverable unit failure.
2. Join proposals, validator decisions, repairs, application and skip evidence
once by stable identities in `correction_ledger.go`. Use the recorded
execution class from Stage 9. Project one candidate row with correct
disposition, lineage and literal empty replacement values. Retain
module-only failures. Preserve serialized fields and logical ordering.
3. In runner application attribution, reuse section/segment lookups and
aggregate skipped evidence once. Preserve the distinction between stale
apply-time conflicts and response filtering. Keep exact correction,
filtered-proposal and validation-decision identities across projection.
Verify decoded YAML, Markdown counts, and ledger facts against an independently
specified mixed run, including partial decisions, repair/filter interaction,
later failure, and an overridden validator class. Update the
[output contract](../integrations/output-files.md) and diagnostics internals.
## Stage 18 — Retain evidence according to selected outputs
Audit #32 and supporting work for #30, #33, and #44.
1. Derive evidence needs once from enabled ordinary reports and effective debug
capture. Keep compact CLI totals and available failure facts in summary-only
mode. Allocate work-unit detail only when a consumer needs it, and avoid
eager whole-run grids and debug-only candidate/decision histories in
summary-only paths. Keep complete enabled report matrices and debug evidence.
2. Discard active validation, exchange and repair state after their last
consumer, including data copied solely for obsolete runner projections.
Remove duplicate operation lists and unused module totals after checking
their projections. Keep native observations and requested self-contained
artifacts when capture is selected.
3. Compare live/retained state for representative long runs with summary only,
ordinary reports, debug, and their existing combinations. Check shared
totals and logical results across modes, with detail present where selected.
Resource measurements support reduction of waste without setting a
constant-memory or disk-use requirement.
## Stage 19 — Emit native processing progress and update operator guidance
Audit #5, #57, and the operator-facing parts of #8, #24 and #25.
1. Use a small event callback from the live lifecycle owners. Supply stage and
chunk activity, proposal/validation/repair phases and useful completions
with available identities and observed counts. Reuse suitable PromptKit
events where they convey useful activity. Keep event delivery within the
invocation and finish workers before the reporter closes.
2. Wire that callback in CLI to `Invocation.Reporter().Progress` in pinned
CommandKit. Let CommandKit determine visibility, routing, and presentation
failure status, independently of transcript/report processing outcomes.
Preserve stdout transcript data. Respect ordinary mode, explicit progress,
debug defaults, explicit `--progress=false`, and quiet suppression.
3. Update the [CLI reference](../cli.md), [operations](../operations.md) and
[subprocess contract](../integrations/subprocess.md) at their owned
boundaries. Cover meaningful progress, caller-owned report destinations and
artifact retention, and the trusted configuration bundle, including
discovery, imports and automatic companions. Keep the distinction between
native configuration inspection and processing readiness.
Use representative command tests for live activity and combined modes,
asserting useful identities and stream separation without exact wording or
concurrent ordering. Keep a progress-specific check separate from delegated
inspection availability.
## Stage 20 — Verify the integrated sprint and release bootstrap
Audit #28, #29, #58, #60–62 and sprint closure.
1. In `.woodpecker/release.yml`, capture `git status --porcelain --untracked-files=all`
in an assignment that checks the command's exit status separately from its
content. Then require empty output. Preserve ReleaseKit's independent
prepared-checkout validation and tag delivery. In `scripts/release_test.go`,
cover clean, dirty and failed-status-with-empty-output behavior at the
command boundary. Remove redundant tests of the comparison helper while
keeping real release target and wiring checks.
2. Verify signal, cancellation and exit behavior through the real executable
or a shared production entry point. Keep narrower checks where a process
adds no evidence. Verify effective validator roles, required prompt
identities, embedded asset preparability, schema identity, and protective
copies through use. Retain a representative repair/filter/later-failure
integration scenario and independent output-choice cases. Remove the
audited abandoned fixture inputs and helpers after checking callers.
3. Recheck Stage 1. Adopt and verify a released upstream fix that became
available using the same scope. Otherwise report the exact outstanding
upstream-dependent behavior and its disposition. Treat that item as
outstanding, while verifying the independent Audita work. Keep the
invalid-input and cancellation cases distinct.
4. Account for the [catalog](audit/catalog.md) against the landed changes and
stage evidence. Entries #2 and #4 need no implementation, and only the
accepted portions of #6 and #18 were selected. Check canonical docs,
examples, and affected links and remove stale implemented-behavior claims.
Keep historical audit observations intact.
5. From the integrated tree, build the CLI and run the full offline
`go test ./...` with the supported Go toolchain. Run targeted race checks
for changed shared-state paths and review the representative measurements
from Stages 6, 8, 10, 14, 16 and 18. Verify formatting and whitespace,
and inspect the final diff. Reuse successful checks for unchanged content.
Existing [release validation](../policy/release.md#candidate-validation)
applies when a release is prepared.
The completed sprint should leave the target architecture reachable through
`audita process` and the standard delegated commands. Report checks and
measurements actually performed, including any outstanding external dependency.