558 lines
33 KiB
Markdown
558 lines
33 KiB
Markdown
|
|
# Reliability and Maintainability Implementation Plan
|
|||
|
|
|
|||
|
|
Implement the [target roadmap](reliability-and-maintainability.md) in one
|
|||
|
|
end-to-end sprint, starting from its application baseline `c4ccd78`. The
|
|||
|
|
[assessment](audit/assessment.md) governs the accepted scope and decisions
|
|||
|
|
D1–D7. Numbers under each stage refer to the [audit catalog](audit/catalog.md).
|
|||
|
|
Use the original audits for reproductions and source references. Where an old
|
|||
|
|
remedy differs from a later decision, follow the assessment and target roadmap.
|
|||
|
|
|
|||
|
|
Work through the numbered stages in order. Each stage is sized for one Luna
|
|||
|
|
implementation prompt and contributes directly to the final design. Intermediate
|
|||
|
|
source may be incomplete or fail to build while adjacent changes are in flight.
|
|||
|
|
Use the next stage to finish those connections. A stage needs neither a
|
|||
|
|
standalone commit nor a passing full suite. Replace obsolete paths directly;
|
|||
|
|
compatibility layers and temporary production adapters are unnecessary.
|
|||
|
|
|
|||
|
|
Use the [development guide](../development.md) and applicable adopted policies,
|
|||
|
|
especially [agents](../policy/agents.md), [architecture](../policy/architecture.md),
|
|||
|
|
[testing](../policy/testing.md), and [documentation](../policy/documentation.md).
|
|||
|
|
Add or adapt focused tests at the behavior owner and update canonical docs when
|
|||
|
|
their implemented behavior changes. Run affected tests when the relevant path
|
|||
|
|
builds, then close the sprint with Stage 20. Stage checks establish behavior
|
|||
|
|
without fixing private names, scheduler sizes, event order, or prompt wording.
|
|||
|
|
The target roadmap defines retained contracts, selected changes, and exclusions.
|
|||
|
|
|
|||
|
|
## Stage 1 — Establish the delegated inspection dependency
|
|||
|
|
|
|||
|
|
Audit #24, delegated part.
|
|||
|
|
|
|||
|
|
1. Use a temporary CLI build to reproduce invalid operator configuration with
|
|||
|
|
`promptkit list prompts` and `process`. Reuse the malformed companion fixture
|
|||
|
|
in `internal/cli/runtime_errors_test.go` and include a separate cancellation
|
|||
|
|
case. The baseline reported status 1 for the malformed companion via
|
|||
|
|
inspection and status 2 via processing. Both commands must remain offline.
|
|||
|
|
2. Trace `internal/cli/app_wiring.go` into the pinned `ckprompt` and `ckconfig`
|
|||
|
|
commands. For the known catalog case, `ckprompt.runCatalogList` converts the
|
|||
|
|
loader error to `definitionFailure` even when a typed input cause is
|
|||
|
|
available. Check relevant configuration-scope acquisition paths separately.
|
|||
|
|
Preserve typed causes in Audita's loader. Classify only input failures at
|
|||
|
|
the responsible boundary, leaving cancellation and runtime failures distinct.
|
|||
|
|
3. Check for a released upstream fix for each demonstrated delegated case. If a
|
|||
|
|
suitable release exists, adopt the smallest compatible update using the
|
|||
|
|
[OSS policy](../policy/oss.md) and verify classification at the Audita CLI.
|
|||
|
|
Otherwise record the affected command, reproduction, owner, and needed
|
|||
|
|
behavior for an upstream handoff. Continue the independent stages and use
|
|||
|
|
Stage 20 to integrate and verify a supporting release if available. Keep the
|
|||
|
|
native declarations and routing throughout.
|
|||
|
|
|
|||
|
|
A fix is demonstrated by status 2 and actionable diagnostics for the invalid
|
|||
|
|
input, and status 1 for cancellation. While the supporting release is pending,
|
|||
|
|
record the mismatch explicitly. A test that asserts only nonzero status does
|
|||
|
|
not verify this fix.
|
|||
|
|
|
|||
|
|
## Stage 2 — Remove application credential substitution
|
|||
|
|
|
|||
|
|
Audit #3, #9, #31 (source bytes), #63–65. Use the
|
|||
|
|
[focused removal inventory](audit/14-redaction.md) for affected sites.
|
|||
|
|
|
|||
|
|
1. Switch ordinary reports and debug artifacts to direct typed JSON/YAML
|
|||
|
|
encoding. Keep encoded effective configuration from ConfigKit. In Markdown,
|
|||
|
|
retain escaping and remove substitution. Keep malformed `final_content` as
|
|||
|
|
the captured string. Write the acquired transcript bytes to the debug source
|
|||
|
|
artifact and keep the parsed and normalized views separate.
|
|||
|
|
2. Send domain `repairFindings` directly for initial feedback and subsequent
|
|||
|
|
findings. Preserve the proposal association and typed unavailable assessment.
|
|||
|
|
Present native errors, warnings, and actual paths through CommandKit. This
|
|||
|
|
step removes substitution. Stage 12 exposes the native setup detail that is
|
|||
|
|
currently hidden by generic failure presentation.
|
|||
|
|
3. Remove writer/renderer secret arguments, `DebugInput.Secrets`, discovery
|
|||
|
|
assertions and closures, repair role-list combination, cache and role
|
|||
|
|
tracking, runtime secret discovery, and `internal/core/redaction`. Keep the
|
|||
|
|
prepared and terminal callbacks, source-path tracking, native effective
|
|||
|
|
rendering, and selected provider authentication.
|
|||
|
|
4. Delete masking-only assertions, scanners, fixtures, and stale redaction
|
|||
|
|
descriptions identified in the inventory. Move independent assertions to
|
|||
|
|
writer or caller tests. Check decoded schema and scalar types, literal source
|
|||
|
|
bytes, malformed response evidence, actual paths, and native auth using
|
|||
|
|
synthetic data. Check feedback by decoding a production repair request.
|
|||
|
|
|
|||
|
|
## Stage 3 — Unify optional interaction capture
|
|||
|
|
|
|||
|
|
Audit #33 and #43.
|
|||
|
|
|
|||
|
|
1. Define one small optional interaction sink at the shared execution port.
|
|||
|
|
Compose the existing filesystem writer from the run directory in CLI wiring.
|
|||
|
|
Pass the sink through proposal, validation, and repair. Each role supplies
|
|||
|
|
its own payload and stage/chunk/operation coordinates, including proposal,
|
|||
|
|
batch, validator, and repair turn where applicable. Keep returned paths and
|
|||
|
|
write errors available for ledger association and warnings.
|
|||
|
|
2. Reuse the write path and artifact envelope in
|
|||
|
|
`internal/framework/llm/diagnostics.go`. Remove private writer adapters and
|
|||
|
|
repeated construction. Keep artifacts self-contained, including revalidation
|
|||
|
|
and a final unusable or undelivered response when available. Keep debug
|
|||
|
|
directory placement, naming safety, and creation modes.
|
|||
|
|
3. Collect optional write warnings safely across workers, grouping equivalent
|
|||
|
|
failures by operation and cause with counts and affected artifact information.
|
|||
|
|
Continue later capture after a write error. Keep the warning path outside
|
|||
|
|
correction decisions and required file delivery.
|
|||
|
|
|
|||
|
|
Verify role, chunk, batch, proposal, and attempt associations with decoded
|
|||
|
|
artifacts. Exercise a failed optional write followed by later capture. Update
|
|||
|
|
the [diagnostics internals](../internal/diagnostics-reporting.md) for the sink.
|
|||
|
|
|
|||
|
|
## Stage 4 — Bind publication to checked destinations
|
|||
|
|
|
|||
|
|
Audit #1, #6 (cooperative boundaries), #7, #8, #16, #26, #42.
|
|||
|
|
|
|||
|
|
1. In `internal/core/diagnostics/output_session.go` and its CLI callers, resolve
|
|||
|
|
checked transcript and report destinations with filesystem semantics.
|
|||
|
|
Carry those destination and parent choices through staging and publication.
|
|||
|
|
Keep protected-source checks, intentional in-place replacement, ordinary
|
|||
|
|
symlink and stream behavior, and disabled reporting's inactive destination.
|
|||
|
|
Add no directory creation or locking for required destinations.
|
|||
|
|
2. Give the required-output owner a shared regular-file staging primitive for
|
|||
|
|
transcript and reports. Keep optional debug setup separate from that owner.
|
|||
|
|
Keep temporary paths until removal succeeds or absence is established.
|
|||
|
|
Surface cleanup errors alongside the initiating error, including after
|
|||
|
|
partial preparation, publication failure, or cancellation.
|
|||
|
|
3. Make report rendering return an explicit error through initialization and
|
|||
|
|
finalization. Publish the transcript first when processing succeeded.
|
|||
|
|
Select a failed bundle before the first final report write when a required
|
|||
|
|
delivery or cleanup failure is already known. If a report write first
|
|||
|
|
establishes failure, switch the remaining writes to failed status and make
|
|||
|
|
one best-effort pass over reports that still need failed evidence. If the
|
|||
|
|
failed bundle cannot be rendered, keep initialized or published evidence
|
|||
|
|
and return the rendering error.
|
|||
|
|
4. Check context when transcript acquisition returns, at natural preparation
|
|||
|
|
and publication boundaries, and after stream delivery returns. A cancellation
|
|||
|
|
observed after bytes escape still fails the invocation. Join cancellation
|
|||
|
|
and I/O/cleanup errors where available and keep the initiating failure
|
|||
|
|
distinguishable. Use ordinary blocking I/O and retain completed transcript
|
|||
|
|
publication if a later report write fails.
|
|||
|
|
5. Escape input/output paths as literal Markdown text in `core/reporting`,
|
|||
|
|
preserving raw values in structured artifacts. Update
|
|||
|
|
[operations](../operations.md) for caller-owned destination isolation,
|
|||
|
|
retention, and the resulting publication lifecycle.
|
|||
|
|
|
|||
|
|
Exercise symlink/path resolution, in-place and stream output, cancellation
|
|||
|
|
after I/O, cleanup errors, initialized evidence when rendering fails, and
|
|||
|
|
write behavior after known failure. Observe publication attempts as well as
|
|||
|
|
final files. Leave processing fact and ledger projection to Stage 17.
|
|||
|
|
|
|||
|
|
## Stage 5 — Consolidate typed transcript data
|
|||
|
|
|
|||
|
|
Audit #13, #31, #51, #52, #54, #62 (file helper).
|
|||
|
|
|
|||
|
|
1. In `core/schema`, detect the two accepted root shapes while keeping the
|
|||
|
|
segment array as raw JSON. Decode typed segments directly. Preserve source
|
|||
|
|
ID presence, native integer range, duplicate checks, ignored unknown fields,
|
|||
|
|
and existing format and error behavior. Keep one live source-to-canonical
|
|||
|
|
conversion and retire obsolete parser wrappers after moving meaningful tests.
|
|||
|
|
2. Centralize transcript cloning in the model. Use it at the runner and other
|
|||
|
|
copy boundary, preserving independence and nil versus explicit empty
|
|||
|
|
mutable slices. Share one typed segment projection across both encoders in
|
|||
|
|
`core/schema` and `core/outputschema`. Preserve the envelopes, version,
|
|||
|
|
numeric fidelity, category and omission rules, and encoding error behavior.
|
|||
|
|
3. Release raw input after its final parse or source-snapshot write. Drop the
|
|||
|
|
parsed source after normalization and its debug consumers. Avoid retaining
|
|||
|
|
it via CLI finalization. Remove the audited unused file helper after checking
|
|||
|
|
callers. Keep both file and stream inputs and current output delivery
|
|||
|
|
semantics. Use a representative large fixture to check that the removed
|
|||
|
|
representations no longer overlap needlessly. A fully streaming encoder is
|
|||
|
|
optional if supported by the measurements and existing error semantics.
|
|||
|
|
|
|||
|
|
Test both envelopes, adjacent accepted IDs beyond exact float precision where
|
|||
|
|
supported by native `int`, clone independence with independent expected data,
|
|||
|
|
and the distinction between raw and typed debug snapshots. Update the
|
|||
|
|
[input contract](../integrations/transcript-glossary-files.md) and
|
|||
|
|
[encoder internals](../internal/output-schemas.md) where affected.
|
|||
|
|
|
|||
|
|
## Stage 6 — Correct and simplify normalization
|
|||
|
|
|
|||
|
|
Audit #17, #21, #35, #53, #59, #62 (merge helper).
|
|||
|
|
|
|||
|
|
1. In `core/normalization`, sort stably by the existing time key and retain
|
|||
|
|
input order for ties. Compute the prospective merge end as the later end
|
|||
|
|
and use that interval for the duration limit, merged segment, and next gap.
|
|||
|
|
Keep overlap acceptance and the existing speaker, gap, and token rules.
|
|||
|
|
2. Use one implementation of the existing token estimate across normalization,
|
|||
|
|
chunking, and its consumers. Maintain the components needed for that exact
|
|||
|
|
estimate as a merge grows. Accumulate text and category membership/order
|
|||
|
|
incrementally with the existing joiners. Avoid rescanning or copying a
|
|||
|
|
growing prefix on each append. Retire the unreachable merge helper.
|
|||
|
|
3. Consolidate overlapping nonmerge tests. Keep independent fixtures for
|
|||
|
|
nested intervals, ties, joiners, category ordering, and exact estimates.
|
|||
|
|
Compare allocations or growth on a representative run of same-speaker merges.
|
|||
|
|
|
|||
|
|
## Stage 7 — Make feasible explicit section targets succeed
|
|||
|
|
|
|||
|
|
Audit #20 and the infeasibility portion of #24.
|
|||
|
|
|
|||
|
|
1. In `core/chunking/sections.go`, use indivisible, ordered segments as the
|
|||
|
|
partition units. Preserve the empty-transcript behavior and the singleton
|
|||
|
|
exception for segments individually above the token maximum. Keep
|
|||
|
|
`min_section_tokens` validation and compatibility without adding a new
|
|||
|
|
partition constraint.
|
|||
|
|
2. For an explicit target, compute the farthest legal end for each start
|
|||
|
|
with a rolling token sum, honoring the oversized singleton exception. Work
|
|||
|
|
backward to compute each suffix's minimum section count as one plus the
|
|||
|
|
count at its farthest end. In the forward builder, reserve enough segments
|
|||
|
|
for the remaining nonempty sections and use the suffix count to reject cuts
|
|||
|
|
that leave too little capacity. Prefer the current average-token target
|
|||
|
|
among the feasible cuts. Count sections by segment presence, including
|
|||
|
|
zero-token segments. Preserve implicit-target behavior.
|
|||
|
|
3. Return a typed infeasibility error for impossible requested targets. Carry
|
|||
|
|
it through the runner and CLI so the existing input status applies while
|
|||
|
|
internal and cancellation failures retain their classes.
|
|||
|
|
|
|||
|
|
Verify the audited weights `[2,5,1]` with target 2 and maximum 6, impossible
|
|||
|
|
targets, and the oversized singleton case. Check contiguity, order, coverage,
|
|||
|
|
and feasibility independently of the chosen cut. Keep a single chunking pass.
|
|||
|
|
|
|||
|
|
## Stage 8 — Share edit location and protection evidence
|
|||
|
|
|
|||
|
|
Audit #12, #18 (punctuation portion), #19, #38, #47.
|
|||
|
|
|
|||
|
|
1. In `framework/proposals`, search for a second literal starting offset for
|
|||
|
|
require-unique replacements, including overlaps. Use the same applicability
|
|||
|
|
rule in previews and apply. Preserve replace-all's nonoverlapping behavior
|
|||
|
|
and the existing blank-result check.
|
|||
|
|
2. Share the positional segment lookup for initial proposals and repair
|
|||
|
|
revisions. Carry raw invalid model references to evidence, keep segment
|
|||
|
|
positions distinct from candidate and validator response indices, and
|
|||
|
|
revalidate a repair that retargets to another segment.
|
|||
|
|
3. Move protected-name comparison to the glossary owner. Compare usable
|
|||
|
|
full-segment before/after text and repeat the check against the current
|
|||
|
|
segment at apply time. A missing or ambiguous preview cannot support a
|
|||
|
|
protection verdict. An available empty preview may support the verdict
|
|||
|
|
independently of blank-result rejection. Preserve active-glossary
|
|||
|
|
exemptions, case and longest-match rules, and spoken-word repetition cleanup.
|
|||
|
|
Use standard literal-edge checks for punctuation-edged spellings while
|
|||
|
|
retaining Go regexp word-boundary semantics.
|
|||
|
|
4. Reuse the glossary scan for context identity membership without expanding
|
|||
|
|
each occurrence/contributor pair. Keep occurrence evidence for protection
|
|||
|
|
and preserve the order of selected context.
|
|||
|
|
|
|||
|
|
Test overlapping occurrences, full-segment protection, punctuation edges,
|
|||
|
|
ambiguous previews, spoken-word cleanup, and stale apply-time changes at the
|
|||
|
|
owning boundaries. Check the initial and repair lookup against independent
|
|||
|
|
expected references, and compare work on a repeated-glossary fixture.
|
|||
|
|
|
|||
|
|
## Stage 9 — Consolidate validator decisions and reconciliation
|
|||
|
|
|
|||
|
|
Audit #10, #11, #27 (runtime class), #40, #49 (validator port).
|
|||
|
|
|
|||
|
|
1. Use one shared owner for the validator interface, applicability, decisions,
|
|||
|
|
and cardinality helpers. Remove duplicate ports and unused validation-only
|
|||
|
|
envelopes after tracing live consumers. Keep concrete validator chains,
|
|||
|
|
built-in prompt/rule ownership, and reusable LLM batching.
|
|||
|
|
2. Resolve a single authoritative disposition before a candidate advances.
|
|||
|
|
Reject contradictory internal results as fatal contract failures, and derive
|
|||
|
|
redundant serialized approval from the validated disposition. Preserve full
|
|||
|
|
normal cardinality and checked partial results on fatal returns. Preserve
|
|||
|
|
zero-budget shortlisting, independent repair feedback, unavailable
|
|||
|
|
assessments, and deterministic validator ordering.
|
|||
|
|
3. Record the effective execution class selected by the runner on both initial
|
|||
|
|
and revalidation decisions, including supported instance overrides. Keep
|
|||
|
|
the existing default for validators without an explicit class. Document
|
|||
|
|
concurrent-call and borrowed-input behavior at the port and allow
|
|||
|
|
synchronized stateful implementations.
|
|||
|
|
4. Preserve exact numeric references through structured-response decoding
|
|||
|
|
using `json.Number` or raw numeric tokens. Reconcile with the requested
|
|||
|
|
batch-local range before converting to local `int`. Recognize equivalent
|
|||
|
|
integral decimal and exponent forms exactly and ignore extras that cannot
|
|||
|
|
name a requested candidate, including values beyond native integer range.
|
|||
|
|
Keep supplied numeric references in diagnostic evidence. Continue whole
|
|||
|
|
response schema validation and structural repair upstream. One usable
|
|||
|
|
decision keeps its outcome. Omission or duplication affects only its
|
|||
|
|
requested candidate, and ignored extras on a successful batch retain their
|
|||
|
|
existing recovery classification.
|
|||
|
|
|
|||
|
|
Test with actual response decoding on a mixed batch and one legitimate decision
|
|||
|
|
beside an out-of-range reference. Cover equivalent numeric spellings,
|
|||
|
|
contradictory internal results, and class override from execution through the
|
|||
|
|
record supplied to Stage 17. Update [validator internals](../internal/validators.md).
|
|||
|
|
|
|||
|
|
## Stage 10 — Reduce validation context work
|
|||
|
|
|
|||
|
|
Audit #36.
|
|||
|
|
|
|||
|
|
1. In `framework/validators`, consolidate budget and optional-context policy
|
|||
|
|
across the live validation batch builder and trimmer. Measure the required
|
|||
|
|
payload first and stop testing optional material beyond the usable
|
|||
|
|
ordered prefix. Reuse computations where the exact serialized estimate
|
|||
|
|
permits it. Keep context priority, numbering, effective budget, oversized
|
|||
|
|
single-input behavior, and independently measured wire size.
|
|||
|
|
2. Retain a test oracle built from serialized payloads rather than the shared
|
|||
|
|
estimator. Use a representative long optional-context fixture to demonstrate
|
|||
|
|
reduced measurements or allocation. Retain request-shape and available
|
|||
|
|
sibling expectations under over-budget input.
|
|||
|
|
|
|||
|
|
## Stage 11 — Hand checked selections to execution
|
|||
|
|
|
|||
|
|
Audit #24 (readiness), #37, #49 (module port), #50, #55, #56.
|
|||
|
|
|
|||
|
|
1. Consolidate Audita-owned root and stage validation/fallback helpers in
|
|||
|
|
`core/config` after ConfigKit composition. Preserve presence/null,
|
|||
|
|
explicit zero, inherited values, diagnostic paths and provenance, and
|
|||
|
|
stage/role-dependent defaults. ConfigKit retains its load and override paths.
|
|||
|
|
2. Replace duplicate derivation in `framework/preflight` and
|
|||
|
|
`internal/cli/process_pipeline.go` with one checked processing plan. Store
|
|||
|
|
resolved specs, validated role choices, and effective glossary/protection
|
|||
|
|
views. Cache repeated profile/backend inspections and immutable glossary
|
|||
|
|
views only within the captured runtime. Include ordered selection and
|
|||
|
|
exemptions in reuse keys. Preserve readiness check order, first failure
|
|||
|
|
attribution, protected source paths, no model calls during preflight, and
|
|||
|
|
preflight-before-publication. Return typed status-2 failures for invalid
|
|||
|
|
selections; keep runtime/canceled outcomes distinct.
|
|||
|
|
3. Move built-in construction to composition beside the concrete modules or
|
|||
|
|
in CLI wiring. Keep the injected runner factory and dependency-light key
|
|||
|
|
catalog. Document concurrent calls and borrowed data at the module port.
|
|||
|
|
Copy owned inputs where mutation is required.
|
|||
|
|
4. In processing and inspection request construction, merge shared variables
|
|||
|
|
once per call while preserving each path's precedence, the protected-term
|
|||
|
|
default, and operator replacements for bundled prompts.
|
|||
|
|
|
|||
|
|
Check representative effective selections and provenance, first-failure
|
|||
|
|
attribution, provider-free inspection, constructible catalog entries, and both
|
|||
|
|
request precedence paths. Update [CLI/config internals](../internal/cli-config.md),
|
|||
|
|
[module internals](../internal/modules.md), and configuration docs where affected.
|
|||
|
|
|
|||
|
|
## Stage 12 — Unify structured-call completion and failure facts
|
|||
|
|
|
|||
|
|
Audit #23, #25, #41, #44.
|
|||
|
|
|
|||
|
|
1. In `framework/llm` and `framework/structuredoutput`, share once-only
|
|||
|
|
completion between ordinary execution and coordinated turns. Keep
|
|||
|
|
prepared snapshots, call IDs, terminal synchronization, native group and
|
|||
|
|
call counts, coverage and unknown usage, and diagnostic-only content from
|
|||
|
|
undelivered responses. Remove obsolete response/usage copies and callback
|
|||
|
|
payloads after checking report and debug consumers.
|
|||
|
|
2. Carry typed class, reason and cause from adapter through operation records.
|
|||
|
|
Use the internal-failure reason when a reason is absent, retaining the
|
|||
|
|
original class and cause. Apply parent cancellation precedence, identify
|
|||
|
|
native logical-operation deadline expiry, then native transport/attempt
|
|||
|
|
timeout before treating a generic deadline as logical timeout. Use the
|
|||
|
|
existing transport-failure reason for HTTP-only timeouts.
|
|||
|
|
3. Make setup diagnostics actionable at the CLI. Surface the native missing
|
|||
|
|
credential requirement and available role/profile context with the typed
|
|||
|
|
cause intact. Keep ordinary reports on their compact typed projections.
|
|||
|
|
|
|||
|
|
Exercise distinct parent, operation, and attempt timeouts with independently
|
|||
|
|
controlled budgets. Check setup class/cause and useful native information,
|
|||
|
|
plus ordinary/coordinator terminal and undelivered-evidence paths. Update
|
|||
|
|
[LLM runtime internals](../internal/llm-runtime.md).
|
|||
|
|
|
|||
|
|
## Stage 13 — Use one proposal handoff
|
|||
|
|
|
|||
|
|
Audit #46.
|
|||
|
|
|
|||
|
|
1. In `framework/proposal_generation` and `framework/runner`, return one
|
|||
|
|
set of decoded proposal results with raw local references, filtering
|
|||
|
|
evidence, and optional native exchange. Remove the generator's enriched
|
|||
|
|
projection discarded by the live wrapper. Keep runner ownership of
|
|||
|
|
sequential proposal indices within a stage, ordered by chunk and response.
|
|||
|
|
2. Check emitted interaction metadata such as `start_index` separately from
|
|||
|
|
private references. Preserve its meaningful serialized association and
|
|||
|
|
retire fields confirmed to have no surviving purpose.
|
|||
|
|
3. Move proposal-order assertions from the test-only composition harness to
|
|||
|
|
real runner results. Observe out-of-order completion in the fixture and
|
|||
|
|
verify indices and original references. Keep model response and reference
|
|||
|
|
coverage in the generator's owning tests.
|
|||
|
|
|
|||
|
|
## Stage 14 — Own each correction repair branch explicitly
|
|||
|
|
|
|||
|
|
Audit #34, #45, #48, #64 (branch integration).
|
|||
|
|
|
|||
|
|
1. In `framework/runner/repair.go`, give one proposal branch a clear
|
|||
|
|
execution input and result. Make repair capability explicit at setup. Keep
|
|||
|
|
branch state and terminal outcome together. Merge outcomes into the section
|
|||
|
|
after each branch, without mutating unrelated approvals or decisions.
|
|||
|
|
2. Run repairs through PromptKit coordination with full exchange/history and
|
|||
|
|
configured per-correction limits. Preserve initial and subsequent direct
|
|||
|
|
domain feedback, sibling independence, withdrawal, unavailable assessment,
|
|||
|
|
retargeting, and revalidation of the final allowed revision. Move useful
|
|||
|
|
structural repair and isolation tests onto this path before deleting the
|
|||
|
|
direct-continuation-only test branch.
|
|||
|
|
3. Use controlled observations or temporary instrumentation to measure
|
|||
|
|
cumulative turns, available token/byte costs, revalidation calls, and debug
|
|||
|
|
work on a representative multi-correction fixture. Remove unnecessary local
|
|||
|
|
copies. Keep those measurements out of the ordinary report contract.
|
|||
|
|
|
|||
|
|
Check independent approved siblings, attempts/turn counts, actual branch
|
|||
|
|
history and feedback, final-turn revalidation, and associated artifacts in
|
|||
|
|
production coordination.
|
|||
|
|
|
|||
|
|
## Stage 15 — Give stage execution one commit boundary
|
|||
|
|
|
|||
|
|
Audit #14, #15 (evidence), #22 and #41 (failure facts).
|
|||
|
|
|
|||
|
|
1. In `framework/runner`, check context at entry, including empty pipelines,
|
|||
|
|
and before committing a stage. Use one stage completion path to record
|
|||
|
|
outcomes and evidence. Only a successful active stage advances the working
|
|||
|
|
transcript. Preserve previously committed text for partial results.
|
|||
|
|
2. Carry initiating cause, phase and available stage/chunk/operation reference
|
|||
|
|
from the owner. Cancel and join peers on fatal failure or parent cancellation;
|
|||
|
|
keep peer interruption and secondary errors attributable without promoting
|
|||
|
|
them over the initiating failure. Continue recoverable work under existing
|
|||
|
|
partial-success semantics.
|
|||
|
|
3. Collect completed proposal evidence independently of dispatch to validators.
|
|||
|
|
After workers join, include usable out-of-order completions in interrupted
|
|||
|
|
diagnostics with deterministic indices and associations. Keep raw evidence
|
|||
|
|
from undelivered responses separate from usable proposals. Retain the ordered
|
|||
|
|
indexing/validation boundary until Stage 16 evaluates it.
|
|||
|
|
|
|||
|
|
Test canceled entry and precommit, a later completed proposal behind a slow
|
|||
|
|
earlier chunk, and a fatal stage after earlier committed text. Assert causal
|
|||
|
|
origin and preserved evidence as well as returned failure. Leave report/ledger
|
|||
|
|
projection to Stage 17.
|
|||
|
|
|
|||
|
|
## Stage 16 — Bound local preparation and actively drain results
|
|||
|
|
|
|||
|
|
Audit #15 (dispatch) and #30, under D3.
|
|||
|
|
|
|||
|
|
1. Replace the per-chunk goroutines and chunk-count-sized pending channels in
|
|||
|
|
`collectSectionProposals` and `runModulePipeline` with a private rolling
|
|||
|
|
stage window. Admit chunks to the window in input order. Charge a slot for
|
|||
|
|
each chunk still in preparation, model work, or awaiting downstream work.
|
|||
|
|
Release it when the chunk is finished and its final facts have been drained.
|
|||
|
|
Start with a small fixed local window and keep it private. Drain outcomes
|
|||
|
|
while workers run and on cancellation, then join and finish. Keep compact
|
|||
|
|
or requested completed evidence apart from remaining work. Collect results
|
|||
|
|
while the workers can still send them.
|
|||
|
|
2. Measure staged overlap and peak pending work with many sections, a
|
|||
|
|
controlled slow first proposal, and shared PromptKit capacity. Where active
|
|||
|
|
draining and ordered indexing yield adequate progress, keep that simpler
|
|||
|
|
boundary. If ordered indexing still leaves repeatable idle capacity with
|
|||
|
|
validation-ready chunks, use a chunk index and response position to identify
|
|||
|
|
candidates during work. At ordered collection, assign the existing stage
|
|||
|
|
indices and translate every projected proposal reference, including failure
|
|||
|
|
indices, repair lineage, and interaction-to-ledger mappings. Keep the
|
|||
|
|
identity-sensitive artifact data in the charged window until the index is
|
|||
|
|
assigned; use the Stage 3 writer for the final artifacts. Keep batch-local
|
|||
|
|
model references and raw response evidence intact. Check the same mappings
|
|||
|
|
after a cancellation joins the workers.
|
|||
|
|
3. Keep the module barrier and deterministic application. PromptKit retains
|
|||
|
|
admission, queueing, limits, retries and recoverable rejection. Include
|
|||
|
|
nested validation/repair in the local resource and cancellation checks
|
|||
|
|
without duplicating backend admission or holding its permits across a
|
|||
|
|
nested call. Tune the window using the measured workload and contention;
|
|||
|
|
no public window, throughput minimum, or eventual-admission guarantee follows.
|
|||
|
|
|
|||
|
|
Test actual observed overlap, finite native contention, cancellation with work
|
|||
|
|
pending, and identical index ordering for the same usable results under
|
|||
|
|
different completion orders. Report representative pending-work measurements
|
|||
|
|
without pinning a timing or worker-count threshold in the default suite.
|
|||
|
|
|
|||
|
|
## Stage 17 — Project typed report and ledger facts
|
|||
|
|
|
|||
|
|
Audit #22, #27 (ledger), #39, #41 (projection).
|
|||
|
|
|
|||
|
|
1. In `framework/processreport`, build the snapshot from the Stage 12 and 15
|
|||
|
|
facts and Stage 4 delivery outcomes. Preserve phase and available origin for
|
|||
|
|
setup, processing and publication errors. Keep recovery and work-unit counts
|
|||
|
|
and known versus unknown values. Distinguish the primary run error,
|
|||
|
|
interrupted peers, and secondary output failures without counting a fatal
|
|||
|
|
run failure again as a recoverable unit failure.
|
|||
|
|
2. Join proposals, validator decisions, repairs, application and skip evidence
|
|||
|
|
once by stable identities in `correction_ledger.go`. Use the recorded
|
|||
|
|
execution class from Stage 9. Project one candidate row with correct
|
|||
|
|
disposition, lineage and literal empty replacement values. Retain
|
|||
|
|
module-only failures. Preserve serialized fields and logical ordering.
|
|||
|
|
3. In runner application attribution, reuse section/segment lookups and
|
|||
|
|
aggregate skipped evidence once. Preserve the distinction between stale
|
|||
|
|
apply-time conflicts and response filtering. Keep exact correction,
|
|||
|
|
filtered-proposal and validation-decision identities across projection.
|
|||
|
|
|
|||
|
|
Verify decoded YAML, Markdown counts, and ledger facts against an independently
|
|||
|
|
specified mixed run, including partial decisions, repair/filter interaction,
|
|||
|
|
later failure, and an overridden validator class. Update the
|
|||
|
|
[output contract](../integrations/output-files.md) and diagnostics internals.
|
|||
|
|
|
|||
|
|
## Stage 18 — Retain evidence according to selected outputs
|
|||
|
|
|
|||
|
|
Audit #32 and supporting work for #30, #33, and #44.
|
|||
|
|
|
|||
|
|
1. Derive evidence needs once from enabled ordinary reports and effective debug
|
|||
|
|
capture. Keep compact CLI totals and available failure facts in summary-only
|
|||
|
|
mode. Allocate work-unit detail only when a consumer needs it, and avoid
|
|||
|
|
eager whole-run grids and debug-only candidate/decision histories in
|
|||
|
|
summary-only paths. Keep complete enabled report matrices and debug evidence.
|
|||
|
|
2. Discard active validation, exchange and repair state after their last
|
|||
|
|
consumer, including data copied solely for obsolete runner projections.
|
|||
|
|
Remove duplicate operation lists and unused module totals after checking
|
|||
|
|
their projections. Keep native observations and requested self-contained
|
|||
|
|
artifacts when capture is selected.
|
|||
|
|
3. Compare live/retained state for representative long runs with summary only,
|
|||
|
|
ordinary reports, debug, and their existing combinations. Check shared
|
|||
|
|
totals and logical results across modes, with detail present where selected.
|
|||
|
|
Resource measurements support reduction of waste without setting a
|
|||
|
|
constant-memory or disk-use requirement.
|
|||
|
|
|
|||
|
|
## Stage 19 — Emit native processing progress and update operator guidance
|
|||
|
|
|
|||
|
|
Audit #5, #57, and the operator-facing parts of #8, #24 and #25.
|
|||
|
|
|
|||
|
|
1. Use a small event callback from the live lifecycle owners. Supply stage and
|
|||
|
|
chunk activity, proposal/validation/repair phases and useful completions
|
|||
|
|
with available identities and observed counts. Reuse suitable PromptKit
|
|||
|
|
events where they convey useful activity. Keep event delivery within the
|
|||
|
|
invocation and finish workers before the reporter closes.
|
|||
|
|
2. Wire that callback in CLI to `Invocation.Reporter().Progress` in pinned
|
|||
|
|
CommandKit. Let CommandKit determine visibility, routing, and presentation
|
|||
|
|
failure status, independently of transcript/report processing outcomes.
|
|||
|
|
Preserve stdout transcript data. Respect ordinary mode, explicit progress,
|
|||
|
|
debug defaults, explicit `--progress=false`, and quiet suppression.
|
|||
|
|
3. Update the [CLI reference](../cli.md), [operations](../operations.md) and
|
|||
|
|
[subprocess contract](../integrations/subprocess.md) at their owned
|
|||
|
|
boundaries. Cover meaningful progress, caller-owned report destinations and
|
|||
|
|
artifact retention, and the trusted configuration bundle, including
|
|||
|
|
discovery, imports and automatic companions. Keep the distinction between
|
|||
|
|
native configuration inspection and processing readiness.
|
|||
|
|
|
|||
|
|
Use representative command tests for live activity and combined modes,
|
|||
|
|
asserting useful identities and stream separation without exact wording or
|
|||
|
|
concurrent ordering. Keep a progress-specific check separate from delegated
|
|||
|
|
inspection availability.
|
|||
|
|
|
|||
|
|
## Stage 20 — Verify the integrated sprint and release bootstrap
|
|||
|
|
|
|||
|
|
Audit #28, #29, #58, #60–62 and sprint closure.
|
|||
|
|
|
|||
|
|
1. In `.woodpecker/release.yml`, capture `git status --porcelain --untracked-files=all`
|
|||
|
|
in an assignment that checks the command's exit status separately from its
|
|||
|
|
content. Then require empty output. Preserve ReleaseKit's independent
|
|||
|
|
prepared-checkout validation and tag delivery. In `scripts/release_test.go`,
|
|||
|
|
cover clean, dirty and failed-status-with-empty-output behavior at the
|
|||
|
|
command boundary. Remove redundant tests of the comparison helper while
|
|||
|
|
keeping real release target and wiring checks.
|
|||
|
|
2. Verify signal, cancellation and exit behavior through the real executable
|
|||
|
|
or a shared production entry point. Keep narrower checks where a process
|
|||
|
|
adds no evidence. Verify effective validator roles, required prompt
|
|||
|
|
identities, embedded asset preparability, schema identity, and protective
|
|||
|
|
copies through use. Retain a representative repair/filter/later-failure
|
|||
|
|
integration scenario and independent output-choice cases. Remove the
|
|||
|
|
audited abandoned fixture inputs and helpers after checking callers.
|
|||
|
|
3. Recheck Stage 1. Adopt and verify a released upstream fix that became
|
|||
|
|
available using the same scope. Otherwise report the exact outstanding
|
|||
|
|
upstream-dependent behavior and its disposition. Treat that item as
|
|||
|
|
outstanding, while verifying the independent Audita work. Keep the
|
|||
|
|
invalid-input and cancellation cases distinct.
|
|||
|
|
4. Account for the [catalog](audit/catalog.md) against the landed changes and
|
|||
|
|
stage evidence. Entries #2 and #4 need no implementation, and only the
|
|||
|
|
accepted portions of #6 and #18 were selected. Check canonical docs,
|
|||
|
|
examples, and affected links and remove stale implemented-behavior claims.
|
|||
|
|
Keep historical audit observations intact.
|
|||
|
|
5. From the integrated tree, build the CLI and run the full offline
|
|||
|
|
`go test ./...` with the supported Go toolchain. Run targeted race checks
|
|||
|
|
for changed shared-state paths and review the representative measurements
|
|||
|
|
from Stages 6, 8, 10, 14, 16 and 18. Verify formatting and whitespace,
|
|||
|
|
and inspect the final diff. Reuse successful checks for unchanged content.
|
|||
|
|
Existing [release validation](../policy/release.md#candidate-validation)
|
|||
|
|
applies when a release is prepared.
|
|||
|
|
|
|||
|
|
The completed sprint should leave the target architecture reachable through
|
|||
|
|
`audita process` and the standard delegated commands. Report checks and
|
|||
|
|
measurements actually performed, including any outstanding external dependency.
|