307 lines
41 KiB
Markdown
307 lines
41 KiB
Markdown
# Audit assessment and decision register
|
||
|
||
**Reviewed baseline:** `faba8ab` (2026-11-15). **Status:** assessment complete;
|
||
remediation not implemented. This document owns classification, recommended
|
||
scope, and pending decisions for the [catalog](catalog.md). The source audits
|
||
retain their evidence; their suggested remedies are not approved requirements.
|
||
|
||
The catalog covers 103 source findings in 62 entries. Every catalog entry is
|
||
assessed below. The source tree is unchanged since the audited implementation
|
||
at `c4ccd78`; subsequent commits added audit documentation. This review used
|
||
the source audits, current contracts and ADRs, and targeted implementation
|
||
inspection. It did not independently rerun all audit probes, benchmarks, or the
|
||
test suite. Reproductions reported by the audits remain attributed to them;
|
||
allocation/call-path observations establish opportunities, not measured
|
||
production performance.
|
||
|
||
## Evaluation rules
|
||
|
||
The maintainer approved the taxonomy and evaluation process, not every proposed
|
||
remedy. Correctness restores established or reasonably expected behavior;
|
||
observable change alone does not make a fix a new policy. Refactor size is not
|
||
an approval criterion. Efficiency and ownership improvements should proceed even
|
||
when substantial restructuring is appropriate.
|
||
|
||
| Code | Category | Treatment |
|
||
| --- | --- | --- |
|
||
| C | Correctness | Fix substantiated departures from intended behavior. |
|
||
| E | Efficiency and resource use | Improve work, allocation, and retention while preserving domain and operator policy. |
|
||
| A | Architecture and ownership | Consolidate responsibilities and improve boundaries; retain distinct domain behavior. |
|
||
| T | Tests and verification | Protect consequential behavior at real boundaries; remove redundant and implementation-bound assertions. |
|
||
| M | Maintenance and documentation | Remove obsolete machinery and clarify implemented behavior. |
|
||
| P | Substantive policy or product change | Requires express maintainer approval before implementation. |
|
||
|
||
**Ready** means suitable for implementation planning within the stated scope,
|
||
not implemented or validated by this document. **Investigate** denotes technical
|
||
work the agent can undertake without a product decision. **D1–D7** identify
|
||
pending substantive choices; recommendations are not decisions. **Closed**
|
||
denotes no remediation for the specified scope. Mixed entries deliberately
|
||
separate ready work from decisions and exclusions.
|
||
|
||
Evidence is judged against the [architecture policy](../../policy/architecture.md),
|
||
[operations contract](../../operations.md),
|
||
[CLI contract](../../cli.md),
|
||
[output contract](../../integrations/output-files.md),
|
||
[validator contract](../../internal/validators.md), and relevant catalog sources.
|
||
An audit author's expectation alone does not establish a requirement.
|
||
Internal-only exposure reduces urgency, not the value of a valid invariant fix.
|
||
Tests and acceptance criteria must follow the [testing policy](../../policy/testing.md).
|
||
|
||
The maintainer's three WONTFIX decisions remain controlling: #2 in full,
|
||
#6 for interruption of already-blocked I/O, and #18 for Unicode behavior of Go
|
||
regexp word boundaries. Using the appropriate standard primitive for Audita's
|
||
own requirement is legitimate; replacing or compensating for accepted Go
|
||
semantics is not selected. The
|
||
[agent policy](../../policy/agents.md#implementation-discipline) also excludes
|
||
invented hostile-filesystem and power-loss guarantees. Source recommendations
|
||
that conflict with these dispositions must not re-enter an implementation plan.
|
||
|
||
## Entry assessments
|
||
|
||
Each entry links to the catalog's original evidence. Category codes can combine;
|
||
the final column bounds the recommended remediation and distinguishes confirmed
|
||
behavior from proposed extensions.
|
||
|
||
| Entry | Categories | Disposition | Assessment and scope |
|
||
| --- | --- | --- | --- |
|
||
| [1. Bind output writes to the checked destinations](catalog.md#1-bind-output-writes-to-the-checked-destinations) | C, A | Ready | The wrong-file reproduction and repeated report-path resolution substantiate application defects. Resolve paths with filesystem semantics, carry checked parent/destination information through publication, and share staging ownership. Preserve symlinks and intentional in-place replacement. No hostile-directory race-proof guarantee, cross-file transaction, or custom filesystem machinery is selected. |
|
||
| [2. Keep model requests within the configured transport boundary](catalog.md#2-keep-model-requests-within-the-configured-transport-boundary) | — | Closed: WONTFIX | Accepted Go HTTP redirect and credential-forwarding behavior. The source audit's proposed redirect restrictions do not authorize implementation. |
|
||
| [3. Protect credentials inside nested debug responses](catalog.md#3-protect-credentials-inside-nested-debug-responses) | C; P | Ready in part; D1 | Nested JSON can retain an encoded credential after outer redaction. Sanitize known structured-response fields after decoding and verify decoded artifacts. Raw malformed-body retention is a separate policy choice in D1; fixing valid nested content does not depend on it. |
|
||
| [4. Make debug artifact permissions private by default](catalog.md#4-make-debug-artifact-permissions-private-by-default) | C, M | Ready | The observed public directory/file modes expose confidential debug content. Use private creation modes and a secure per-user implicit root with standard facilities; retain explicit work-directory selection and nonfatal debug setup errors. Exact default-path selection is engineering work, with an operator documentation update. Do not change permissions on unrelated existing directories or add a hostile-filesystem guarantee. |
|
||
| [5. Define the trust boundary for discovered configuration](catalog.md#5-define-the-trust-boundary-for-discovered-configuration) | M; P | Ready for documentation; extension excluded | Automatic companions are intentional configuration composition, not a discovery bug. Explain that the searched locations, selected bundle, imports, and companions are trusted configuration, and keep uploaded transcripts separate. Disabling discovery or adding a new isolation mode is outside this remediation; no ConfigKit replacement loader. |
|
||
| [6. Make blocking transcript input and delivery cancellation-aware](catalog.md#6-make-blocking-transcript-input-and-delivery-cancellation-aware) | C; — | Ready in part; WONTFIX in part | Honor cancellation before regular-file publication and when blocked delivery returns; produce failure status even if bytes have escaped. Do not promise interruption of already-blocked standard I/O. Do not ban FIFOs, add detached writer goroutines, or revive the source audits' context-aware stream machinery. |
|
||
| [7. Publish truthful failure reports and preserve initialized evidence](catalog.md#7-publish-truthful-failure-reports-and-preserve-initialized-evidence) | C, A, T | Ready | Publishing completed reports after a known delivery failure contradicts their meaning. Select failed status before the first report write; preserve initialized evidence if rendering fails. Carry rendering errors explicitly and retain bounded best-effort finalization. Cross-file atomicity remains outside the contract. |
|
||
| [8. Prevent concurrent runs from clobbering ordinary reports](catalog.md#8-prevent-concurrent-runs-from-clobbering-ordinary-reports) | M; P | Ready for documentation; D2 | The collision is demonstrated, but independent concurrent ownership of fixed report basenames is not promised. Document per-invocation directory selection now. Automatic run-specific placement, locking, or rejection of competing writers changes the workflow and belongs to D2. |
|
||
| [9. Preserve the ordinary YAML schema during redaction](catalog.md#9-preserve-the-ordinary-yaml-schema-during-redaction) | C | Ready | Redacting an application-owned key or status breaks the report schema. Redact input-derived fields before serialization; retain fixed keys, enums, and reason codes. A literal coincidence between a credential and a fixed schema word does not make that word credential-derived data. |
|
||
| [10. Reject contradictory validation decisions](catalog.md#10-reject-contradictory-validation-decisions) | C, A | Ready | Approved-plus-unavailable contradicts the existing acceptance invariant, although bundled producers are consistent. Establish an authoritative internal disposition and reject invalid internal results before use; derive redundant serialized approval as needed. This is an internal contract failure, distinct from an unavailable model assessment. |
|
||
| [11. Preserve usable validator decisions when extra references cannot be decoded](catalog.md#11-preserve-usable-validator-decisions-when-extra-references-cannot-be-decoded) | C, T | Ready | The mixed-response probe violates existing sibling-preservation rules. Retain exact numeric references until range reconciliation, accepting integral JSON number spellings without float rounding. Ignore extras before conversion to local indices. Preserve whole-response structural validation; do not add semantic retries or general malformed-response salvage. |
|
||
| [12. Use full-segment evidence for protected-name checks](catalog.md#12-use-full-segment-evidence-for-protected-name-checks) | C, A, T | Ready | The full-segment examples establish a mismatch with existing protection intent. Share glossary-owned assessment using usable before/after segment evidence; recheck current text at application. Preserve active-glossary exemptions and spoken-word repetition rules. Missing or ambiguous previews cannot justify a fabricated protection verdict; retain independently supported applicability findings. Empty-result previews can still provide evidence. |
|
||
| [13. Preserve transcript input fidelity during JSON shape detection](catalog.md#13-preserve-transcript-input-fidelity-during-json-shape-detection) | C, E, T | Ready | Generic numeric decoding demonstrably changes accepted IDs and processes ignored fields unnecessarily. Keep the selected array as raw JSON and decode typed fields directly. Preserve both input shapes, optional IDs, accepted ranges, and unknown-field handling; no new input restrictions. |
|
||
| [14. Honor cancellation at the runner commit boundary](catalog.md#14-honor-cancellation-at-the-runner-commit-boundary) | C, A, T | Ready | Canceled entry and precommit paths violate cooperative cancellation and stage ownership. Check natural boundaries and assemble results once, preserving earlier committed stages. Do not claim atomic synchronization between cancellation and filesystem or memory operations, or introduce forcible interruption of standard calls. |
|
||
| [15. Retain completed proposal evidence behind slow sections](catalog.md#15-retain-completed-proposal-evidence-behind-slow-sections) | C, E, A | Ready | Completed proposal evidence must survive independently of ordered dispatch. Retain it on interruption and assign deterministic identities after joining workers. Reducing within-stage head-of-line waiting is a valid efficiency change if identities, validation requirements, application order, and the module barrier survive; it does not authorize independent whole-pipeline traversal. |
|
||
| [16. Keep ownership of failed staging cleanup](catalog.md#16-keep-ownership-of-failed-staging-cleanup) | C, A, T | Ready | The cleanup probe demonstrates lost resource ownership. Keep temporary paths until removal succeeds or absence is established, and surface cleanup errors without losing the primary failure. Share cleanup mechanics with publication. No persistent scavenger or indefinite retry service is required. |
|
||
| [17. Preserve the complete time interval when merging segments](catalog.md#17-preserve-the-complete-time-interval-when-merging-segments) | C, T | Ready | Nested intervals demonstrably truncate speech coverage. Use the full interval for merge eligibility, output, and subsequent gaps. Preserve accepted overlap and merge rules; rejecting overlapping input would be a separate restriction. |
|
||
| [18. Match protected names at Unicode and punctuation boundaries](catalog.md#18-match-protected-names-at-unicode-and-punctuation-boundaries) | C; — | Ready in part; WONTFIX in part | Keep the accepted ASCII semantics of Go regexp word boundaries. Recognizing configured punctuation-edged spellings is an application matching defect: express appropriate literal-edge checks with standard primitives, preserving longest-match selection, exemptions, and case behavior. Do not turn this into Unicode boundary emulation or a replacement regexp engine. |
|
||
| [19. Detect overlapping occurrences when requiring a unique edit span](catalog.md#19-detect-overlapping-occurrences-when-requiring-a-unique-edit-span) | C, T | Ready | Require-unique means one possible literal location; the overlapping-span probe has two. Use standard search primitives to detect a second starting offset. This corrects Audita's choice of counting operation, not Go's nonoverlapping Count semantics. Preserve replace-all's existing nonoverlapping behavior. |
|
||
| [20. Satisfy feasible explicit chunk targets](catalog.md#20-satisfy-feasible-explicit-chunk-targets) | C, E, T | Ready | The feasible partition counterexample establishes a chunker defect. Guarantee count and maximum constraints before optimizing balance. Choose a proportionate deterministic algorithm and test feasibility independently; no approval is needed for the algorithm or a different balanced cut. Do not change configured budgets or add a new balancing objective. |
|
||
| [21. Keep input order for exactly tied timestamps](catalog.md#21-keep-input-order-for-exactly-tied-timestamps) | C, T | Ready | Arbitrary reorder of tied utterances has no domain justification. Preserve source order for equal sort keys with a stable sort. No new end-time ordering criterion is implied; retain the current primary chronological key. |
|
||
| [22. Preserve failure phase and origin in run reports](catalog.md#22-preserve-failure-phase-and-origin-in-run-reports) | C, A, T | Ready | Required phase and available operation origin are lost between owners. Carry structured facts and assemble run records once, keeping primary failure separate from canceled peers and secondary delivery failures. Preserve documented counts and serialized identities. |
|
||
| [23. Classify provider attempt timeouts separately from operation deadlines](catalog.md#23-classify-provider-attempt-timeouts-separately-from-operation-deadlines) | C, T | Ready | The attempt-timeout probe and classifier order establish misclassification. Prefer native PromptKit classifications and parent cancellation precedence. Use the existing transport-failure reason for an HTTP-only timeout; a new public reason code is unnecessary for this fix. |
|
||
| [24. Use input diagnostics for operator-correctable failures](catalog.md#24-use-input-diagnostics-for-operator-correctable-failures) | C, T | Ready; technical ownership check | Processing readiness and chunk infeasibility should follow the existing input-error contract while I/O and cancellation remain runtime failures. Delegated inspection has a demonstrated classification mismatch, but locate responsibility in Audita's loader versus ckprompt/CommandKit before editing. Fix or request a fix at that owner; do not create a local replacement command surface. |
|
||
| [25. Identify missing required credentials in safe diagnostics](catalog.md#25-identify-missing-required-credentials-in-safe-diagnostics) | C, M | Ready | The missing-credential cause is available but obscured. Surface a safe requirement and role/profile context from typed metadata, preserving the native cause. Keep PromptKit responsible for credential validation; do not move provider preparation or secret inspection into a new local subsystem. |
|
||
| [26. Render input and output paths as plain Markdown text](catalog.md#26-render-input-and-output-paths-as-plain-markdown-text) | C, T | Ready | Input paths are data; rendered Markdown currently gives some filenames unintended syntax. Escape them as literal text without altering structured path values. This restores report presentation rather than imposing filename restrictions. |
|
||
| [27. Keep the validator execution class used by the runner](catalog.md#27-keep-the-validator-execution-class-used-by-the-runner) | C, A | Ready | Runtime and ledger can disagree for supported injected validator classes. Record the effective execution class with decisions and project it into the ledger. Preserve existing output fields and fallback behavior; no new validator registration policy. |
|
||
| [28. Verify signal and exit handling in the production executable](catalog.md#28-verify-signal-and-exit-handling-in-the-production-executable) | T | Ready | The startup helper duplicates the production boundary, so coverage does not prove actual executable signal wiring. Exercise representative startup and cancellation through the real executable or shared production entry point; keep narrower tests for other CLI behavior. |
|
||
| [29. Fail the release bootstrap when cleanliness cannot be established](catalog.md#29-fail-the-release-bootstrap-when-cleanliness-cannot-be-established) | C, T | Ready | The shell probe establishes that a failed status command can appear clean. Check its exit result explicitly and test that boundary. ReleaseKit already owns the independent delivery validation; do not add another local release validator or publication gate. |
|
||
| [30. Bound pending proposal work before backend admission](catalog.md#30-bound-pending-proposal-work-before-backend-admission) | E, A, T; P | Investigate implementation; D3 for policy | Fan-out and retained results scale with chunks; finite native admission rejection is itself working as configured. Design bounded local preparation/dispatch and concurrent result draining without changing backend ownership, stable identities, cancellation joins, or configured rejection semantics. Measure representative memory/contention to size the implementation. Guaranteeing a turn despite capacity rejection or adding a new scheduling control requires D3. |
|
||
| [31. Reduce peak memory for large transcript input and output](catalog.md#31-reduce-peak-memory-for-large-transcript-input-and-output) | E, A; P | Ready in part; D4 | Remove the generic JSON round trip and release whole-input representations after their last use. Share typed output projection; streaming into regular-file staging is an implementation option. Preserve encoding-failure and stream-delivery semantics. An application input-size limit or regular-file-only restriction is not an optimization; D4 covers size limits, and FIFO exclusion is not selected. |
|
||
| [32. Retain only the run evidence required by enabled outputs](catalog.md#32-retain-only-the-run-evidence-required-by-enabled-outputs) | E, A | Ready | The production consumers establish that much completed evidence is unused without its corresponding output. Retain only required live repair state, compact counts, and selected report/debug detail. Derive not-started counts without a full eager grid where possible; keep complete matrices when enabled and preserve fatal-origin evidence. |
|
||
| [33. Limit debug retention costs within a run](catalog.md#33-limit-debug-retention-costs-within-a-run) | E, A; P | Ready in part; D5 | Reduce duplicate event representations and serialization; coalesce equivalent warnings while retaining failure counts and affected evidence. Those are efficiency improvements. Quotas, automatic deletion, stopping future capture, or replacing self-contained interactions with reference-based artifacts change retained evidence and need D5. |
|
||
| [34. Account for cumulative correction-repair history and work](catalog.md#34-account-for-cumulative-correction-repair-history-and-work) | E; P | Investigate costs; D6 | Repeated full context follows the deliberately selected repair architecture, so growth alone is not a defect. Measure cumulative requests, bytes, revalidation, and debug costs using existing observations or development instrumentation. Avoid duplicate local representations. New aggregate stopping budgets or reduced model context require D6; do not add ordinary-report fields merely to perform this analysis. |
|
||
| [35. Avoid rebuilding growing normalization prefixes](catalog.md#35-avoid-rebuilding-growing-normalization-prefixes) | E | Ready | Growing merge prefixes are repeatedly scanned and copied. Use incremental text/category accumulation and exact estimator components while retaining joiner behavior, punctuation rounding, limits, and category order. Benchmarks should verify the improvement; large refactoring is not a reason to retain quadratic work. |
|
||
| [36. Skip validation context work beyond the usable budget](catalog.md#36-skip-validation-context-work-beyond-the-usable-budget) | E, A, T | Ready | Unusable optional context is serialized needlessly. Stop measurement once required-only or ordered-prefix costs settle the fit. Share trimming/batching policy without losing the independent serialized estimator oracle or changing context priority and oversized-item handling. |
|
||
| [37. Reuse checked processing selections and matching views](catalog.md#37-reuse-checked-processing-selections-and-matching-views) | E, A | Ready | Readiness and execution derive the same selections separately. Carry one checked processing plan and reuse immutable effective views and inspections per captured runtime. Preserve provenance and first-failure attribution; cache keys must include ordered selections and exemptions. Do not add cross-invocation caches or reload configuration. |
|
||
| [38. Select glossary context by identity membership](catalog.md#38-select-glossary-context-by-identity-membership) | E, A | Ready | Context needs identity membership, not a record for every occurrence/contributor pair. Share scanning but avoid expanding multiplicity for context. Preserve glossary order and keep occurrence evidence for protection. |
|
||
| [39. Avoid rescanning sections for skipped edits](catalog.md#39-avoid-rescanning-sections-for-skipped-edits) | E | Ready | Skipped-edit reporting repeats segment searches and filtering merges. Reuse a lookup and aggregate once. Preserve the distinction between stale conflicts and actual response filtering; do not broaden rejection or recovery rules. |
|
||
| [40. Use one validator contract and one owner for decision semantics](catalog.md#40-use-one-validator-contract-and-one-owner-for-decision-semantics) | A, C, M | Ready | Duplicate ports and runner-specific verdict rewriting obscure one existing validation contract. Consolidate types, applicability, and decision meaning with their owner; remove unused envelope fields. Preserve zero-budget shortlisting, independent repair feedback, cardinality, and unavailable outcomes. No broad plugin framework or new validator policy is needed. |
|
||
| [41. Carry structured failure facts through model operations and reporting](catalog.md#41-carry-structured-failure-facts-through-model-operations-and-reporting) | A, C | Ready | Failure class and reason already exist but are discarded and reconstructed. Carry typed facts through role-specific handling and report counts. Use a consistent internal-failure reason fallback for missing reasons while retaining the original failure class and cause. Avoid reclassifying an operation from its display reason. |
|
||
| [42. Keep required output publication separate from optional debug retention](catalog.md#42-keep-required-output-publication-separate-from-optional-debug-retention) | A, C | Ready | Required publication and optional retained evidence have distinct lifecycles. Separate their APIs and share regular-file staging where behavior is common. Preserve transcript-before-report ordering, nontransactional finalization, private staging, and nonfatal debug errors. Coordinate with #1, #7, #16, and #22. |
|
||
| [43. Inject one interaction diagnostics contract](catalog.md#43-inject-one-interaction-diagnostics-contract) | A, C | Ready | Equivalent writer contracts and concrete filesystem/credential wiring cross role boundaries. Inject one diagnostics sink, retaining role-owned semantic payloads, lazy redaction inputs, combined repair/validation secrets, and nonfatal write warnings. D1 and D5 constrain only changes to what is retained. |
|
||
| [44. Simplify structured-call completion and remove obsolete record fields](catalog.md#44-simplify-structured-call-completion-and-remove-obsolete-record-fields) | A, M, T | Ready | Ordinary calls and coordinator turns share completion responsibilities; obsolete payload copies have no production reader. Use explicit once-only completion and remove superseded state after moving meaningful assertions. Preserve native terminal counts, typed causes, and undelivered diagnostic evidence. |
|
||
| [45. Retire the test-only direct continuation branch](catalog.md#45-retire-the-test-only-direct-continuation-branch) | M, T | Ready | Direct continuation is test-only; production uses coordination. Move sibling-isolation and structural-repair coverage onto the coordinator before removing the branch. Removing the redundant adapter surface does not remove PromptKit capability or production repair. |
|
||
| [46. Use a single proposal handoff and test real runner indexing](catalog.md#46-use-a-single-proposal-handoff-and-test-real-runner-indexing) | A, E, M, T | Ready | Generator enrichment is discarded and real indices are assigned by the runner. Keep one proposal handoff with original local references, and assert actual deterministic runner indices under observed out-of-order completion. Review emitted diagnostic start_index separately before dropping it; move useful tests before removing their harness. |
|
||
| [47. Centralize request-local segment reference resolution](catalog.md#47-centralize-request-local-segment-reference-resolution) | A, M | Ready | Initial and repaired proposals duplicate positional reference translation. Share a small protocol lookup while retaining raw references, invalid-target evidence, distinct reference namespaces, and retargeted revalidation. Independent caller expectations should survive consolidation. |
|
||
| [48. Give one repair branch a clear execution and result boundary](catalog.md#48-give-one-repair-branch-a-clear-execution-and-result-boundary) | A | Ready | Repair branch state is spread through closures and section aggregation. Give one branch its own execution/result boundary and make capabilities explicit at setup. Keep PromptKit's conversation and turn-limit ownership, independent siblings, final-turn validation, and current repair policy. |
|
||
| [49. Define concurrency and ownership in module and validator contracts](catalog.md#49-define-concurrency-and-ownership-in-module-and-validator-contracts) | M, A | Ready | Shared instances and borrowed data already impose concurrent-call and ownership obligations. Document those obligations at the port, permit synchronized stateful implementations, and use owned copies where mutation is needed. This clarifies the existing runtime, not a new ban on state or a forced-cancellation guarantee. |
|
||
| [50. Consolidate root and per-module configuration rules](catalog.md#50-consolidate-root-and-per-module-configuration-rules) | A, M | Ready | Root and stage checks duplicate application fallback rules. Consolidate typed checks and resolution helpers while preserving null, zero, inheritance, provenance, diagnostic paths, and stage-dependent defaults. ConfigKit continues to own all composition before handoff; do not reproduce its machinery. |
|
||
| [51. Unify transcript conversion and retire redundant parser wrappers](catalog.md#51-unify-transcript-conversion-and-retire-redundant-parser-wrappers) | A, M, C, T | Ready | The strict parser's panic is real but limited to an unused production path. Remove obsolete wrappers after moving useful tests and share the live source-to-canonical conversion. Fixing and retaining the obsolete parser would preserve unnecessary machinery. Keep optional source IDs and normalized output behavior. |
|
||
| [52. Give public transcript encoders one segment projection](catalog.md#52-give-public-transcript-encoders-one-segment-projection) | A, E, T | Ready | Two public encoders duplicate segment projection. Give the output boundary one projection and test both envelopes with independent expected fields. Preserve omission, category, numeric, and schema/version behavior. |
|
||
| [53. Use one simple token estimate across text-processing packages](catalog.md#53-use-one-simple-token-estimate-across-text-processing-packages) | A, E, T | Ready | Both counters implement the same heuristic. Share its implementation/components and preserve exact estimates, consumer budgets, and independent expected-count tests. A new tokenizer or changed heuristic is outside this consolidation. |
|
||
| [54. Centralize transcript cloning](catalog.md#54-centralize-transcript-cloning) | A, M, T | Ready | Deep copying belongs with the transcript model. Preserve nil versus explicitly empty mutable slices in the shared clone, both caller copy boundaries, and current external encoding. Verify independence using fixtures not constructed by the same clone. This internal fidelity choice needs no product decision. |
|
||
| [55. Assemble built-in modules at the composition boundary](catalog.md#55-assemble-built-in-modules-at-the-composition-boundary) | A | Ready | Production built-in assembly belongs at composition; the runner already accepts a factory. Move construction while keeping the configuration catalog dependency-light and retaining override seams. Package layout is an implementation choice, not a product decision. |
|
||
| [56. Merge shared prompt variables once per model call](catalog.md#56-merge-shared-prompt-variables-once-per-model-call) | E, M | Ready | Processing loads the same shared values twice. Merge once while preserving processing and inspection's different precedence and the protected-term default. Do not accidentally weaken operator prompt-definition replacement support or invent new shared-variable precedence. |
|
||
| [57. Clarify what the inherited progress flag does in processing](catalog.md#57-clarify-what-the-inherited-progress-flag-does-in-processing) | M; P | Ready for clarification; D7 | The standard flag is exposed without processing events. Accurate help/documentation can proceed, preserving standard command acceptance. Adding a processing progress stream is a user-visible feature: D7 selects scope. Any standard help/capability gap belongs in CommandKit rather than local replacement routing. |
|
||
| [58. Test registered validator roles and embedded assets through their behavior](catalog.md#58-test-registered-validator-roles-and-embedded-assets-through-their-behavior) | T, M | Ready | Registry tests duplicate representation assertions and prompt-count checks can miss substitutions. Check required identities, effective roles, preparability, schema identity, and protective copies at their actual owners. Do not replace them with prompt wording or private implementation assertions. |
|
||
| [59. Consolidate overlapping normalization fixtures](catalog.md#59-consolidate-overlapping-normalization-fixtures) | T, M | Ready | Two fixtures protect the same nonmerge outcome. Consolidate their meaningful assertions, keeping distinct normalization behavior and independent input expectations. Coordinate with #51 so copied parser conversion is not retained in the tests. |
|
||
| [60. Focus integration tests on decoded artifacts and independent output choices](catalog.md#60-focus-integration-tests-on-decoded-artifacts-and-independent-output-choices) | T | Ready | Decoded artifact identities and evidence associations are the contract; escaping and repeated large fixtures are incidental. Keep one full repair/filter/later-failure attribution scenario and use smaller independent-output fixtures. Preserve real role selection, successive text, and output-choice coverage. |
|
||
| [61. Remove abandoned test support and unused fixture inputs](catalog.md#61-remove-abandoned-test-support-and-unused-fixture-inputs) | M, T | Ready | The reachability audit found abandoned helpers and unused fixture inputs after checking registrations and serialization. Delete them, retaining compile-time interface fixtures and active behavior assertions. Recheck callers when implementing if the code has advanced. |
|
||
| [62. Remove unreachable merge and file helpers](catalog.md#62-remove-unreachable-merge-and-file-helpers) | M | Ready | Unreachable merge and file helpers duplicate or bypass live owners. Remove them after a current reference check, retaining live normalization, input diagnosis, and output-session behavior. |
|
||
|
||
## Implementation planning implications
|
||
|
||
Group related work by owner rather than implementing catalog numbers as 62
|
||
independent tasks. Fixes and their supporting ownership changes belong together:
|
||
|
||
- Output publication and diagnostic integrity: #1, #3–9, #16, #22, #26,
|
||
#42–43, with D1/D2/D5 limited to their policy portions.
|
||
- Transcript and glossary correctness: #11–13, #17–21, #35–39, #51–54.
|
||
- Runner and model-operation lifecycle: #10, #14–15, #23–25, #27, #30,
|
||
#32, #40–41, #44–49, #55–56.
|
||
- Verification and obsolete machinery: #28–29, #58–62 and the tests attached
|
||
to each other change.
|
||
|
||
These are dependency groups, not a commitment to one large rewrite. Preserve
|
||
independent fixture expectations when consolidating implementations. Move useful
|
||
coverage to the live boundary before deleting test-only execution paths. Verify
|
||
resource improvements with representative measurements; do not require production
|
||
benchmarks before removing plainly redundant work.
|
||
|
||
Technical follow-up does not require another approval round: locate delegated
|
||
error ownership for #24; design and measure local dispatch/result draining for
|
||
#30; measure aggregate repair costs for #34. Escalate only if the concrete remedy
|
||
crosses a policy boundary below. Native ConfigKit, PromptKit, CommandKit, and
|
||
ReleaseKit responsibilities remain upstream-owned. Submitting a feature request
|
||
is preferable to maintaining a local workaround when an upstream API is the gap.
|
||
|
||
Earlier review questions about full-segment protection, cloning details, internal
|
||
decision representation, and ordinary publication ownership are resolved here as
|
||
correctness/engineering work. They are not additional open product decisions.
|
||
Independent whole-pipeline chunk traversal and semantic recovery of unavailable
|
||
validator responses remain [separate deferred features](../future.md).
|
||
|
||
## Open Decisions
|
||
|
||
All seven decisions below are pending. Accepting or rejecting an optional
|
||
extension must not delay the independent ready portions of its catalog entries.
|
||
|
||
### D1. What evidence should remain when a raw model response cannot be safely decoded?
|
||
|
||
**Applies to:** #3; implemented through the writer ownership in #43.
|
||
|
||
**Recommendation:** retain safe interaction metadata and an explicit omission
|
||
marker, but omit an undecodable raw response body. Continue retaining sanitized
|
||
structured content when decoding succeeds. This preserves useful diagnosis
|
||
without pretending that exact-string substitution covers every representation
|
||
of a secret in malformed content. Optional evidence loss remains a warning,
|
||
not a processing failure.
|
||
|
||
**Alternative:** omit the entire affected response artifact and report a safe
|
||
warning, retaining its operation metadata elsewhere. This is simpler and avoids
|
||
partial artifact interpretation, but loses more useful evidence and associations.
|
||
A richer malformed-text sanitizer could preserve more content, but needs a
|
||
bounded, demonstrable redaction contract and additional machinery; unconditional
|
||
raw retention would relax the existing secret-protection requirement and is not
|
||
recommended.
|
||
|
||
**Why approval:** this deliberately reduces retained evidence for a class of
|
||
responses. Decoded nested-JSON redaction and preservation of ordinary report
|
||
schemas remain ready fixes regardless of this choice.
|
||
|
||
### D2. Should Audita manage concurrent ownership of ordinary report destinations?
|
||
|
||
**Applies to:** #8.
|
||
|
||
**Recommendation:** retain the current placement contract and document that
|
||
concurrent callers select distinct report directories. This avoids new locking,
|
||
waiting, or placement semantics and works with the existing explicit destination
|
||
selection. The audit establishes clobbering under shared use, not a broken
|
||
promise of per-run isolation.
|
||
|
||
**Alternative:** give runs automatically isolated report directories, with a
|
||
documented way to discover them. This makes concurrent jobs easier and preserves
|
||
each run's evidence, but changes default artifact locations and downstream
|
||
collection. Serializing or rejecting competing writers is another option, but
|
||
adds contention and recovery semantics merely to retain shared names.
|
||
|
||
**Why approval:** automatic placement, waiting, and rejection change the operator
|
||
workflow. Correct publication to the checked destination in #1 is independent.
|
||
|
||
### D3. Should local dispatch change what backend saturation means?
|
||
|
||
**Applies to:** #30, with scheduling interactions in #15.
|
||
|
||
**Recommendation:** retain PromptKit's configured waiting and recoverable
|
||
capacity-rejection behavior. Improve local resource use and dispatch as
|
||
engineering work, but do not add an outer capacity-retry loop, a separate
|
||
backend limiter, or a promise that every chunk eventually receives admission.
|
||
Use native settings where operators want more waiting capacity.
|
||
|
||
**Alternative:** explicitly select an application policy that holds pending
|
||
chunks until they can receive a turn, subject to cancellation and existing
|
||
operation budgets. That could avoid losing useful work under finite queues, but
|
||
requires defining admission timing, fairness across proposal/validation/repair,
|
||
and whether deadlines cover the new wait. Seek suitable PromptKit support before
|
||
implementing local admission policy.
|
||
|
||
**Why approval:** replacing configured rejection with guaranteed waiting changes
|
||
run outcomes and timing. A bounded internal work window alone is not inherently
|
||
a policy change; its implementation must demonstrate that it stays on the chosen
|
||
side of this boundary. Native rejection is not a PromptKit defect.
|
||
|
||
### D4. Should Audita impose an application-level transcript input budget?
|
||
|
||
**Applies to:** the optional restriction in #31.
|
||
|
||
**Recommendation:** do not add a new input limit in this remediation. Remove
|
||
avoidable copies and let upload-serving integrations impose their own upload
|
||
and process budgets. The source audit identifies an exposure condition, not a
|
||
measured size threshold suitable for all CLI workloads.
|
||
|
||
**Alternative:** add an explicit opt-in input-byte budget checked during
|
||
acquisition. This helps integrations without their own enforcement, but adds
|
||
configuration, a new rejection condition, and diagnostic/documentation work.
|
||
A default cap would additionally reject legitimate large jobs and needs
|
||
independent justification.
|
||
|
||
**Why approval:** accepted input would become conditional on a new constraint.
|
||
No input-format narrowing or regular-file-only policy is selected.
|
||
|
||
### D5. Should debug capture acquire quotas, automatic retention rules, or an early-stop policy?
|
||
|
||
**Applies to:** the policy portion of #33.
|
||
|
||
**Recommendation:** preserve requested evidence and best-effort capture; optimize
|
||
its representation internally and coalesce equivalent write warnings with counts
|
||
and affected-artifact information. Do not add quotas, automatic deletion, or
|
||
disable later writes after the first failure in this remediation.
|
||
|
||
**Alternative:** adopt explicit opt-in limits or a documented stop-capture policy
|
||
after persistent failures. This can protect disk and reduce repeated failed
|
||
work, but intentionally loses requested evidence and must expose what was
|
||
omitted. Automatic cleanup across runs requires separate retention ownership.
|
||
Replacing self-contained interaction artifacts with shared-history references
|
||
also needs a deliberate consumer contract.
|
||
|
||
**Why approval:** these policies change what operators receive from debug mode.
|
||
Private permissions/default-root safety in #4 and elimination of duplicate
|
||
in-memory representations are ready improvements without these policies.
|
||
|
||
### D6. Should correction repair gain aggregate budgets or reduced conversation context?
|
||
|
||
**Applies to:** the optional behavior changes in #34.
|
||
|
||
**Recommendation:** retain the selected full conversation history and
|
||
per-correction limits. Measure cumulative work and remove local duplication
|
||
before proposing any new stopping or context policy. The resource growth partly
|
||
reflects deliberate useful work rather than an implementation defect.
|
||
|
||
**Alternative:** add an opt-in section/run budget covering repair and
|
||
revalidation, with explicit termination outcomes; or separately evaluate reduced
|
||
context. Budgets provide predictable resource envelopes but may leave later
|
||
corrections unrepaired. Context reduction can lower costs but can also reduce
|
||
editorial quality. Neither follows automatically from the audit's asymptotic
|
||
description.
|
||
|
||
**Why approval:** these changes affect which corrections get another chance or
|
||
what evidence the model sees. Measurements using existing observations do not
|
||
require approval or new ordinary-report schema fields.
|
||
|
||
### D7. Should this remediation add processing progress events?
|
||
|
||
**Applies to:** #57.
|
||
|
||
**Recommendation:** add a small useful processing progress stream through
|
||
CommandKit's existing reporter, driven by real stage/chunk lifecycle events.
|
||
Keep stdout for transcript output and let CommandKit own reporting-mode
|
||
precedence. This would make the inherited flag useful without inventing a second
|
||
reporting system. Exact wording and event granularity are implementation choices.
|
||
|
||
**Alternative:** retain standard flag acceptance and accurately clarify that
|
||
processing currently supplies no progress events. This is smaller and avoids
|
||
new event plumbing, but leaves the advertised capability unhelpful for long jobs.
|
||
If standard help cannot express availability, request the improvement upstream.
|
||
|
||
**Why approval:** emitting a new stream of user-facing processing information is
|
||
a feature addition, not removal of dead code. Documentation clarification can
|
||
proceed independently.
|