17 KiB
Credential-redaction removal and simplification audit
Reviewed: 2026-11-15 at a68a951; application code remains at c4ccd78.
Status: audit complete; implementation pending.
This review follows the maintainer's resolved D1 decision. It identifies machinery to remove or simplify after withdrawal of the application-wide credential-redaction requirement. Native library behavior, authentication, diagnostic evidence, and ordinary output correctness remain independently useful. The architecture policy has already been updated.
Scope and verification
The review traced credential discovery into CLI, reports, debug writers, proposal generation, validation, and repair. Knowledge-graph searches and caller traces were checked against source, including interface assertions and closures that the graph does not resolve completely. Tests were read to distinguish redaction-only expectations from authentication, serialization, artifact association, and failure behavior. Documentation and previous audit remedies were reviewed for assumptions that would reintroduce the withdrawn requirement.
This was a static review: no code changes, live provider calls, credential-value inspection, test execution, or performance measurements. Removed passes and allocations below are visible in the code; their production cost is unmeasured.
1. Remove credential discovery and tracking used only for substitution
Remove. Runtime.ConfiguredSecrets enumerates every captured profile and
backend, caches environment-variable names with sync.Once, and reads their
values whenever called. It includes raw and trimmed forms and deduplicates
additional names. Discovery uses context.WithoutCancel. RoleClient separately
collects prepared credential names under a mutex, solely to supply this method.
None of this selects or supplies authentication to a provider.
Discovery and reads ·
Runtime cache ·
Prepared-name tracking ·
Role accessor
Consumers are CLI report/debug preparation and finalization, the error redactor,
proposal and validation writer setup, and combined repair/validation secret
collection. Remove secrets.go, the runtime cache fields, role credential fields
and locking, accessor methods, and secret-source assertions after updating those
consumers. Keep OnPrepared request capture, terminal-event synchronization,
call IDs, source-path tracking, and native engine execution; those have separate
consumers. Do not leave empty secret-provider methods or no-op redactors.
CLI consumers ·
Finalization ·
Repair collection
Delete TestConfiguredSecretsIncludesUnselectedBackends. In
TestRoleClientUsesNativeTargetAndRedactsCredential, retain the assertion that
PromptKit sends the selected, trimmed Authorization value; remove the independent
redactor assertion. Keep native credential selection/optional-absence and
provider-free inspection tests. Update the credential-tracking description in
the runtime guide when removal lands.
Catalog: #63; prerequisite consumers are covered by §§2, 4–6.
2. Replace sanitizing serialization with direct encoding
Remove / simplify. The entire core/redaction package serves substitution:
raw/escaped secret expansion, longest-first matching, generic JSON decoding and
tree reconstruction, key-collision rejection, and YAML node/comment traversal.
Its JSON path marshals, decodes with UseNumber, walks, then marshals again;
its YAML path parses and re-encodes already serialized data. These passes run
even when the secret list is empty. Matching keys and fixed strings also causes
the schema corruption already recorded in catalog #9.
Redactor ·
Report YAML
Use direct JSON serialization in RunDirectory and DiagnosticsWriter, and
direct typed YAML serialization for ordinary reports. Write already-produced
YAML bytes without another parse/walk/encode pass. Remove secret parameters and
stored redactors. In Markdown, remove substitution and the redactor parameter
to writeMatrix, retaining the independent text/HTML/table escaping.
Run artifacts ·
Interaction artifacts ·
Markdown
Retain serialization errors and contextual write errors, valid artifact
envelopes, numeric and boolean types, optional fields, and raw final_content
as a string even when its contents are malformed model JSON. No nested-content
sanitizer or response-discard fallback is needed. Keep ConfigKit's own effective
configuration rendering; only Audita's subsequent YAML transformation goes away.
Move useful scalar/schema and failed-write assertions from redactor tests to the owning report/artifact tests before deleting the package. Update diagnostics/reporting internals. Catalog: #3 and #9; preserve #26's independent Markdown correction and coordinate with #33's redundant-serialization work without resolving D5.
3. Save the acquired source snapshot without reconstructing it
Simplify. WriteSourceTranscript converts the acquired bytes to a string,
decodes a generic JSON tree, then sends it through the redactor's second
serialization/decode/walk/serialization sequence. The comment explicitly ties
this reconstruction to avoiding unredacted bytes. The CLI already has both the
acquired source bytes and parsed source, and calls this writer only for debug.
Snapshot writer ·
Caller
Write the acquired bytes directly to the source artifact, preserving the input that was actually read. Keep parsed-source and normalized artifacts as separately serialized typed views. Do not reopen the input file, which could have changed, or preserve a second parser as a validation gate. Broader input-buffer lifetime work remains catalog #31's concern.
Move source/normalized snapshot coverage out of the deleted redactor package. Check source-byte fidelity at this boundary and decoded fields for the typed views; preserve nonfatal debug-write behavior. Describe the source/parsed distinction in the diagnostics guide when implemented. Catalog: #31; depends on the direct writer changes in §2.
4. Remove secret plumbing from writer and preflight contracts
Simplify. Proposal generation discovers secrets through an optional client
interface and constructs an adapter-specific writer. Validation uses a lazy
func() []string closure and recreates a writer per interaction. Repair combines
two roles' secret lists for its writer. These are extra dependencies on model
clients for a filesystem concern.
Proposal fallback ·
Validation setup ·
Validation adapter ·
Repair writer
Remove the discovery assertions, closure, combined-role helper, and secret constructor arguments. Complete the already-proposed common interaction writer contract without preserving these seams. Role owners still supply their payloads and stage/section/proposal/attempt/validator coordinates; the writer returns artifact paths and write errors. Retain the optional diagnostic-error callback and its synchronization, which report nonfatal write failures.
Also remove DebugInput.Secrets, the corresponding PrepareDebug and
NewRunDirectory parameters, and secret captures in report-rendering closures.
Preflight readiness, initial/failed report construction, output-session cleanup,
and debug setup remain useful without any credential values.
Preflight handoff ·
Debug preparation ·
Report closures
Adapt proposal, validation, repair, and preflight fixtures to the smaller contracts. Preserve role attribution, distinct repair artifact paths, and nonfatal write-failure coverage. Update the diagnostics guide with the resulting ownership. Catalog: #43, coordinated with #42 and #48; no retention or execution-policy change is implied.
5. Stop rewriting validator explanations sent to repair
Remove / simplify. Inside repairCandidates, safeFindings first constructs
domain feedback, then replaces substrings matching credentials from both repair
and validation clients. This runs for the initial rejection and subsequent
revalidation findings, independently of debug capture. It changes the next model
request, not just a saved diagnostic. A matching substring in otherwise useful
validator text therefore changes the corrective instruction.
Feedback transformation ·
Repair request
Use repairFindings directly at both points and remove the wrapper and its
secret dependency. Retain per-proposal filtering, validator/reason labels, and
the explicit unavailable-assessment representation: operational unavailability
is not an actionable editorial explanation. Preserve bounded coordination,
revalidation, withdrawal, and sibling independence.
Domain feedback
Replace TestRepairFeedbackRedactsValidationCredentials with focused coverage
of actual repair feedback if existing feedback/request tests do not already
cover it. Assert explanation and proposal association through decoded request
data, without introducing an environment-driven passthrough contract or prompt
wording checks. Update the repair-flow description if needed.
Catalog: #64; coordinate removal of repairSecrets with §4.
6. Simplify CLI presentation while retaining typed failure ownership
Remove / simplify. runProcess invokes Runtime.RedactError for preparation
failures. finishProcess unconditionally discovers credentials and builds a
redactor, even without reports or debug; it rewrites warnings, final errors, and
announced report/debug paths. Removing these calls eliminates environment reads
and permits paths to identify their actual destinations.
Preparation failure ·
Final presentation
Present those existing errors and paths directly through CommandKit, preserving
typed causes, reporting modes, completion behavior, and exit classification.
Drop redaction-only parameters such as finishProcess's loaded argument once
its only use disappears. Retain the required-output/nonfatal-debug distinction.
Simplify the path-announcement subprocess fixture; retain useful destination,
output-mode, and error assertions at their existing test owners.
There is also a distinct information-loss boundary: classifiedFailure reduces
native setup errors to a generic message, and Failure.Error() returns only
that message. Deleting substring replacement does not itself make missing
credential diagnostics actionable. Address catalog #25 by carrying the native
setup detail into the surfaced error, while retaining Class, Reason, cause
unwrapping, cancellation precedence, and operation attribution.
Classification ·
Failure wrapper
Do not indiscriminately unwrap every cause into every report. The compact
reason/message projection in processreport serves the ordinary report schema,
and structured failure types drive recoverability and peer cancellation. Retain
those purposes; revise blanket comments/tests prohibiting cause presentation
where they enforce only the withdrawn guarantee. Test setup diagnostics for
actionable context and typed classification, not exact incidental wording.
Report projection ·
Failure tests
Catalog: #65 and #25; coordinate with #22–24 and #41 without changing their classification or report-schema scope. Update operations and internal runtime documentation alongside the implementation.
7. Retire obsolete assertions and documentation without losing coverage
Remove / simplify. Redaction assertions are mixed into tests with independent value, and current documentation still describes the implemented sanitizer. Remove pure substitution tests and synthetic credential-echo fixture modes used only for them. In mixed tests, preserve native authentication, decoded request and response evidence, schema metadata, repair path uniqueness, scalar types, failure counts, and optional-artifact failure behavior.
| Location | Treatment when implementation lands |
|---|---|
core/redaction/redaction_test.go, llm/secrets_test.go |
Delete removed-mechanism tests; relocate useful writer assertions as described above. |
llm/diagnostics_test.go, proposal_generation/generate_test.go, runner/runner_test.go |
Retain artifact contents/associations and diagnostic failure coverage; remove secret providers and masking expectations. |
runner/repair_test.go |
Keep unique branch/attempt artifacts and repair behavior; remove secret plumbing and rewrite the feedback test at its domain boundary. |
core/reporting/report_test.go, llm/native_adapter_test.go |
Retain typed report encoding and actual native authentication; remove attached substitution assertions. |
cmd/audita/main_integration_test.go |
Retire the padded-credential masking scenario and secret-based path fixture; preserve representative malformed-response evidence, outcomes, and destination announcements. |
internal/testsupport/files.go |
Remove the already-unused secret scanners with catalog #61, rather than repurposing them. |
Review operations, diagnostics/reporting, runtime, contributor recipes, component overview, and the migration guide when code lands. Remove broad redaction claims and obsolete implementation instructions. Preserve native configuration inspection/redaction documentation and the effective-config artifact name; neither requires a naming migration merely because local postprocessing goes away. Historical audit evidence stays intact, with revised remedies in the catalog and assessment. Do not add a negative redaction policy.
Catalog: #3, #60, and #61, and supporting work for all findings above. No new broad secret scanner or prompt-language change detector should replace the removed tests.
Retained responsibilities and boundaries
- Native authentication and inspection: PromptKit still owns selected
credentials and transport; ConfigKit/ckconfig/ckprompt own their normal
rendering and inspection. Keep Audita's
WriteEffectivecall and standard command wiring. Prepared snapshots contain messages and prompt metadata; removing substitution does not require adding headers or environment dumps. Configuration rendering · Command wiring · Prepared snapshot - Ordinary filesystem behavior: retain unique run-directory creation, run-ID validation, stage-name-to-filename conversion, required-output staging, destination checks, cleanup, and useful write errors. These prevent collisions or incorrect writes. No extra private-root or hostile-filesystem mechanism was found to remove in the debug writer. Catalog #4's previously proposed private defaults lack a requirement under D1; leave current location/modes unchanged in this cleanup. Catalog #1's ordinary wrong-file defects remain.
- Domain and report behavior: retain transcript parsing, validation, feedback classification, typed operation failures, stable report identities, Markdown escaping, and nonfatal optional diagnostics. Removing credential substitution does not imply dropping these boundaries.
Suggested implementation order
- Replace serialization and source-snapshot reconstruction, removing writer and report secret arguments together with their callers (§§2–4).
- Remove repair-feedback and CLI substitution, then delete now-unreferenced discovery state and the redaction package (§§1, 5–6). These are a coordinated cleanup, not an interim design with empty secret lists or no-op methods.
- Complete common interaction-writer ownership and actionable setup diagnostics with catalog #43/#25. Update affected tests and canonical documentation in each implementing change (§7).
Use targeted boundary tests and then the full suite for the cross-package code change. Check decoded artifact fidelity, raw source preservation, native auth, repair feedback, report types, and failure behavior. No new product decision is needed for these D1 removals; D2–D7 remain separate and unresolved.