Withdraw application-wide credential redaction policy
This commit is contained in:
@@ -1,13 +1,15 @@
|
||||
# Audit assessment and decision register
|
||||
|
||||
**Reviewed baseline:** `faba8ab` (2026-11-15). **Status:** assessment complete;
|
||||
remediation not implemented. This document owns classification, recommended
|
||||
remediation not implemented. D1 was resolved on 2026-11-15 and the architecture
|
||||
policy updated; its implementation cleanup awaits the focused audit below.
|
||||
This document owns classification, recommended
|
||||
scope, and pending decisions for the [catalog](catalog.md). The source audits
|
||||
retain their evidence; their suggested remedies are not approved requirements.
|
||||
|
||||
The catalog covers 103 source findings in 62 entries. Every catalog entry is
|
||||
assessed below. The source tree is unchanged since the audited implementation
|
||||
at `c4ccd78`; subsequent commits added audit documentation. This review used
|
||||
assessed below. The application implementation remains at the audited baseline
|
||||
`c4ccd78`; later documentation includes the D1 policy revision. This review used
|
||||
the source audits, current contracts and ADRs, and targeted implementation
|
||||
inspection. It did not independently rerun all audit probes, benchmarks, or the
|
||||
test suite. Reproductions reported by the audits remain attributed to them;
|
||||
@@ -33,8 +35,8 @@ when substantial restructuring is appropriate.
|
||||
|
||||
**Ready** means suitable for implementation planning within the stated scope,
|
||||
not implemented or validated by this document. **Investigate** denotes technical
|
||||
work the agent can undertake without a product decision. **D1–D7** identify
|
||||
pending substantive choices; recommendations are not decisions. **Closed**
|
||||
work the agent can undertake without a product decision. **D2–D7** identify
|
||||
pending substantive choices; D1 is recorded below as resolved. **Closed**
|
||||
denotes no remediation for the specified scope. Mixed entries deliberately
|
||||
separate ready work from decisions and exclusions.
|
||||
|
||||
@@ -66,13 +68,13 @@ behavior from proposed extensions.
|
||||
| --- | --- | --- | --- |
|
||||
| [1. Bind output writes to the checked destinations](catalog.md#1-bind-output-writes-to-the-checked-destinations) | C, A | Ready | The wrong-file reproduction and repeated report-path resolution substantiate application defects. Resolve paths with filesystem semantics, carry checked parent/destination information through publication, and share staging ownership. Preserve symlinks and intentional in-place replacement. No hostile-directory race-proof guarantee, cross-file transaction, or custom filesystem machinery is selected. |
|
||||
| [2. Keep model requests within the configured transport boundary](catalog.md#2-keep-model-requests-within-the-configured-transport-boundary) | — | Closed: WONTFIX | Accepted Go HTTP redirect and credential-forwarding behavior. The source audit's proposed redirect restrictions do not authorize implementation. |
|
||||
| [3. Protect credentials inside nested debug responses](catalog.md#3-protect-credentials-inside-nested-debug-responses) | C; P | Ready in part; D1 | Nested JSON can retain an encoded credential after outer redaction. Sanitize known structured-response fields after decoding and verify decoded artifacts. Raw malformed-body retention is a separate policy choice in D1; fixing valid nested content does not depend on it. |
|
||||
| [4. Make debug artifact permissions private by default](catalog.md#4-make-debug-artifact-permissions-private-by-default) | C, M | Ready | The observed public directory/file modes expose confidential debug content. Use private creation modes and a secure per-user implicit root with standard facilities; retain explicit work-directory selection and nonfatal debug setup errors. Exact default-path selection is engineering work, with an operator documentation update. Do not change permissions on unrelated existing directories or add a hostile-filesystem guarantee. |
|
||||
| [3. Protect credentials inside nested debug responses](catalog.md#3-protect-credentials-inside-nested-debug-responses) | A, M, T | Superseded remedy; D1 cleanup audit | The encoded-secret observation tested the withdrawn redaction guarantee. Do not implement nested sanitization or discard malformed response bodies for that purpose. Audit and remove local credential discovery, sanitization, and their supporting abstractions and tests; retain useful diagnostic evidence and native upstream behavior. |
|
||||
| [4. Make debug artifact permissions private by default](catalog.md#4-make-debug-artifact-permissions-private-by-default) | M; P | Reassess under D1 scope | The permission observation is factual, but hostile local filesystem access is outside the maintainer's threat model and operators own artifact access. The prior mandatory private-root/security remediation is withdrawn. The follow-up audit should distinguish ordinary file-creation mechanics from machinery justified only by local-adversary protection; any remaining default-location or permission proposal needs its own justification. |
|
||||
| [5. Define the trust boundary for discovered configuration](catalog.md#5-define-the-trust-boundary-for-discovered-configuration) | M; P | Ready for documentation; extension excluded | Automatic companions are intentional configuration composition, not a discovery bug. Explain that the searched locations, selected bundle, imports, and companions are trusted configuration, and keep uploaded transcripts separate. Disabling discovery or adding a new isolation mode is outside this remediation; no ConfigKit replacement loader. |
|
||||
| [6. Make blocking transcript input and delivery cancellation-aware](catalog.md#6-make-blocking-transcript-input-and-delivery-cancellation-aware) | C; — | Ready in part; WONTFIX in part | Honor cancellation before regular-file publication and when blocked delivery returns; produce failure status even if bytes have escaped. Do not promise interruption of already-blocked standard I/O. Do not ban FIFOs, add detached writer goroutines, or revive the source audits' context-aware stream machinery. |
|
||||
| [7. Publish truthful failure reports and preserve initialized evidence](catalog.md#7-publish-truthful-failure-reports-and-preserve-initialized-evidence) | C, A, T | Ready | Publishing completed reports after a known delivery failure contradicts their meaning. Select failed status before the first report write; preserve initialized evidence if rendering fails. Carry rendering errors explicitly and retain bounded best-effort finalization. Cross-file atomicity remains outside the contract. |
|
||||
| [8. Prevent concurrent runs from clobbering ordinary reports](catalog.md#8-prevent-concurrent-runs-from-clobbering-ordinary-reports) | M; P | Ready for documentation; D2 | The collision is demonstrated, but independent concurrent ownership of fixed report basenames is not promised. Document per-invocation directory selection now. Automatic run-specific placement, locking, or rejection of competing writers changes the workflow and belongs to D2. |
|
||||
| [9. Preserve the ordinary YAML schema during redaction](catalog.md#9-preserve-the-ordinary-yaml-schema-during-redaction) | C | Ready | Redacting an application-owned key or status breaks the report schema. Redact input-derived fields before serialization; retain fixed keys, enums, and reason codes. A literal coincidence between a credential and a fixed schema word does not make that word credential-derived data. |
|
||||
| [9. Preserve the ordinary YAML schema during redaction](catalog.md#9-preserve-the-ordinary-yaml-schema-during-redaction) | C, A, M | Ready through D1 cleanup | Schema corruption is real, but preserving the redactor is no longer the remedy. Remove application credential substitution from ordinary report serialization and preserve the typed schema. Retain meaningful output-schema tests; retire tests whose sole purpose is satisfying the withdrawn redaction promise. |
|
||||
| [10. Reject contradictory validation decisions](catalog.md#10-reject-contradictory-validation-decisions) | C, A | Ready | Approved-plus-unavailable contradicts the existing acceptance invariant, although bundled producers are consistent. Establish an authoritative internal disposition and reject invalid internal results before use; derive redundant serialized approval as needed. This is an internal contract failure, distinct from an unavailable model assessment. |
|
||||
| [11. Preserve usable validator decisions when extra references cannot be decoded](catalog.md#11-preserve-usable-validator-decisions-when-extra-references-cannot-be-decoded) | C, T | Ready | The mixed-response probe violates existing sibling-preservation rules. Retain exact numeric references until range reconciliation, accepting integral JSON number spellings without float rounding. Ignore extras before conversion to local indices. Preserve whole-response structural validation; do not add semantic retries or general malformed-response salvage. |
|
||||
| [12. Use full-segment evidence for protected-name checks](catalog.md#12-use-full-segment-evidence-for-protected-name-checks) | C, A, T | Ready | The full-segment examples establish a mismatch with existing protection intent. Share glossary-owned assessment using usable before/after segment evidence; recheck current text at application. Preserve active-glossary exemptions and spoken-word repetition rules. Missing or ambiguous previews cannot justify a fabricated protection verdict; retain independently supported applicability findings. Empty-result previews can still provide evidence. |
|
||||
@@ -106,7 +108,7 @@ behavior from proposed extensions.
|
||||
| [40. Use one validator contract and one owner for decision semantics](catalog.md#40-use-one-validator-contract-and-one-owner-for-decision-semantics) | A, C, M | Ready | Duplicate ports and runner-specific verdict rewriting obscure one existing validation contract. Consolidate types, applicability, and decision meaning with their owner; remove unused envelope fields. Preserve zero-budget shortlisting, independent repair feedback, cardinality, and unavailable outcomes. No broad plugin framework or new validator policy is needed. |
|
||||
| [41. Carry structured failure facts through model operations and reporting](catalog.md#41-carry-structured-failure-facts-through-model-operations-and-reporting) | A, C | Ready | Failure class and reason already exist but are discarded and reconstructed. Carry typed facts through role-specific handling and report counts. Use a consistent internal-failure reason fallback for missing reasons while retaining the original failure class and cause. Avoid reclassifying an operation from its display reason. |
|
||||
| [42. Keep required output publication separate from optional debug retention](catalog.md#42-keep-required-output-publication-separate-from-optional-debug-retention) | A, C | Ready | Required publication and optional retained evidence have distinct lifecycles. Separate their APIs and share regular-file staging where behavior is common. Preserve transcript-before-report ordering, nontransactional finalization, private staging, and nonfatal debug errors. Coordinate with #1, #7, #16, and #22. |
|
||||
| [43. Inject one interaction diagnostics contract](catalog.md#43-inject-one-interaction-diagnostics-contract) | A, C | Ready | Equivalent writer contracts and concrete filesystem/credential wiring cross role boundaries. Inject one diagnostics sink, retaining role-owned semantic payloads, lazy redaction inputs, combined repair/validation secrets, and nonfatal write warnings. D1 and D5 constrain only changes to what is retained. |
|
||||
| [43. Inject one interaction diagnostics contract](catalog.md#43-inject-one-interaction-diagnostics-contract) | A, M | Ready; simplify after D1 audit | Common interaction writing still has an ownership benefit, but the writer should not acquire redaction-only contracts, credential discovery, or combined-role secret plumbing. Audit those paths before choosing the smallest remaining writer interface. Preserve role-owned evidence, artifact association, and nonfatal write failures; D5 still governs retention changes. |
|
||||
| [44. Simplify structured-call completion and remove obsolete record fields](catalog.md#44-simplify-structured-call-completion-and-remove-obsolete-record-fields) | A, M, T | Ready | Ordinary calls and coordinator turns share completion responsibilities; obsolete payload copies have no production reader. Use explicit once-only completion and remove superseded state after moving meaningful assertions. Preserve native terminal counts, typed causes, and undelivered diagnostic evidence. |
|
||||
| [45. Retire the test-only direct continuation branch](catalog.md#45-retire-the-test-only-direct-continuation-branch) | M, T | Ready | Direct continuation is test-only; production uses coordination. Move sibling-isolation and structural-repair coverage onto the coordinator before removing the branch. Removing the redundant adapter surface does not remove PromptKit capability or production repair. |
|
||||
| [46. Use a single proposal handoff and test real runner indexing](catalog.md#46-use-a-single-proposal-handoff-and-test-real-runner-indexing) | A, E, M, T | Ready | Generator enrichment is discarded and real indices are assigned by the runner. Keep one proposal handoff with original local references, and assert actual deterministic runner indices under observed out-of-order completion. Review emitted diagnostic start_index separately before dropping it; move useful tests before removing their harness. |
|
||||
@@ -133,7 +135,8 @@ Group related work by owner rather than implementing catalog numbers as 62
|
||||
independent tasks. Fixes and their supporting ownership changes belong together:
|
||||
|
||||
- Output publication and diagnostic integrity: #1, #3–9, #16, #22, #26,
|
||||
#42–43, with D1/D2/D5 limited to their policy portions.
|
||||
#42–43, incorporating the resolved D1 cleanup and keeping D2/D5 policy
|
||||
portions separate.
|
||||
- Transcript and glossary correctness: #11–13, #17–21, #35–39, #51–54.
|
||||
- Runner and model-operation lifecycle: #10, #14–15, #23–25, #27, #30,
|
||||
#32, #40–41, #44–49, #55–56.
|
||||
@@ -159,34 +162,62 @@ correctness/engineering work. They are not additional open product decisions.
|
||||
Independent whole-pipeline chunk traversal and semantic recovery of unavailable
|
||||
validator responses remain [separate deferred features](../future.md).
|
||||
|
||||
## Resolved D1: withdraw application-wide credential redaction
|
||||
|
||||
**Accepted:** 2026-11-15. **Policy update:** complete. **Implementation:** pending.
|
||||
|
||||
The maintainer withdrew the application-wide credential-redaction requirement
|
||||
and approved removing machinery whose sole purpose is serving it, plus
|
||||
simplifying shared machinery made more complex by it. The architecture policy
|
||||
was updated by deleting the requirement, without adding a negative assertion.
|
||||
Upstream libraries' native credential/configuration behavior remains in place.
|
||||
The earlier proposed response-omission policy is superseded; debug evidence,
|
||||
including malformed responses, remains useful to retain.
|
||||
|
||||
The maintainer's stated scope places artifact access and sharing with the
|
||||
operator and excludes hostile actors with local filesystem access. This changes
|
||||
the premises of #3 and #4 and the remedies for #9 and #43. Ordinary publication
|
||||
correctness, authentication, and required output behavior still have independent
|
||||
purposes. The remaining six decisions retain their IDs.
|
||||
|
||||
### Focused follow-up audit
|
||||
|
||||
Perform a read-only audit before implementation, tracing both callers and data
|
||||
flow. Its deliverable is an inventory of **remove**, **simplify**, and **retain**
|
||||
items, with locations, remaining consumers, rationale, affected tests/docs, and
|
||||
dependencies on catalog entries. It should cover:
|
||||
|
||||
1. Credential discovery performed solely for redaction, including profile/backend
|
||||
enumeration, environment reads, caches, role tracking, and secret-provider
|
||||
interfaces. Separate this from credentials needed by native authentication.
|
||||
2. Generic redactors, encoded-secret handling, serialization round trips,
|
||||
sanitizing copies, and fallback/error paths in report and artifact writers.
|
||||
3. Redaction-related parameters, callbacks, adapters, and data structures across
|
||||
proposal, validation, repair, CLI output, and execution reporting. Simplify
|
||||
remaining responsibilities instead of replacing removed code with no-op seams.
|
||||
4. Error presentation that hides actionable native information only to satisfy
|
||||
the old guarantee. Preserve useful error classifications, attribution, and
|
||||
stable output contracts while examining whether messages can be clearer.
|
||||
5. Tests, fixtures, documentation, comments, and outstanding audit recommendations
|
||||
that perpetuate the withdrawn requirement. Retain independent coverage of
|
||||
serialization, evidence fidelity, output failures, and upstream integration.
|
||||
|
||||
For each item, establish whether it serves an independent application requirement
|
||||
or is supplied natively upstream. Preserve actual authentication and the standard
|
||||
inspection surfaces. Evaluate implementation complexity and data loss, not merely
|
||||
occurrences of the word "redaction." Do not execute live providers or print real
|
||||
credential values during this audit.
|
||||
|
||||
Use the inventory to plan coherent removal/refactoring units. Update implemented
|
||||
behavior documentation alongside those code changes, so documentation does not
|
||||
claim cleanup has already landed. The source audit files remain historical
|
||||
evidence; this assessment controls the revised remediation scope.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
All seven decisions below are pending. Accepting or rejecting an optional
|
||||
The six decisions D2–D7 below remain pending. Accepting or rejecting an optional
|
||||
extension must not delay the independent ready portions of its catalog entries.
|
||||
|
||||
### D1. What evidence should remain when a raw model response cannot be safely decoded?
|
||||
|
||||
**Applies to:** #3; implemented through the writer ownership in #43.
|
||||
|
||||
**Recommendation:** retain safe interaction metadata and an explicit omission
|
||||
marker, but omit an undecodable raw response body. Continue retaining sanitized
|
||||
structured content when decoding succeeds. This preserves useful diagnosis
|
||||
without pretending that exact-string substitution covers every representation
|
||||
of a secret in malformed content. Optional evidence loss remains a warning,
|
||||
not a processing failure.
|
||||
|
||||
**Alternative:** omit the entire affected response artifact and report a safe
|
||||
warning, retaining its operation metadata elsewhere. This is simpler and avoids
|
||||
partial artifact interpretation, but loses more useful evidence and associations.
|
||||
A richer malformed-text sanitizer could preserve more content, but needs a
|
||||
bounded, demonstrable redaction contract and additional machinery; unconditional
|
||||
raw retention would relax the existing secret-protection requirement and is not
|
||||
recommended.
|
||||
|
||||
**Why approval:** this deliberately reduces retained evidence for a class of
|
||||
responses. Decoded nested-JSON redaction and preservation of ordinary report
|
||||
schemas remain ready fixes regardless of this choice.
|
||||
|
||||
### D2. Should Audita manage concurrent ownership of ordinary report destinations?
|
||||
|
||||
**Applies to:** #8.
|
||||
@@ -263,8 +294,8 @@ Replacing self-contained interaction artifacts with shared-history references
|
||||
also needs a deliberate consumer contract.
|
||||
|
||||
**Why approval:** these policies change what operators receive from debug mode.
|
||||
Private permissions/default-root safety in #4 and elimination of duplicate
|
||||
in-memory representations are ready improvements without these policies.
|
||||
Elimination of duplicate in-memory representations remains independent.
|
||||
The permissions/default-root proposal in #4 is being reassessed under D1.
|
||||
|
||||
### D6. Should correction repair gain aggregate budgets or reduced conversation context?
|
||||
|
||||
|
||||
Reference in New Issue
Block a user