255 lines
16 KiB
Markdown
255 lines
16 KiB
Markdown
# Security audit
|
|
|
|
Reviewed 2026-11-15 at `27e58fa`. The review covered application and prompt
|
|
configuration, transcript and glossary input, model transport and responses,
|
|
credentials, output publication, diagnostic serialization, and release
|
|
delivery. Findings identify the deployment conditions that make each issue
|
|
relevant.
|
|
|
|
## Architecture and trust boundaries
|
|
|
|
Audita is a single-process CLI. The invoking account supplies filesystem
|
|
authority and chooses the configuration and destinations. ConfigKit captures
|
|
application sources and PromptKit captures prompt and provider configuration.
|
|
The transcript, glossary text, and provider responses are data. Audita renders
|
|
model requests for selected roles, validates model responses structurally and
|
|
through domain checks, and publishes transcript files and reports. The OS or
|
|
subprocess parent controls who may invoke the CLI and under which account.
|
|
Provider authentication uses configured environment variables, with bearer
|
|
tokens added by the transport. Repair may intentionally send the original
|
|
proposal exchange and branch history to a separately selected provider.
|
|
[Application structure](../../internal/overview.md)
|
|
· [Trust policy](../../policy/architecture.md#trust-boundaries-and-threat-scope)
|
|
· [Configuration sources](../../config.md#loading-and-composition)
|
|
· [Model integration](../../integrations/openai-compatible-llm.md)
|
|
· [Repair data flow](../../operations.md#processing)
|
|
|
|
The audited input-to-model path reads transcript text as data, uses structured
|
|
schemas, checks candidate identity and applicable validators, and rechecks
|
|
protected terms when applying an approved edit. The embedded prompts tell
|
|
models to treat transcript text and repair evidence as untrusted data. Those
|
|
instructions guide editorial behavior. The runner maps model responses to
|
|
domain proposals and applies only approved edits. PromptKit validates the
|
|
initial endpoint, bounds per-response reads, and closes response bodies. For
|
|
host-backed prompt content files, PromptKit resolves the path relative to
|
|
captured provenance and the pinned ConfigKit opens it through a rooted
|
|
filesystem.
|
|
[Prompt data instructions](../../../internal/prompts/assets/shared/prompt_hardening.md)
|
|
· [Repair instructions](../../../internal/prompts/assets/shared/repair_contract.md)
|
|
· [Proposal application checks](../../../internal/framework/proposals/apply.go#L59)
|
|
· [Model and configuration versions](../../../go.mod#L9)
|
|
· [Transport limits reviewed previously](05-resources.md#architecture-and-lifecycle)
|
|
|
|
## Findings
|
|
|
|
### 1. Debug capture uses a shared path and can expose private artifacts
|
|
|
|
Debug is opt-in, and its default parent is `/tmp/audita`. `NewRunDirectory`
|
|
accepts an existing parent and creates the per-run directory with mode `0755`
|
|
before applying the process umask. Snapshot and interaction files use mode
|
|
`0644`. A normal run using a synthetic transcript and umask `022` produced
|
|
exactly those directory and file modes inside a disposable private parent.
|
|
Under a shared, traversable parent, those permissions let other local users
|
|
read the transcript snapshots, prompts, and responses. Even when credential
|
|
redaction succeeds, these artifacts contain confidential data for some runs.
|
|
[Debug default](../../../internal/core/config/config.go#L106)
|
|
· [Run directory creation](../../../internal/core/diagnostics/run_dir.go#L35)
|
|
· [Source snapshot](../../../internal/core/diagnostics/run_dir.go#L97)
|
|
· [Interaction permissions](../../../internal/framework/llm/diagnostics.go#L84)
|
|
· [Documented debug contents](../../operations.md#debug-evidence)
|
|
|
|
The predictable default parent is also a trust boundary. If it already
|
|
resolves to a directory writable by a different account, Audita accepts that
|
|
directory and later opens artifacts by path. Another account with directory
|
|
mutation rights can interfere with where evidence is written or with its
|
|
integrity. The random run ID helps avoid collisions. Directory ownership and
|
|
permissions need their own checks.
|
|
[Directory acquisition](../../../internal/core/diagnostics/run_dir.go#L43)
|
|
· [Run ID generation](../../../internal/cli/process_inputs.go#L142)
|
|
|
|
Use a per-user location for the implicit debug root and create private run
|
|
directories and files by default. If a caller explicitly selects a shared
|
|
location, document the ownership and permission assumptions. For directories
|
|
that can change concurrently, keep writes scoped to the acquired directory.
|
|
|
|
### 2. Provider redirects can cross the configured transport boundary
|
|
|
|
The process command gives PromptKit no custom HTTP client. PromptKit v0.23.0
|
|
constructs a Go client without a redirect policy and creates its generation
|
|
POST from a replayable byte reader. The Go client's default policy can follow
|
|
body-preserving redirects across origins. Its rules for forwarding authorization
|
|
headers operate independently of the body and use host trust rules rather than
|
|
the complete origin. A redirect can therefore replay transcript-bearing requests
|
|
to a different service. A downgrade on the same host can also retain the
|
|
authorization header while losing TLS protection.
|
|
[CLI transport selection](../../../internal/cli/process_inputs.go#L42)
|
|
· [PromptKit client construction](../../../internal/framework/llm/runtime.go#L103)
|
|
· [Pinned version](../../../go.mod#L11)
|
|
|
|
This matters when a provider gateway responds with a redirect across a routing
|
|
or TLS boundary that the operator has not authorized for transcript data. The
|
|
configured destination can be trusted while the destination of the redirect
|
|
has a different audience or transport security. The dependency paths are
|
|
`internal/llm/openai_compatible_client.go` and `internal/llm/http_evidence.go`,
|
|
with redirect and header rules in Go's `net/http/client.go`.
|
|
|
|
Consider a provider redirect contract that keeps generation on the selected
|
|
origin by default and rejects HTTPS downgrades. Where redirects are needed,
|
|
support explicit trusted destinations and define when credentials may be
|
|
forwarded. Keep the documented ability to deliberately select a local HTTP
|
|
provider separate from protocol changes during a request.
|
|
[Provider contract](../../integrations/openai-compatible-llm.md)
|
|
· [Local HTTP example](../../../examples/local/prompts.yml#L3)
|
|
|
|
### 3. Report destinations can escape the identity checked by preflight
|
|
|
|
Preflight checks report paths for collisions with input, configuration, and
|
|
other destinations. Each report replacement resolves the original path again
|
|
at publication. The concurrency audit verified a change of report directory
|
|
referent between initialization and finalization. If a subprocess processes
|
|
work in a directory whose entries can be changed by a lower-trust account, the
|
|
same behavior can direct a report write to an unintended file writable by the
|
|
Audita account, including a protected input or configuration file whose earlier
|
|
collision check passed. A private staging file protects incomplete bytes. The
|
|
final rename still uses the later resolution.
|
|
[Destination selection](../../../internal/core/diagnostics/output_session.go#L60)
|
|
· [Report finalization](../../../internal/core/diagnostics/output_session.go#L273)
|
|
· [Path resolution and rename](../../../internal/core/diagnostics/output_session.go#L410)
|
|
· [Earlier referent-change evidence](04-concurrency.md#6-report-paths-can-change-referent-between-initialization-and-finalization)
|
|
|
|
Bind publication to the directory and destination semantics checked at
|
|
preflight. For shared output directories, use directory-relative operations or
|
|
reject an unsafe change before writing. Preserve intentional in-place
|
|
replacement and the selected support for output symlinks. Use an
|
|
invocation-owned output directory when a subprocess parent handles
|
|
lower-trust files.
|
|
[Output lifecycle](../../operations.md#outputs-and-lifecycle)
|
|
|
|
### 4. Raw structured debug responses can retain an encoded credential
|
|
|
|
PromptKit returns the model message as `RawOutput`. The role adapter retains
|
|
that string as `FinalContent`, and debug writers store it beside the decoded
|
|
domain result. The artifact redactor parses the outer JSON artifact and
|
|
matches each string against the exact secret value and the escaped spelling
|
|
from Go's JSON encoder. It treats `final_content` as one string, even when that
|
|
field contains another JSON document. A credential present in a provider
|
|
response can therefore be masked in decoded fields but remain recoverable from
|
|
the raw response field when the provider used a different valid representation
|
|
of the same JSON string. The existing nested-content test covers the spelling
|
|
produced by Go's encoder.
|
|
[Raw output retention](../../../internal/framework/llm/adapter.go#L171)
|
|
· [Proposal artifact](../../../internal/framework/proposal_generation/generate.go#L158)
|
|
· [Validator artifact](../../../internal/framework/validators/llm_validators.go#L148)
|
|
· [Repair artifact](../../../internal/framework/runner/repair.go#L173)
|
|
· [Redactor](../../../internal/core/redaction/redaction.go#L18)
|
|
· [Nested JSON test](../../../internal/core/redaction/redaction_test.go#L43)
|
|
|
|
The condition is a provider returning credential-bearing content and an
|
|
operator retaining debug data. A gateway that echoes authentication material
|
|
into diagnostic model content can cause that exposure even if its JSON
|
|
encoder and Audita's differ only in escaping. Sharing the affected artifact
|
|
can disclose a reusable credential. Redact structured response content after
|
|
decoding it as JSON, or retain only the sanitized decoded view. If malformed
|
|
raw response text is useful for diagnosis, define how to retain it without
|
|
claiming decoded credential redaction for content that could not be interpreted.
|
|
[Credential handling requirement](../../policy/architecture.md#trust-boundaries-and-threat-scope)
|
|
|
|
### 5. Configuration discovery extends trust to the main file and its neighbors
|
|
|
|
Without `--config`, the working directory is searched before user and system
|
|
configuration. Once a main source is selected, conventional prompt and
|
|
glossary companions beside it are loaded as configuration. An explicit main
|
|
file still enables its companions. This is useful for an operator-managed
|
|
project, but it changes the trust boundary when a subprocess parent uses an
|
|
upload-controlled working directory or places trusted configuration beside
|
|
untrusted project files. Configuration discovered there can choose the model
|
|
destination and a credential variable present in the child process. In that
|
|
deployment, both text and a credential can be sent outside the destination
|
|
the integrator intended. A transcript passed as a data file to a trusted
|
|
working directory with a trusted configuration bundle does not carry those
|
|
configuration privileges.
|
|
[Application discovery](../../../internal/cli/app_wiring.go#L30)
|
|
· [Prompt imports](../../../internal/framework/llm/runtime.go#L36)
|
|
· [Captured main filesystem context](../../../internal/framework/llm/runtime.go#L75)
|
|
· [Subprocess guidance](../../integrations/subprocess.md#invocation)
|
|
· [Configuration composition](../../config.md#loading-and-composition)
|
|
|
|
Document that the selected main file, its automatic companions, and each
|
|
searched working directory are part of the trusted operator configuration.
|
|
For an integration accepting uploads, use a trusted bundle and pass the
|
|
transcript independently. Give the child process only credentials appropriate
|
|
to the provider choices authorized for that integration. An explicit discovery
|
|
option could support deployments that need a single configuration file.
|
|
|
|
### 6. Untrusted transcript filenames can become active Markdown
|
|
|
|
The CLI puts the input path in the report snapshot. The Markdown renderer
|
|
escapes HTML, table separators, backticks, and line breaks for path fields.
|
|
Markdown link and image punctuation still reaches the renderer as syntax. The
|
|
stage-label renderer already escapes those constructs. If a subprocess keeps
|
|
an uploader-controlled filename and displays rendered `report.md`, the name
|
|
can render as a link with a label that obscures the actual path. That changes
|
|
what the operator sees and can direct navigation to an unexpected destination.
|
|
The concern is the integrity of the rendered report.
|
|
[Path in the snapshot](../../../internal/cli/process_output.go#L24)
|
|
· [Path rendering](../../../internal/core/reporting/markdown.go#L41)
|
|
· [Path escaping](../../../internal/core/reporting/markdown.go#L191)
|
|
· [Stage label escaping](../../../internal/core/reporting/markdown.go#L132)
|
|
· [Existing stage-label test](../../../internal/core/reporting/markdown_test.go#L101)
|
|
|
|
Render paths as literal text with a context-appropriate Markdown encoder.
|
|
Keep exact path values in structured artifacts when the output contract calls
|
|
for them.
|
|
|
|
### 7. Untrusted input size and blocking I/O can exhaust subprocess workers
|
|
|
|
The transcript is read whole with `os.ReadFile`, then decoded and re-encoded
|
|
before chunking. Backend admission and per-response byte limits apply later.
|
|
For a parent accepting untrusted uploads without an input or process memory
|
|
budget, a sufficiently large valid transcript can exhaust the child or a
|
|
shared worker host before model calls are reached. A local integration that
|
|
allows input paths to resolve to a FIFO can also lose a worker to a read that
|
|
outlives cooperative cancellation. The size and cancellation paths were traced
|
|
in the resource audit, and the FIFO behavior was verified in the concurrency
|
|
audit.
|
|
[Input acquisition](../../../internal/cli/process_inputs.go#L46)
|
|
· [Whole-file reader](../../../internal/core/io/files.go#L9)
|
|
· [JSON acquisition and conversion](../../../internal/core/schema/transcript.go#L35)
|
|
· [Memory finding](05-resources.md#4-whole-transcript-copies-amplify-memory-at-input-and-publication)
|
|
· [FIFO finding](04-concurrency.md#2-waiting-for-transcript-input-can-prevent-shutdown-before-preflight)
|
|
|
|
Keep the CLI suitable for trusted large jobs by avoiding redundant whole-input
|
|
copies. Integrations accepting untrusted input can enforce their own upload and
|
|
process budgets and require input from controlled storage. An application-level
|
|
input budget should be an explicit deployment choice. Where stream inputs are
|
|
supported, make their acquisition and reads cancellable.
|
|
[Subprocess timeout guidance](../../integrations/subprocess.md#invocation)
|
|
|
|
## Release boundary and verification
|
|
|
|
Release CI verifies the event commit and runs the tagged ReleaseKit version with
|
|
a delivery credential. The checked-in procedure relies on the forge and CI
|
|
operator to authorize who can create release tags and receive the delivery
|
|
secret. The local review covered the workflow, tag/commit checks, pinned
|
|
executable selection, and shell quoting. Forge permissions and running CI
|
|
secret settings were outside the observed state. The earlier bootstrap status
|
|
error is covered by ReleaseKit's independent prepared-checkout validation
|
|
before delivery.
|
|
[CI workflow](../../../.woodpecker/release.yml#L5)
|
|
· [Publication guards](../../policy/release.md#publication-guards)
|
|
· [Delivery procedure](../../release.md#binary-delivery)
|
|
· [Earlier bootstrap finding](06-error-handling.md#7-ci-bootstrap-treats-a-failed-git-status-command-as-clean-if-stdout-is-empty)
|
|
|
|
- The complete offline `go test -count=1 ./...` suite passed on Go 1.27.3.
|
|
Existing tests exercise credential handling, redaction, filesystem collision
|
|
checks, cancellation, and model-validation boundaries.
|
|
[Credential redaction test](../../../cmd/audita/main_integration_test.go#L965)
|
|
· [Collision check test](../../../internal/core/diagnostics/output_session_test.go#L14)
|
|
· [Validation and transport test](../../../internal/framework/validators/llm_integration_test.go#L25)
|
|
- `go mod verify` verified cached module content. The pinned dependency source
|
|
was reviewed at the boundaries above. The checksum check does not establish
|
|
advisory status.
|
|
- A normal synthetic debug run substantiated the artifact modes. Other findings
|
|
are grounded in the cited source and contracts or in evidence already recorded
|
|
by the earlier audits. The review used local code and files throughout.
|