739 lines
36 KiB
Markdown
739 lines
36 KiB
Markdown
# Library Adoption Review Findings
|
||
|
||
**Reviewed source:** `2844ce7e6a6b881bd4658577fd1177eae729f04d`.
|
||
**Scope:** All 15 stages in the [implementation plan](implementation.md),
|
||
compared with the [adoption roadmap](library-adoption.md) and pre-sprint
|
||
revision `40420a2`.
|
||
**Status:** Open.
|
||
|
||
The follow-up integration audit compared the two binaries against a loopback
|
||
provider. A normal grammar correction produced the same transcript and
|
||
completed both validators on each revision. Differences at the output,
|
||
credential, and glossary boundaries are recorded in R10–R12.
|
||
|
||
The algorithmic audit traced the implementation and its pinned library calls
|
||
without performance measurements. R13–R20 identify avoidable work within the
|
||
adopted design. Fixed transcript chunks, ordered stages, native provider
|
||
admission, glossary selection and matching rules, and report schemas remain
|
||
the basis of the analysis.
|
||
|
||
## Architecture Assessment
|
||
|
||
ConfigKit selects and composes the v3 application configuration, including
|
||
main-relative glossary sources and overlays, and records root and stage
|
||
resolution. PromptKit loads its bundled catalog, embedded Audita prompts, and
|
||
operator resources using the selected main's context. One engine supplies
|
||
stage-specific proposal and validation clients and owns provider admission.
|
||
CommandKit provides routing, reporting modes, and the standard configuration,
|
||
resource, request, and smoke commands.
|
||
|
||
Audita resolves independent context and protection views for each stage.
|
||
Preflight checks readiness and coordinates the output session before model
|
||
work. Processing normalizes once, then the runner chunks once, applies stages
|
||
in order, and carries recoverable failures into reports. ReleaseKit owns
|
||
validation, source publication, and asynchronous Linux delivery.
|
||
|
||
| Stages | Reviewed behavior |
|
||
| --- | --- |
|
||
| 1 and 15 | Dependency and toolchain pins, release profile, launcher, tag verification, and candidate evidence. A disposable ReleaseKit check passed with Go 1.27.3, including race tests, vet, Markdown and module checks, both Linux builds, and version output. The plan records the annotated-tag delivery rehearsal. |
|
||
| 2–4 | Native application load, main-relative engine inputs, prompt resources and source accounting, prepared role clients, retries and repairs, and redaction. See R2, R4, R10, R13, and R14. |
|
||
| 5–6 | Native app and PromptKit scopes, typed routing, preview, validation, and explicit smoke, including controlled transport tests. See R5 and R13. |
|
||
| 7–8 | Destination checks, deferred device/pipe delivery, transcript replacement, ordinary reports, optional debug, and reporting modes. The output-path loop compares a growing source list against at most four destinations. See R1, R7, R9, R11, and R18. |
|
||
| 9–10 | Native identified glossary composition, typed extensions and selections, recorded defaults, and explicit clearing after CLI assignments. See R15. |
|
||
| 11–12 | Stage-specific glossary views, late-stage readiness, fixed chunks, ordered mutation, and partial success. See R3, R9, R12, R15–R17, and R20. |
|
||
| 13–14 | Stage attribution, report schemas, canonical docs, migration, examples, ADR, and attribution. The complete stage-by-chunk matrices are part of enabled reports. See R1, R6–R8, R18, and R19. |
|
||
|
||
## R1 — P2: Keep Failure Records and Counts Consistent
|
||
|
||
**Status:** Completed. Failed report counts are derived from their records,
|
||
and fatal attribution uses the returned processing error and known stage.
|
||
|
||
**Category**
|
||
|
||
Incorrect ordinary reports in Stages 7, 8, and 13.
|
||
|
||
**Evidence**
|
||
|
||
[Finalization](../../internal/cli/process_output.go#L24) appends
|
||
run-level output errors after the report snapshot's counts have been computed.
|
||
A real run with an empty pipeline and `--output /dev/full` returned status 1.
|
||
Both YAML summaries still showed `run_errors: 0`, while `errors.yml` contained
|
||
an `output_failure` record. The Markdown report uses the same stale count.
|
||
[Report initialization](../../internal/cli/process_inputs.go#L89) prepares a
|
||
failed-status bundle by changing only the status, so a failed initialization
|
||
can leave reports with no record of the known failure.
|
||
|
||
[Pipeline failure attribution](../../internal/cli/process_pipeline.go#L48)
|
||
also takes the first operation with any failure reason from the last result.
|
||
A recoverable timeout or unusable decision set can precede the operation
|
||
that actually stopped processing. This can overwrite the cancellation reason
|
||
and cite a different chunk or validator. A stage-only origin is also rendered
|
||
as chunk zero by [report mapping](../../internal/framework/processreport/report.go#L119).
|
||
|
||
**Recommended change**
|
||
|
||
Build failed reports from the actual run failures and
|
||
known output failures, deriving counts and records together. Keep fatal
|
||
classification tied to the error that stopped the runner. Add stage, chunk,
|
||
and operation attribution only where the runner has that evidence.
|
||
|
||
**Acceptance**
|
||
|
||
A late transcript or report failure produces consistent
|
||
failed-status records and counts in the remaining reports. A recoverable error
|
||
followed by cancellation or a fatal operation retains both facts separately.
|
||
The primary failure and any already published transcript survive secondary
|
||
report failures.
|
||
|
||
## R2 — P2: Preserve Host Prompt Source Context Through Symlinks
|
||
|
||
**Status:** Completed. Prompt imports use the recorded native host context and protect the resolved sources.
|
||
|
||
**Category**
|
||
|
||
Main-relative resource loading divergence in Stage 3.
|
||
|
||
**Evidence**
|
||
|
||
[Engine assembly](../../internal/framework/llm/runtime.go#L58)
|
||
passes ConfigKit's passive host resource view back as an opaque native
|
||
filesystem. That view lacks native symlink inspection. In a disposable project,
|
||
`prompts.yml -> defs/resources.yml` acquired the prompt but resolved its
|
||
`content_file: message.txt` beside the link, failing despite the file being
|
||
present beside `defs/resources.yml`. An ordinary manifest import of the same
|
||
target worked. Adding a file beside the link also made the linked definition
|
||
load, demonstrating that it used the wrong base.
|
||
|
||
**Recommended change**
|
||
|
||
Carry the recorded host base and containment root into
|
||
PromptKit as a supported host context, so ConfigKit can acquire host sources
|
||
and resolve links using its native host rules. Keep embedded/native filesystem
|
||
inputs separate. If the public handoff cannot express the required context,
|
||
request the missing upstream facility with this fixture.
|
||
|
||
**Acceptance**
|
||
|
||
A prompt acquired through a supported in-root host symlink
|
||
resolves its message file beside the acquired definition. Source evidence and
|
||
output collision checks still protect consumed files. Preserve the absent-main
|
||
behavior and the recorded wider-root case.
|
||
|
||
## R3 — P2: Separate Glossary Context From Protection in Proposal Prompts
|
||
|
||
**Status:** Completed. Proposal requests carry separate context and protected spellings, with an empty preview default.
|
||
|
||
**Category**
|
||
|
||
An unintended prompt restriction in Stage 11.
|
||
|
||
**Evidence**
|
||
|
||
[The runner's specifications](../../internal/framework/contracts/contracts.go#L101)
|
||
correctly derive independent rich entries and protected vocabulary. The
|
||
[proposal builder](../../internal/framework/proposal_generation/module_proposal.go#L35)
|
||
passes only the rich entries. The embedded [grammar](../../internal/prompts/assets/prompts/modules.grammar.proposal.yaml#L21),
|
||
[homophones](../../internal/prompts/assets/prompts/modules.homophones.proposal.yaml#L19),
|
||
and [spoken-word](../../internal/prompts/assets/prompts/modules.spoken_word.proposal.yaml#L23)
|
||
prompts call those entries protected. With `glossaries: [party]` and
|
||
`protected_terms: []`, they still tell the model to preserve party spellings.
|
||
A protection-only selection also has no distinct representation in these prompts.
|
||
The deterministic checks see the intended independent selection, but they
|
||
cannot recover valid proposals that the model was instructed to omit.
|
||
|
||
**Recommended change**
|
||
|
||
Describe rich entries as context and give protection
|
||
instructions the effective protected spellings separately wherever they are
|
||
needed. Use the same stage-local exemption for glossary correction. Retain
|
||
the domain limits on each module, including acoustic plausibility and the
|
||
spoken-word repetition exception.
|
||
|
||
**Acceptance**
|
||
|
||
Captured requests distinguish context-only and protection-only
|
||
selections. Explicit empty protection imposes no glossary-derived prompt
|
||
restriction. The deterministic and apply-time checks still use the effective
|
||
stage vocabulary.
|
||
|
||
## R4 — P2: Show Native Configuration Diagnostics During Processing
|
||
|
||
**Status:** Completed. Processing now presents native ConfigKit error details through CommandKit and preserves the typed load error. CLI coverage checks migration guidance, an invalid setting, and the absence of output on failure.
|
||
|
||
**Category**
|
||
|
||
Missing migration guidance and actionable failure detail in
|
||
Stages 2 and 8.
|
||
|
||
**Evidence**
|
||
|
||
The [version rule](../../internal/core/config/config.go#L139)
|
||
provides migration guidance as a ConfigKit diagnostic. The
|
||
[process error path](../../internal/cli/process.go#L22) prints the load error's
|
||
summary. A real invocation using a v2 main file returned status 2 with
|
||
`configkit: configuration load failed with 1 error` and no migration guidance.
|
||
`config validate --scope app` on that same file printed the path, location,
|
||
and migration guidance.
|
||
|
||
**Recommended change**
|
||
|
||
Surface the native safe findings for application-load
|
||
failures in the process command, using CommandKit's normal error presentation
|
||
and retaining the typed cause. Reuse the native diagnostics rather than
|
||
maintaining a second configuration validator or formatter.
|
||
|
||
**Acceptance**
|
||
|
||
Processing an obsolete main file explains the migration before
|
||
creating outputs or calling a model. An ordinary invalid setting identifies
|
||
the actionable configuration problem through the same path.
|
||
|
||
## R5 — P3: Keep PromptKit Configuration Comparison Sides Independent
|
||
|
||
**Status:** Completed. The native configuration scope acquires each comparison side independently. A comparison with equal selections detects a resource edit between loads. Request commands retain invocation-local capture.
|
||
|
||
**Category**
|
||
|
||
Native behavior overridden in Stage 5.
|
||
|
||
**Evidence**
|
||
|
||
[PromptKit's loader](../../internal/cli/app_wiring.go#L46) memoizes
|
||
engines by deep equality of the selection within the invocation state. That
|
||
state is shared by the native configuration suite. The released ckprompt
|
||
comparison adapter acquires both sides independently, including equal
|
||
selections. Audita returns the left-hand capture for an equal right-hand
|
||
selection, adding a shared-snapshot rule the plan explicitly avoids.
|
||
|
||
**Recommended change**
|
||
|
||
Give the comparison loader independent acquisitions.
|
||
Keep shared acquisition within a selected processing or request workflow
|
||
where it serves the captured-engine requirement.
|
||
|
||
**Acceptance**
|
||
|
||
Each comparison side reaches the loader and gets its own
|
||
native evidence. Other standard commands continue to use the same source
|
||
selection rules and native comparison status.
|
||
|
||
## R6 — P3: Render Stage IDs as Data in Markdown
|
||
|
||
**Status:** Completed. Markdown escapes stage labels as literal cell text. Tests cover Markdown syntax, a pipe next to a backslash, and exact YAML IDs. A rendered table check confirmed the label stays in one cell.
|
||
|
||
**Category**
|
||
|
||
Incomplete output escaping in Stage 13.
|
||
|
||
**Evidence**
|
||
|
||
[Markdown rendering](../../internal/core/reporting/markdown.go#L174)
|
||
handles table separators, backticks, newlines, and HTML, but a valid ID such as
|
||
`[party](https://example.invalid/)` still renders as a link. Emphasis delimiters
|
||
can similarly hide literal characters. The stage ID is an exact operator label
|
||
and the same text is preserved in machine records.
|
||
|
||
**Recommended change**
|
||
|
||
Render IDs as literal text within report labels using
|
||
Markdown-appropriate escaping. Continue to allow arbitrary supported ID text
|
||
and use generated coordinates for paths.
|
||
|
||
**Acceptance**
|
||
|
||
A representative ID containing Markdown syntax renders with
|
||
its literal text, retains the exact YAML/JSON ID, and stays in its intended table
|
||
cell. Existing unnamed and repeated stage labels remain distinguishable.
|
||
|
||
## R7 — P3: Retire Disconnected Cutover Code and Test the Live Paths
|
||
|
||
**Status:** Completed. Removed retired report and debug builders, utilization and timing bookkeeping, and obsolete invocation fields. Report destination tests now drive the processing command. Internal and offline subprocess suites passed.
|
||
|
||
**Category**
|
||
|
||
Dead production code and misleading test coverage in Stages 7,
|
||
8, 13, and 15.
|
||
|
||
**Evidence**
|
||
|
||
The old report-directory helper in `internal/cli/config_input.go`
|
||
has only a test caller. [Processing](../../internal/cli/process_inputs.go#L72)
|
||
implements directory selection separately. The former
|
||
configuration explanation in `internal/framework/llm/safe_view.go` and
|
||
utilization view in `internal/framework/processreport/llm.go` also have
|
||
only test callers. The runner still collects validator timing for the latter.
|
||
The single `GlossaryPath` survives through report data and prints an empty
|
||
`Glossary input:` line after the standalone glossary loader was removed.
|
||
The debug invocation record still emits `modules: null` on a run with active
|
||
stages. The CLI writes resolved stages to the effective configuration artifact
|
||
instead. [Invocation metadata](../../internal/cli/process_inputs.go#L104)
|
||
|
||
**Recommended change**
|
||
|
||
Put destination-selection coverage on the live path.
|
||
Remove superseded formatters, bookkeeping, DTO fields, and tests whose only
|
||
consumer was a retired artifact. Preserve required stage attribution,
|
||
redaction, and native configuration provenance on their active paths.
|
||
|
||
**Acceptance**
|
||
|
||
Regression tests exercise actual report directory selection.
|
||
Remove the stale standalone glossary path from current evidence. Retain
|
||
diagnostics with an active consumer or a documented requirement.
|
||
|
||
## R8 — P3: Clarify the Configuration Cutover and Preflight Documentation
|
||
|
||
**Status:** Completed. Migration guidance now distinguishes the application schema from the executable version and explains glossary descriptions and completion counts. Subprocess guidance reflects early transcript parsing, and the v3.0.0 reference points to the tagged format.
|
||
|
||
**Category**
|
||
|
||
Documentation divergence in Stage 14.
|
||
|
||
**Evidence**
|
||
|
||
[The migration guide](../migrations/v3.md) calls the cutover
|
||
“Audita v3”, although Audita 3.x already shipped with application schema v2.
|
||
It also omits the change that default-mode processing hides completion
|
||
counts, including when ordinary reports are disabled.
|
||
It leaves out the rename from legacy glossary `summary` to `description`.
|
||
A copied `summary` is retained as an extension, so migration can appear
|
||
successful while the existing prompt builder loses that context.
|
||
[Legacy format](../migrations/v2.md) · [Glossary projection](../../internal/core/glossary/selection.go#L29)
|
||
[Subprocess guidance](../integrations/subprocess.md#result-handling) says
|
||
failed reports can describe invalid input after initialization. Transcript
|
||
parsing now occurs before report preparation.
|
||
[The v3.0.0 note](../releases/v3.0.0.md) pins historical examples to its tag
|
||
but still links the old format description to the current configuration guide.
|
||
|
||
**Recommended change**
|
||
|
||
Distinguish application schema v3 from the executable
|
||
release version. Explain how to see completion counts at the cutover, link
|
||
historical format guidance to its applicable revision, and align subprocess
|
||
guidance with early transcript validation. Include the glossary field mapping
|
||
so operators can preserve their prompt context.
|
||
|
||
**Acceptance**
|
||
|
||
A v3.1.x operator can identify which schema is changing and
|
||
how to observe completed runs. A migrated glossary entry retains the intended
|
||
description in prompt context. Current guidance explains which failures can
|
||
leave reports. Historical references continue to describe their own versions.
|
||
|
||
## R9 — P2: Honor Cancellation in Normalization-Only Processing
|
||
|
||
**Status:** Completed. Processing checks cancellation after normalization and before delivery. A controlled empty-pipeline test preserves the existing transcript, finalizes both YAML reports as failed, and retains the cancellation cause.
|
||
|
||
**Category**
|
||
|
||
Cancellation regression in Stages 8 and 12.
|
||
|
||
**Evidence**
|
||
|
||
[Processing](../../internal/cli/process.go#L44) lost the
|
||
post-normalization context check during the cutover. The
|
||
[runner](../../internal/framework/runner/runner.go#L172) checks for cancellation
|
||
through stage work, which an empty pipeline never enters. In a disposable
|
||
normalization-only run, sending SIGINT just after the initial reports appeared
|
||
returned status 0 and published the transcript.
|
||
|
||
**Recommended change**
|
||
|
||
Check the invocation context at the processing handoff
|
||
and before committing a successful result. Use the existing canceled outcome
|
||
and failed-report finalization for cancellation observed before delivery.
|
||
|
||
**Acceptance**
|
||
|
||
A controlled cancellation after preparation and before
|
||
successful delivery of an empty-pipeline run leaves the final transcript
|
||
untouched and attempts failed-status reports. An uncanceled empty pipeline
|
||
still produces normalized output without provider calls.
|
||
|
||
## R10 — P2: Mask the Credential Value PromptKit Actually Sends
|
||
|
||
**Status:** Completed. Redaction now includes the raw and trimmed forms of referenced credential values. A local provider regression echoes both forms and checks retained diagnostics, reports, and command output.
|
||
|
||
**Category**
|
||
|
||
Credential normalization and diagnostic redaction disagree in Stage 4.
|
||
|
||
**Evidence**
|
||
|
||
PromptKit trims surrounding whitespace from its environment credential before
|
||
setting the Authorization header. Audita supplies the literal environment
|
||
value to its existing exact-value redactor. With a synthetic credential padded
|
||
by spaces and a tab, a loopback provider observed the trimmed value and echoed
|
||
it in a schema-invalid response. The run completed with a recoverable model
|
||
error and wrote the trimmed credential to its proposal response artifact.
|
||
The same response with an unpadded credential was masked.
|
||
[Credential collection](../../internal/framework/llm/secrets.go#L40) ·
|
||
[Proposal debug output](../../internal/framework/proposal_generation/generate.go#L155)
|
||
|
||
**Recommended change**
|
||
|
||
Supply both raw and normalized nonempty values for referenced credential
|
||
variables to the existing redactor. Keep credential selection in PromptKit.
|
||
|
||
**Acceptance**
|
||
|
||
A controlled provider that echoes the effective value of a padded synthetic
|
||
credential leaves neither representation in retained diagnostics, ordinary
|
||
reports, or surfaced errors. Credential availability and processing outcomes
|
||
keep their native behavior.
|
||
|
||
## R11 — P2: Preserve Pipes That Appear After Output Preparation
|
||
|
||
**Status:** Completed. Delivery checks the destination type before opening a prepared stream or replacing a file. Tests cover both type changes, preservation of the replacement destination, and cleanup of staged output.
|
||
|
||
**Category**
|
||
|
||
The new Stage 7 preparation changes the existing special-file delivery behavior.
|
||
|
||
**Evidence**
|
||
|
||
Preparation classifies a missing destination as a regular-file output. Final
|
||
delivery then renames the staged transcript over that path without checking
|
||
its current type. In a controlled run, the provider fixture created a named
|
||
pipe at the selected path after the first proposal request arrived. The
|
||
previous binary wrote the transcript to the pipe and left it in place. The
|
||
new binary replaced it with a regular file, and both reported success.
|
||
[Output preparation](../../internal/core/diagnostics/output_session.go#L140) ·
|
||
[Transcript publication](../../internal/core/diagnostics/output_session.go#L309)
|
||
|
||
**Recommended change**
|
||
|
||
Recheck the destination type at delivery using the existing filesystem
|
||
behavior. Deliver to a current pipe or device directly, or return a clear
|
||
failure if its change makes the prepared destination unusable. Keep the
|
||
ordinary per-file replacement and temporary-file cleanup.
|
||
|
||
**Acceptance**
|
||
|
||
A FIFO created after preparation survives delivery. An existing FIFO remains
|
||
unopened during preflight. Ordinary destinations still publish from the
|
||
prepared sibling file, and failed delivery cleans up that file.
|
||
|
||
## R12 — P2: Preserve Longest-Match Context for Glossary Exemptions
|
||
|
||
**Status:** Completed. Active spellings remain longest-match boundaries without contributing protected identities. Tests cover full-name and span edits, standalone and longer protected terms, stage isolation, and apply-time agreement.
|
||
|
||
**Category**
|
||
|
||
The new Stage 11 exemption changes how the existing matcher identifies
|
||
overlapping terms.
|
||
|
||
**Evidence**
|
||
|
||
The projection removes active spellings before building the protected-term
|
||
matcher. That lets a shorter protected spelling match inside an exempt active
|
||
name. With `Greater Mage Armor` and `Greater Magus Armor` active and
|
||
`Mage Armor` protected from another collection, the previous glossary stage
|
||
accepted the full-name correction under controlled validation. The new stage
|
||
rejected it before either model validator ran. It treated the suffix of the
|
||
active name as a separate protected occurrence.
|
||
[Vocabulary projection](../../internal/core/glossary/vocabulary.go#L19) ·
|
||
[Longest-first matcher](../../internal/framework/validators/protected_terms.go#L29) ·
|
||
[Deterministic validator](../../internal/framework/validators/deterministic.go#L139)
|
||
|
||
**Recommended change**
|
||
|
||
Keep active spellings available as match boundaries and exempt the matched
|
||
active occurrence from protection. Apply the same interpretation in
|
||
deterministic validation and when evaluating the edited segment at apply time.
|
||
|
||
**Acceptance**
|
||
|
||
An exempt full name can be corrected when a shorter protected term occurs
|
||
inside it. A standalone occurrence of the shorter term and a distinct longer
|
||
protected name retain protection. Other module stages keep their independently
|
||
selected vocabulary.
|
||
|
||
## R13 — P2: Collect Prompt Message Paths From Effective Data
|
||
|
||
**Status:** Completed. Processing walks one effective PromptKit projection and uses per-field origins for host message paths. Inspection and request commands skip collision accounting. Tests cover inherited, collection-relative, and symlinked message sources.
|
||
|
||
**Category**
|
||
|
||
Repeated configuration history replay in Stages 3, 5, and 6.
|
||
|
||
**Evidence and complexity**
|
||
|
||
Audita enumerates historical fields and calls `Explain` for every effective
|
||
host-backed `content_file`, then reads only the effective YAML scalar.
|
||
ConfigKit v1.2.0's implementation of `Explain` replays source applications
|
||
and clones configuration checkpoints on each call. With `M` such fields and
|
||
`H` source history records, scanning history alone costs at least
|
||
`Ω(M × H)`. One file-backed message per source gives quadratic growth in
|
||
the number of sources, before accounting for replay and cloning costs.
|
||
The native resource commands also do the work because they build a runtime
|
||
and then retain only its engine.
|
||
[Source-path capture](../../internal/framework/llm/source_paths.go#L40) ·
|
||
[Engine wiring](../../internal/cli/app_wiring.go#L62)
|
||
|
||
**Refactoring opportunity**
|
||
|
||
Collect collision paths only for processing. Use one walk of the native
|
||
effective projection with per-field origin lookups, or use an upstream
|
||
effective-only accessor if it can expose the required path evidence. Keep
|
||
the actual load and source-relative resolution with the owning libraries.
|
||
|
||
**Acceptance**
|
||
|
||
Source accounting for processing uses a single effective pass and origin
|
||
lookups without replaying history per message. Inherited and relocated
|
||
message files still receive collision protection, including the host context
|
||
covered by R2. Inspection retains its native history commands.
|
||
|
||
## R14 — P3: Reuse Credential Source Discovery Across Diagnostics
|
||
|
||
**Status:** Completed. Credential source names are discovered once per runtime, with current values read at each masking boundary. Interaction and finalization redactors are reused, and deterministic validators receive no interaction writer. Tests cover credential changes and newly prepared names.
|
||
|
||
**Category**
|
||
|
||
Repeated catalog scans in Stages 4 and 8.
|
||
|
||
**Evidence and complexity**
|
||
|
||
`ConfiguredSecrets` lists and inspects every backend and profile each time.
|
||
Debug processing calls it per proposal interaction and per validator in each
|
||
active section, including deterministic validators that never write a model
|
||
interaction. Finalization also recollects it for each warning. With `D`
|
||
redaction preparations and `C` catalog resources, this is at least
|
||
`Θ(D × C)` resource visits, plus the work of copying each inspection result.
|
||
[Credential collection](../../internal/framework/llm/secrets.go#L11) ·
|
||
[Validator writer setup](../../internal/framework/runner/runner.go#L620) ·
|
||
[Warning presentation](../../internal/cli/process_output.go#L14)
|
||
|
||
**Refactoring opportunity**
|
||
|
||
Discover deduplicated credential source names once per captured engine.
|
||
Read values at the existing masking boundary, including additional names
|
||
supplied by prepared requests and normalized values from R10. Reuse writers
|
||
or redactors where they already share a value set, and construct interaction
|
||
writers only for consumers that can use them. This reduces catalog discovery
|
||
to one pass while retaining the cost of reading the referenced values.
|
||
|
||
**Acceptance**
|
||
|
||
Increasing the number of sections or warnings does not cause repeated
|
||
resource inventory scans. All existing masking behavior remains available,
|
||
and PromptKit still controls credential selection and availability.
|
||
|
||
## R15 — P3: Avoid Repeated Glossary Selection and Membership Scans
|
||
|
||
**Status:** Completed. List defaults clone only when inherited. Selection skips repeated collection names, readiness short-circuits without a rich projection, and contributor membership uses a set while preserving order. Existing inheritance, readiness, and glossary tests passed.
|
||
|
||
**Category**
|
||
|
||
Redundant list work and quadratic deduplication in Stages 10–12.
|
||
|
||
**Evidence and complexity**
|
||
|
||
- `resolveSettings` clones root lists before ConfigKit checks whether a stage
|
||
already supplied a list. Across `S` stages and `R` root list elements,
|
||
this copies `Θ(S × R)` elements even if every stage overrides both lists.
|
||
- `Select` walks a collection on every occurrence of its name. Selecting a
|
||
collection of `G` entries `K` times visits `Θ(K × G)` entries and
|
||
returns only `G`. Readiness also materializes a full rich selection just
|
||
to test whether it is empty.
|
||
- `Vocabulary` scans a spelling's existing contributor slice for every
|
||
insertion. When `K` distinct entries share an alias, the membership tests
|
||
for that alias alone take `Θ(K²)` comparisons.
|
||
|
||
[Recorded defaults](../../internal/core/config/resolve.go#L21) ·
|
||
[Glossary selection](../../internal/core/glossary/selection.go#L19) ·
|
||
[Readiness](../../internal/framework/preflight/preflight.go#L95) ·
|
||
[Contributor grouping](../../internal/core/glossary/vocabulary.go#L19)
|
||
|
||
**Refactoring opportunity**
|
||
|
||
Clone only lists that inherit. Skip repeated collection names before walking
|
||
their entries, and stop the readiness test at the first populated selection.
|
||
Use a membership set keyed by spelling and entry identity while keeping the
|
||
ordered output slices. That gives expected constant-time membership per
|
||
contribution, excluding the cost of normalizing and hashing the text.
|
||
|
||
**Acceptance**
|
||
|
||
First occurrence, contributor order, exact collection-scoped IDs, detached
|
||
inherited lists, and the distinction between omission and explicit clearing
|
||
are preserved. Readiness keeps the same late-stage diagnostics.
|
||
|
||
## R16 — P2: Reuse Prepared Glossary Views and Index Validation Context
|
||
|
||
**Status:** Completed. Each stage prepares shared protection and context matchers and serializes its proposal glossary variables once. Context gathers matched entry positions in selection order. Tests cover shared identities, ordering, exemptions, and request content.
|
||
|
||
**Category**
|
||
|
||
Per-section rebuilds and per-candidate glossary scans in Stages 11 and 12.
|
||
|
||
**Evidence and complexity**
|
||
|
||
The selected stage vocabulary stays fixed, but protected-term validation
|
||
compiles its matcher for each active section. Each LLM validator rebuilds the
|
||
context matcher again for its section. With `Q` such invocations and
|
||
`B` the cost of building a matcher, that is `Θ(Q × B)` preparation for
|
||
the same data. Proposal requests also re-encode identical glossary JSON in
|
||
each section. Within LLM validation, after matching text, every candidate
|
||
scans all `G` selected entries to recover their order. That adds
|
||
`Θ(P × G)` work for `P` candidates, even when nothing matches.
|
||
[Protected-term validation](../../internal/framework/validators/deterministic.go#L139) ·
|
||
[LLM validator request](../../internal/framework/validators/llm_validators.go#L252) ·
|
||
[Context selection](../../internal/framework/validators/llm_context.go#L12) ·
|
||
[Proposal variables](../../internal/framework/proposal_generation/variables.go#L14)
|
||
|
||
**Refactoring opportunity**
|
||
|
||
Prepare immutable stage-local matchers and, where used, serialized glossary
|
||
variables once. Build an identity-to-entry-position map alongside the context
|
||
view. Gather just the matched positions and order those positions before
|
||
materializing entries. After indexing, selecting `k` distinct matches per
|
||
candidate costs expected `O(k + k log k)` apart from text matching, rather
|
||
than a full glossary scan. Sharing the encoding saves local serialization
|
||
while leaving the selected glossary payload in each request.
|
||
|
||
**Acceptance**
|
||
|
||
Rich entries retain selection order and all contributing identities. Scope
|
||
and overlap rules remain as specified, including R3 and R12. Matchers and
|
||
immutable render inputs are shared safely across sections of the same stage.
|
||
|
||
## R17 — P3: Reuse Validation Payload Costs While Fitting and Batching
|
||
|
||
**Status:** Completed. The built-in estimator composes compact JSON costs across context fitting and greedy batching. Other estimators retain exact serialization. Equivalence tests cover trimmed context, oversized items, token counts, batch splits, escaped text, and local references.
|
||
|
||
**Category**
|
||
|
||
Quadratic prefix serialization in the existing budgeting path now used with
|
||
stage-specific budgets and shared glossary context.
|
||
|
||
**Evidence and complexity**
|
||
|
||
`fitValidationContext` removes one optional entry at a time and serializes
|
||
the remaining item again. Removing most of `K` similarly sized entries
|
||
serializes `Θ(K²)` entry data. `ChunkLLMValidationItems` copies, renumbers,
|
||
and marshals the entire tentative batch on each append. A batch that grows
|
||
to `N` similarly sized candidates costs `Θ(N²)` in those operations.
|
||
Small budgets bound the batch size, but the configured budget may grow and
|
||
many glossary entries can match one candidate.
|
||
[Context fitting](../../internal/framework/validators/llm_context.go#L43) ·
|
||
[Batch construction](../../internal/framework/validators/llm_batching.go#L16) ·
|
||
[Stage budget use](../../internal/framework/validators/llm_validators.go#L74)
|
||
|
||
**Refactoring opportunity**
|
||
|
||
Keep the existing drop order, greedy batch boundaries, and production
|
||
estimator. For that estimator, compose counts of whitespace-delimited runs
|
||
and punctuation from encoded components, accounting for join boundaries
|
||
and rounding once on the complete payload. Use those costs for prefix
|
||
selection and incremental batch construction, then render the selected
|
||
payload. Keep exact serialization for estimators whose costs cannot be
|
||
composed. This can reduce production work to one pass over the encoded
|
||
components and inexpensive checks over the candidate prefixes.
|
||
|
||
**Acceptance**
|
||
|
||
Representative payloads produce the same retained context, estimates,
|
||
oversized-candidate decisions, and batch boundaries. Include local reference
|
||
numbering and punctuation or escaped text at encoding boundaries.
|
||
|
||
## R18 — P3: Skip Evidence Work for Disabled Outputs
|
||
|
||
**Status:** Completed. Disabled reporting skips report payloads and detailed rows while retaining completion counts. Detailed observation hooks require an active debug writer. Subprocess coverage compares output and nonzero correction, error, and recovery counts across all four settings.
|
||
|
||
**Category**
|
||
|
||
Unused report rendering and event collection in Stages 8 and 13.
|
||
|
||
**Evidence and complexity**
|
||
|
||
Preparation renders both initial report bundles and finalization renders
|
||
the final bundle even with reporting disabled. For `S` stages and `Q`
|
||
chunks, this still builds two `Θ(S × Q)` Markdown matrices at finalization,
|
||
in addition to serializing error and recovery records. Processing also
|
||
retains all detailed LLM observations, copies them at completion, and sorts
|
||
them before testing whether debug artifacts are active. For `E` calls and
|
||
events, that is `Θ(E)` storage and up to `O(E log E)` sort comparisons.
|
||
Ordinary outcome counts use the role client's terminal event, independently
|
||
of that detailed collector.
|
||
[Report preparation](../../internal/cli/process_inputs.go#L90) ·
|
||
[Report delivery](../../internal/cli/process_output.go#L32) ·
|
||
[Observation wiring](../../internal/cli/process_pipeline.go#L18) ·
|
||
[Snapshot sorting](../../internal/framework/llm/observations.go#L148)
|
||
|
||
**Refactoring opportunity**
|
||
|
||
Build file payloads only when ordinary reporting is enabled. Install detailed
|
||
observation hooks only for an active debug writer, while retaining terminal
|
||
counts and compact completion facts for ordinary reports and CLI summaries.
|
||
Coordinate removal of unused utilization bookkeeping with R7.
|
||
|
||
**Acceptance**
|
||
|
||
Disabled outputs skip their formatting and optional observation costs.
|
||
Enabled reports and debug artifacts retain their data, and verbose completion
|
||
counts remain available with ordinary reporting disabled.
|
||
|
||
## R19 — P3: Index Interrupted Correction Ledger Entries
|
||
|
||
**Status:** Completed. The ledger retains each stage's proposal decision index and looks up interrupted candidates by generated stage instance. A repeated-module test confirms attribution, partial decisions, and ordering.
|
||
|
||
**Category**
|
||
|
||
Repeated joins in the Stage 13 debug ledger.
|
||
|
||
**Evidence and complexity**
|
||
|
||
The first pass indexes decisions by proposal for each stage and discards the
|
||
index after emitting completed entries. Every interrupted candidate then
|
||
searches stage results and scans the matching stage's entire decision list
|
||
again. With `I` interrupted candidates, `S` stages, and `D` decisions in
|
||
one failed stage, lookup can cost `Θ(I × (S + D))`. Interruption late in a
|
||
large stage can make both `I` and `D` grow with candidate count.
|
||
[Decision indexing and interrupted fallback](../../internal/framework/processreport/correction_ledger.go#L64)
|
||
|
||
**Refactoring opportunity**
|
||
|
||
Reuse the first-pass decision index through the interrupted-entry join and
|
||
index stage metadata by generated instance. Expected lookup and grouping
|
||
become linear in stages, candidates, and decisions visited, plus the
|
||
existing final sort. Keep operator IDs as labels.
|
||
|
||
**Acceptance**
|
||
|
||
Interrupted entries retain the same stage identity, partial validator
|
||
decisions, and output order without rescanning decisions for each candidate.
|
||
|
||
## R20 — P3: Skip Apply-Time Copies for Sections With No Approved Edits
|
||
|
||
**Status:** Completed. Sections without approved proposals retain their existing segments. The runner still records every work unit, and sections with approved changes use the existing ordered application and protection checks. No-edit coverage verifies transcript and work-unit results.
|
||
|
||
**Category**
|
||
|
||
Avoidable transcript copying during Stage 12 application.
|
||
|
||
**Evidence and complexity**
|
||
|
||
The runner invokes application for every section even when its approved
|
||
proposal slice is empty. Application clones the section's segments and
|
||
categories and builds the segment-ID map before inspecting proposals. For a
|
||
stage with `N` segments and no approved edits, that is `Θ(N)` segment
|
||
copies and map entries, plus category copies, to preserve identical text.
|
||
Across repeated no-op stages this grows with the number of stages times the
|
||
transcript size.
|
||
[Section application](../../internal/framework/runner/runner.go#L289) ·
|
||
[Per-section clone and index](../../internal/framework/proposals/apply.go#L42)
|
||
|
||
**Refactoring opportunity**
|
||
|
||
Retain the existing section when it has no approved proposals. Invoke
|
||
application and its protection checks only for sections with candidates.
|
||
Keep the runner's work-unit bookkeeping for every section.
|
||
|
||
**Acceptance**
|
||
|
||
No-edit and all-recoverable-failure runs produce the same normalized
|
||
transcript and reports. Sections with approved proposals still use ordered
|
||
application and recheck the current text before each edit.
|