Record debug bundle organization and latest symlink

This commit is contained in:
2026-11-19 18:55:24 +00:00
parent 9c4c2fc88d
commit 5618adea8e

View File

@@ -24,8 +24,9 @@ PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target.
Item 2 selects the application command structure and input conventions. Item 3
selects native composition, shared loaders, and application preflight. Item 4
selects native reporting, machine output, and outcomes. Item 5 selects conditional
debug capture through the shared `--debug` flag and native diagnostics. Debug
content/organization, exact run-report fields, and verification remain open below.
debug capture through the shared `--debug` flag, native diagnostics, and a run/
operation bundle with a `latest` symlink. Capture depth, exact run-report fields,
and verification remain open below.
The PromptKit architecture and checkpoint decisions remain in force. This
discussion develops the CommandKit portion of that migration and its interactions
@@ -646,9 +647,10 @@ retains the bookkeeping boundary; U9 still owns concrete durable record choices.
**Discussion / decision:** Controls, native capture hooks, lifecycle, failure
policy, and optional diagnostics agreed. Use `--debug` for native verbosity and
durable capture with `/tmp/notarius-debug` as the default root. Operator-owned
data handling and recording each fact once are agreed. The summary/trace
distinction, concrete capture inventory, and bundle organization remain under
discussion. Implementation is pending.
data handling and recording each fact once are agreed. Replace the summary/trace
hierarchy with a compact run index, configuration/composition snapshots, and
operation records; maintain a `latest` symlink at the debug root. Capture depth
and concrete serialized fields remain under discussion. Implementation is pending.
### Shared Debug Control With Conditional Capture
@@ -731,6 +733,15 @@ and the destination and run identity are known, before engine construction and
pipeline preflight. Argument and configuration-load failures use native
diagnostics and may have no bundle. Loaders remain free of storage allocation.
After creating the run directory, update `<debug-root>/latest` to a relative
symlink targeting that run ID before engine construction or preflight starts.
This points to the newly started debug session, including one that later fails;
it is not a last-successful-run selector. Runs without capture leave it alone.
Do not update it again at completion, so an older overlapping run cannot take
the link back when it finishes. Concurrent starts use the last successful link
update; individual run directories remain the actual storage locations.
Treat link-update failures under the requested-capture failure policy below.
One application-owned recorder records initial context, accepts native execution
evidence and application decisions, and finalizes once with the known pipeline,
publication, and capture outcomes. Debug remains separate from output/cache,
@@ -793,25 +804,66 @@ Neither debug verbosity nor enabled capture initiates additional provider probes
Smoke reports retain native support, applicability, and result distinctions;
status zero alone does not establish that a check passed.
### Open Discussion: Summary And Trace
### Selected Bundle Organization
**Agreed principle:** Record each fact once and reference it where needed. Use
shared application outcome facts and native execution evidence rather than
parallel summary/trace accounting or repeated snapshots.
The current `summary/` and `trace/` layout is not selected as the new design.
Discuss what diagnostic questions the bundle should answer and which application
records and native evidence it needs. Summary/trace are not separate security
classes. Keep these choices open:
Organize the bundle around the run and its operations, replacing `summary/` and
`trace/` as separate categories. Use a compact overview for navigation and keep
detailed evidence with its operation. Summary/trace are not security classes.
- Whether a separate summary is useful, and whether it is a compact index/final
outcome rather than another copy of execution records.
- Whether summary/trace are useful conceptual categories or directory boundaries.
- Which prepared details, outputs, conversation snapshots, and application
decisions to persist, and at what granularity.
- How self-contained a failed-run bundle should be, given that it may have no
published output bundle, and when to reference existing artifacts instead of
copying them.
Illustrative layout: `latest` is the selected link name; other file names and
exact serialized fields remain implementation choices:
```text
<debug-root>/
latest -> <run-id>/
<run-id>/
run.json
config/
application.yml
promptkit.yml
pipeline.json
operations/
<operation-id>/
...
```
| Component | Purpose |
| --- | --- |
| Run overview | Identify the invocation, outcome, published output, and available diagnostic records. |
| Configuration snapshots | Preserve effective settings through native inspection. |
| Resolved pipeline | Describe selected steps, lanes, modules, validators, and reference relationships. |
| Operation records | Connect diagnostic evidence with its stage, chunk, lane, or validator. |
The overview is an index rather than a second execution ledger. Its terminal
facts come from the same application outcome used for the CLI report, and it
references detailed records instead of duplicating requests, responses, findings,
or error history. Capture configuration and resolved composition once when
available. Use native execution IDs with application scope to connect records.
Reused work has a checkpoint/reuse decision and reference, not a fabricated model
execution.
Retain enough available context to investigate a run that fails before publishing
output. Do not depend entirely on a successful output manifest or automatically
copy every original input and published artifact. Select a conversation
representation that avoids repeatedly saving the complete accumulated history
after each turn or maintaining an independent competing transcript.
### Open Discussion: Capture Depth
The organization is selected; precise evidence and granularity remain open:
- Which prepared request details, outputs, intermediate artifacts, validation
decisions, and correction content to retain.
- How to represent conversation history without repeated prefixes while using
PromptKit's native evidence.
- How much failed-run context to retain independently, and when references to
existing inputs, checkpoints, or published artifacts suffice.
- Which desired evidence is available through native public hooks and which
gaps would justify an upstream feature request.
Evidence: [native shared debug binding](../../../commandkit/docs/consumers/guide.md#invocation-reporting),
[reporting-mode ordering](../../../commandkit/docs/cli.md#modes-and-visibility),