166 lines
7.0 KiB
Markdown
166 lines
7.0 KiB
Markdown
# PromptKit Architecture Discussion
|
||
|
||
## Purpose And Status
|
||
|
||
Working notepad for reviewing the ownership boundaries in the
|
||
[PromptKit v0.23.2 adoption roadmap](promptkit.md). Work through the topics
|
||
below in order, recording questions, evidence, decisions, and follow-up work.
|
||
The six initial topics are proposals for discussion, not accepted changes.
|
||
The Audita comparison is a seventh topic and may inform earlier decisions.
|
||
|
||
The central question is how much reusable LLM machinery Notarius should retain
|
||
as PromptKit, ConfigKit, and CommandKit take on more responsibility. A working
|
||
principle to evaluate is that application ownership of a policy does not
|
||
necessarily require application ownership of its implementation.
|
||
|
||
No architectural changes have been agreed or implemented through this notepad.
|
||
The adoption roadmap remains the migration plan; reconcile it with decisions
|
||
from this discussion as they are made. Significant changes to accepted
|
||
architecture need successor ADRs under the
|
||
[documentation policy](../policy/documentation.md#architecture-decision-records).
|
||
Current boundaries are described in [Architecture](../policy/architecture.md)
|
||
and [LLM Runtime Internals](../internal/llm.md).
|
||
|
||
## 1. LLM API And Diagnostic Boundary
|
||
|
||
Consider using native PromptKit contracts in LLM-facing implementation packages
|
||
and reducing parallel completion, message, usage, execution-summary, and debug
|
||
types. Retain application contracts where they serve typed artifact handling,
|
||
stage context, durable output compatibility, or disclosure policy.
|
||
|
||
Questions:
|
||
|
||
- Which abstractions express Notarius behavior, and which merely copy PromptKit?
|
||
- Where should direct PromptKit dependencies be permitted?
|
||
- Can detailed debug material flow to an application-controlled writer without
|
||
being translated through generic pipeline contracts?
|
||
- Which persisted representations need compatibility or redaction adapters?
|
||
|
||
**Discussion / decision:** Pending. Start here.
|
||
|
||
**Follow-up:** Identify the smallest useful application boundary and any
|
||
architecture-policy changes it requires.
|
||
|
||
## 2. Prompt Assets And Schema Assembly
|
||
|
||
Consider replacing custom asset registration and virtual filesystem remounting
|
||
with native embedded sources, imports, collections, and source-relative assets.
|
||
Modules would continue to own prompt meaning and typed interpretation.
|
||
|
||
Questions:
|
||
|
||
- Can native sources represent the existing shared fragments and module assets?
|
||
- Which registry, remounting, and hashing helpers could disappear?
|
||
- What private schema identity is still needed for checkpoint reuse?
|
||
- How do we preserve message ordering and cache controls while simplifying
|
||
assembly?
|
||
|
||
**Discussion / decision:** Pending.
|
||
|
||
**Follow-up:** Separate private LLM schema plumbing from durable artifact
|
||
schemas; do not assume a captured prompt hash covers schema content.
|
||
|
||
## 3. Retry And Semantic Correction Ownership
|
||
|
||
Consider narrowing outer stage retries once PromptKit has exhausted its own
|
||
transport and structural-repair budgets. Notarius would retain application
|
||
assessment, semantic correction, fallback, and acceptance decisions.
|
||
|
||
Questions:
|
||
|
||
- Which operational failures justify repeating an entire stage?
|
||
- Should semantic correction preserve only the latest response or accumulate
|
||
conversation history?
|
||
- How should retries for non-LLM operations differ from LLM execution retries?
|
||
- Would a native orchestration API reduce machinery without obscuring typed
|
||
validation and terminal disposition?
|
||
|
||
**Discussion / decision:** Pending.
|
||
|
||
**Follow-up:** Evaluate behavior changes against
|
||
[ADR-0014](../adr/0014-feedback-aware-validation-retries.md) and
|
||
[ADR-0016](../adr/0016-feedback-aware-module-requested-retries.md).
|
||
PromptKit v0.23.2's `Coordinate` continues an exchange; it is not a direct
|
||
replacement for the existing latest-response-only policy or full stage loop.
|
||
|
||
## 4. Global Scheduling And Admission
|
||
|
||
Consider whether backend capacity limits and pipeline worker bounds meet the
|
||
operational need, or whether a separate cross-backend ceiling remains useful.
|
||
If it remains useful, consider implementing that reusable mechanism upstream.
|
||
|
||
Questions:
|
||
|
||
- What resource or operational budget does each limit protect?
|
||
- Should a global permit remain occupied during backend queueing and repair?
|
||
- Can the Notarius scheduler be removed, or should PromptKit gain engine-wide
|
||
admission support?
|
||
|
||
**Discussion / decision:** Pending.
|
||
|
||
**Follow-up:** Account for the existing global-limit contract. PromptKit
|
||
v0.23.2's per-backend limits do not directly replace it.
|
||
|
||
## 5. Execution Identity And Checkpoint Reuse
|
||
|
||
Consider moving identity for PromptKit-owned execution semantics upstream.
|
||
Notarius would combine opaque upstream identities with its source, module,
|
||
reference, schema, and policy identities and retain checkpoint eligibility and
|
||
storage decisions.
|
||
|
||
Questions:
|
||
|
||
- Which identity can be computed before a stage runs, and which needs a concrete
|
||
prepared request?
|
||
- Which changes should invalidate reuse, and at what granularity?
|
||
- Should a conservative application fingerprint be an interim implementation
|
||
rather than the architectural destination?
|
||
- Does this justify upstream work and a later adoption target?
|
||
|
||
**Discussion / decision:** Pending.
|
||
|
||
**Follow-up:** Review PromptKit's deferred semantic execution-target fingerprint
|
||
proposal. This capability is not supplied by the v0.23.2 adoption target.
|
||
|
||
## 6. Inspection And Execution Bookkeeping
|
||
|
||
Extend the roadmap's use of native inspection, `Run`, `OnPrepared`, observations,
|
||
and `ExecutionGroup` to reconsider surrounding wrappers and copied report types.
|
||
Notarius retains stage correlation, artifact provenance, storage, and disclosure
|
||
policy; CommandKit is the planned owner of standardized operator presentation.
|
||
|
||
Questions:
|
||
|
||
- Which reports and accounting mechanisms can use upstream results directly?
|
||
- Which application projections remain necessary for durable contracts?
|
||
- Can the PromptKit migration avoid building temporary machinery that the
|
||
CommandKit adoption would immediately replace?
|
||
|
||
**Discussion / decision:** Pending.
|
||
|
||
**Follow-up:** Coordinate this topic with item 1 and the roadmap's CommandKit
|
||
follow-up so report and adapter decisions are made once.
|
||
|
||
## 7. Compare Audita's PromptKit Workflow
|
||
|
||
Review the sibling Audita checkout at `../audita` relative to the Notarius
|
||
repository root. Audita uses PromptKit for a workflow the user identifies as
|
||
very similar, potentially identical. Verify the similarities and differences
|
||
before drawing architectural conclusions.
|
||
|
||
Questions:
|
||
|
||
- What are Audita's PromptKit version, execution flow, and ownership boundaries?
|
||
- How does it handle candidate assessment, correction, retries, scheduling,
|
||
configuration, inspection, and diagnostics?
|
||
- Which simpler patterns could Notarius adopt directly?
|
||
- Which differences arise from actual product requirements?
|
||
- Does duplication across the two applications reveal work that belongs in
|
||
PromptKit, ConfigKit, or CommandKit?
|
||
|
||
**Discussion / decision:** Pending; Audita has not yet been reviewed for this
|
||
comparison.
|
||
|
||
**Follow-up:** Record the reviewed Audita revision and evidence, then feed the
|
||
findings back into items 1–6 and the adoption roadmap.
|