Record reusable LLM flags and scoped convenience aliases
This commit is contained in:
@@ -84,7 +84,11 @@ The [PromptKit v0.19.0 roadmap](promptkit-v0.19.0.md#targeted-end-state)
|
|||||||
now owns concrete request preview, applicable request flags, namespace selection,
|
now owns concrete request preview, applicable request flags, namespace selection,
|
||||||
and the shared configuration inspection extensions. Its execution flags will be
|
and the shared configuration inspection extensions. Its execution flags will be
|
||||||
public reusable `ckprompt` collections for standard and application-owned CLI
|
public reusable `ckprompt` collections for standard and application-owned CLI
|
||||||
commands, composed through existing core bindings. These are CLI features;
|
commands, composed through existing core bindings.
|
||||||
|
[Roadmap OD-2](promptkit-v0.19.0.md#od-2-request-controls-and-provider-parameter-grammar)
|
||||||
|
records the agreed canonical naming, scoped convenience aliases, and integration
|
||||||
|
installation collection; no core extension is required. Additional request-input
|
||||||
|
features remain open for individual review. These are CLI features;
|
||||||
general-purpose helpers for application-internal PromptKit use are outside
|
general-purpose helpers for application-internal PromptKit use are outside
|
||||||
CommandKit's ownership boundary.
|
CommandKit's ownership boundary.
|
||||||
|
|
||||||
|
|||||||
@@ -95,7 +95,7 @@ in the decisions at the end of this document.
|
|||||||
| Operator task | Proposed CLI | Owning module and report |
|
| Operator task | Proposed CLI | Owning module and report |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| Inspect what a concrete prompt request prepares | `promptkit show prepared PROMPT_ID --profile … --input … --var …` | `ckprompt`: YAML report of selected IDs, effective settings, rendered/appended messages, output contract and schema, session, hashes, and timing from `PreparedRun`. |
|
| Inspect what a concrete prompt request prepares | `promptkit show prepared PROMPT_ID --profile … --input … --var …` | `ckprompt`: YAML report of selected IDs, effective settings, rendered/appended messages, output contract and schema, session, hashes, and timing from `PreparedRun`. |
|
||||||
| Reproduce target/provider settings without editing configuration | Applicable existing smoke override flags on validation and preview; `--extra-params JSON` on these and smoke operations | `ckprompt`: native request forwarding; preview and smoke reports show returned effective settings. |
|
| Reproduce target/provider settings without editing configuration | Applicable existing smoke override flags on validation and preview; `--llm-extra-params JSON` on these and smoke operations | `ckprompt`: native request forwarding; preview and smoke reports show returned effective settings. |
|
||||||
| Resolve unqualified resource selections in chosen namespaces | Independent prompt/profile/backend namespace flags | `ckprompt`: delegate to native namespace views, retaining the distinction from catalog filters. |
|
| Resolve unqualified resource selections in chosen namespaces | Independent prompt/profile/backend namespace flags | `ckprompt`: delegate to native namespace views, retaining the distinction from catalog filters. |
|
||||||
| Find configuration paths and inspect their explicitness | `config explain --list --scope promptkit`, with historical selection; metadata alongside path explanation | `ckconfig`, wired through `ckprompt.ConfigScope`: observed paths, effective presence, explicitness/null/origin evidence, and available history. |
|
| Find configuration paths and inspect their explicitness | `config explain --list --scope promptkit`, with historical selection; metadata alongside path explanation | `ckconfig`, wired through `ckprompt.ConfigScope`: observed paths, effective presence, explicitness/null/origin evidence, and available history. |
|
||||||
| Understand both sides of configuration changes | `config diff --scope promptkit --details` | `ckconfig`: native paths, left/right snapshots, presence/redaction, and hidden-change evidence. |
|
| Understand both sides of configuration changes | `config diff --scope promptkit --details` | `ckconfig`: native paths, left/right snapshots, presence/redaction, and hidden-change evidence. |
|
||||||
@@ -185,9 +185,8 @@ have different native retry defaults. Preserve native explicit zero/empty
|
|||||||
values and whole-map replacement semantics. Support deliberate subsets for
|
values and whole-map replacement semantics. Support deliberate subsets for
|
||||||
actual command needs while retaining common flag names and meanings, using a
|
actual command needs while retaining common flag names and meanings, using a
|
||||||
proportionate composition API. Keep parsing in CommandKit and domain validity
|
proportionate composition API. Keep parsing in CommandKit and domain validity
|
||||||
in PromptKit. Existing application builders remain usable; resolve override
|
in PromptKit. Existing application builders remain authoritative as agreed in
|
||||||
precedence and the remaining request controls under OD-2 before documenting new
|
OD-2; additional request-input controls remain open for individual review.
|
||||||
public behavior.
|
|
||||||
|
|
||||||
### Namespace selection
|
### Namespace selection
|
||||||
|
|
||||||
@@ -299,8 +298,10 @@ errors remain errors even if an override would replace the affected value.
|
|||||||
3. **Implement request inspection and selection.** Add preview, applicable
|
3. **Implement request inspection and selection.** Add preview, applicable
|
||||||
override flags, provider-parameter input, and namespace selectors. Implement
|
override flags, provider-parameter input, and namespace selectors. Implement
|
||||||
public `ckprompt` execution-flag collections on existing core bindings and
|
public `ckprompt` execution-flag collections on existing core bindings and
|
||||||
reuse them in the standard commands. Reuse the standard suite's lifecycle
|
reuse them in the standard commands. Generate scoped convenience aliases and
|
||||||
and preserve existing application adapter seams.
|
provide a collection for installing the commands and routing declarations.
|
||||||
|
Reuse the standard suite's lifecycle and preserve existing application adapter
|
||||||
|
seams.
|
||||||
4. **Implement shared configuration inspection.** Add discovery and metadata
|
4. **Implement shared configuration inspection.** Add discovery and metadata
|
||||||
within `config explain`, structured diagnostics within `config validate`,
|
within `config explain`, structured diagnostics within `config validate`,
|
||||||
detailed diff presentation, and source evidence on failure in `ckconfig`.
|
detailed diff presentation, and source evidence on failure in `ckconfig`.
|
||||||
@@ -349,6 +350,15 @@ Default tests remain offline and independent of real credentials.
|
|||||||
metadata, and selected subsets remain consistent; omitted retry values retain
|
metadata, and selected subsets remain consistent; omitted retry values retain
|
||||||
the applicable native defaults. Application execution behavior remains outside
|
the applicable native defaults. Application execution behavior remains outside
|
||||||
the collection.
|
the collection.
|
||||||
|
- [ ] Canonical `--llm-` flags and scoped convenience aliases follow OD-2.
|
||||||
|
Alias generation covers only enabled commands and applicable flags, shares
|
||||||
|
canonical binding/presence behavior, and preserves mixed-spelling occurrence
|
||||||
|
order. Generic spellings remain available to unrelated application commands.
|
||||||
|
- [ ] The convenient installation collection includes commands and their scoped
|
||||||
|
aliases; canonical-only installation remains available. Help and migration
|
||||||
|
guidance explain both forms without requiring consumer-maintained alias lists.
|
||||||
|
- [ ] Output-contract, inline-input, and appended-message options each receive
|
||||||
|
an explicit scope decision before implementation.
|
||||||
- [ ] Namespace defaults, explicit clearing, qualified selections, and catalog
|
- [ ] Namespace defaults, explicit clearing, qualified selections, and catalog
|
||||||
filters are forwarded according to the selected design.
|
filters are forwarded according to the selected design.
|
||||||
- [ ] The full configuration suite retains six subcommands. Explanation
|
- [ ] The full configuration suite retains six subcommands. Explanation
|
||||||
@@ -446,32 +456,83 @@ text presentation may be considered later; it is outside the initial scope.
|
|||||||
|
|
||||||
### OD-2: Request controls and provider-parameter grammar
|
### OD-2: Request controls and provider-parameter grammar
|
||||||
|
|
||||||
**Agreed design direction:** execution flags are public reusable CLI collections
|
**Agreed:** execution flags are public reusable CLI collections owned by
|
||||||
owned by `ckprompt`, available to standard and application-owned commands using
|
`ckprompt`, available to standard and application-owned commands using existing
|
||||||
existing core APIs. The remaining decisions here concern request controls and
|
core APIs. Give validation and preview the applicable execution controls already
|
||||||
grammar, not whether application commands may reuse the collections.
|
on smoke tests, and add a singleton `--llm-extra-params JSON` object option to
|
||||||
|
all three. Use native whole-map replacement, including native behavior for
|
||||||
**Recommendation:** give validation and preview the applicable execution flags
|
|
||||||
already on smoke tests, and add a singleton `--extra-params JSON` object option
|
|
||||||
to all three. Use native whole-map replacement, including native behavior for
|
|
||||||
empty maps. Specify JSON number decoding deliberately so supported values reach
|
empty maps. Specify JSON number decoding deliberately so supported values reach
|
||||||
PromptKit without accidental precision loss or new numeric guarantees. Retain
|
PromptKit without accidental precision loss or new numeric guarantees. The
|
||||||
the current authoritative-builder rule: the standard flags seed a request and
|
single-object form is selected over repeated key assignments.
|
||||||
an application builder can return its complete replacement.
|
|
||||||
|
|
||||||
**Alternative:** repeatable `--extra-param KEY=JSON` assignments. This is easier
|
Retain the authoritative-builder rule: standard flags seed a request, and an
|
||||||
for individual shell overrides, but requires rules for duplicate keys and makes
|
application builder can return its complete replacement. Do not reapply CLI
|
||||||
whole-map replacement less obvious. It must still construct a single native map
|
values after the builder returns.
|
||||||
rather than imply an upstream field-by-field merge.
|
|
||||||
|
|
||||||
**Remaining request surface:** recommend keeping output-contract overrides,
|
**Agreed naming and aliases:** use `--llm-` canonical names throughout the
|
||||||
inline-input flags, and appended-message flags as separately selectable CLI
|
reusable request-control family. Standard `promptkit` commands also accept the
|
||||||
extensions after execution-flag parity. Existing file inputs and application
|
unprefixed convenience spellings wherever the corresponding flag applies.
|
||||||
builders continue to support preview. A viable alternative is to include a
|
Application-owned commands receive canonical names from the reusable collections;
|
||||||
whole-output-contract file option and message/input options now, supporting
|
applications may deliberately install convenience aliases on their own commands.
|
||||||
more reproduction workflows at the cost of additional syntax, precedence, and
|
|
||||||
reporting decisions. No general-purpose request-building API follows from
|
| Canonical flag | Convenience alias within applicable `promptkit` commands |
|
||||||
either choice.
|
| --- | --- |
|
||||||
|
| `--llm-endpoint` | `--endpoint` |
|
||||||
|
| `--llm-model` | `--model` |
|
||||||
|
| `--llm-temperature` | `--temperature` |
|
||||||
|
| `--llm-max-tokens` | `--max-tokens` |
|
||||||
|
| `--llm-top-p` | `--top-p` |
|
||||||
|
| `--llm-timeout-seconds` | `--timeout-seconds` |
|
||||||
|
| `--llm-service-tier` | `--service-tier` |
|
||||||
|
| `--llm-reasoning-effort` | `--reasoning-effort` |
|
||||||
|
| `--llm-api-key-env` | `--api-key-env` |
|
||||||
|
| `--llm-transient-retries` | `--transient-retries` |
|
||||||
|
| `--llm-extra-params` | `--extra-params` |
|
||||||
|
| `--llm-backend` | `--backend` |
|
||||||
|
| `--llm-profile` | `--profile` |
|
||||||
|
| `--llm-prompt` | `--prompt` |
|
||||||
|
| `--llm-session-id` | `--session-id` |
|
||||||
|
| `--llm-var` | `--var` |
|
||||||
|
| `--llm-input` | `--input` |
|
||||||
|
|
||||||
|
Retain existing value syntax and units, including integer timeout seconds and
|
||||||
|
file-backed input bindings. Commands taking a positional prompt ID retain that
|
||||||
|
argument; this naming decision does not add a redundant prompt flag. Execution,
|
||||||
|
resource-selection, and request-input collections may remain separately
|
||||||
|
composable. Namespace controls remain subject to OD-3.
|
||||||
|
|
||||||
|
**Agreed assembly approach:** keep alias generation in `ckprompt`; no core
|
||||||
|
upgrade is required. Define each flag-to-alias relationship once and generate
|
||||||
|
existing `FlagAliasFor` routing declarations from the flags selected for each
|
||||||
|
enabled canonical command. Do not maintain a separate hand-written matrix of
|
||||||
|
all command/flag combinations. Aliases target the original flag binding rather
|
||||||
|
than introducing duplicate bindings. Both spellings retain core parsing,
|
||||||
|
presence, and occurrence-order behavior.
|
||||||
|
|
||||||
|
Offer a convenient collection containing the command family and its scoped
|
||||||
|
routing declarations, using existing `[]commandkit.Declaration` installation.
|
||||||
|
Keep `ckprompt.Command(suite)` available for canonical commands without the
|
||||||
|
convenience aliases. Exact helper names remain implementation choices. Generate
|
||||||
|
aliases only for enabled commands and applicable flags; do not install
|
||||||
|
application-wide aliases or make consumers reproduce the command/flag matrix.
|
||||||
|
Applications can install selected aliases on their own commands through the
|
||||||
|
existing core API. Subtree alias scopes and a new core declaration-bundle type
|
||||||
|
are not part of this adoption.
|
||||||
|
|
||||||
|
Help should identify canonical names and their applicable aliases using current
|
||||||
|
core presentation. Review published flag contracts during implementation and
|
||||||
|
document migration, including retention of existing unprefixed spellings through
|
||||||
|
the convenience collection. Do not assume canonical-only installation preserves
|
||||||
|
those spellings.
|
||||||
|
|
||||||
|
**Remaining request surface:** discuss output-contract overrides, inline-input
|
||||||
|
flags, and appended-message flags individually before selecting scope and syntax.
|
||||||
|
Agreement to discuss these separately does not yet include or defer them.
|
||||||
|
Existing file inputs and application builders continue to support preview. A
|
||||||
|
whole-output-contract file option and additional message/input options could
|
||||||
|
support more reproduction workflows, with corresponding syntax, precedence,
|
||||||
|
and reporting decisions. No general-purpose request-building API follows from
|
||||||
|
these CLI choices.
|
||||||
|
|
||||||
### OD-3: Namespace flags and loader defaults
|
### OD-3: Namespace flags and loader defaults
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user