base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Prepare v0.17.0 notes and retire completed roadmaps

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-03 02:56:48 +00:00
parent 32547e1e29
commit a0441fb2d1
6 changed files with 72 additions and 727 deletions
Download Patch File Download Diff File Expand all files Collapse all files

255
docs/roadmap/implementation.md
View File

@@ -1,255 +0,0 @@
# Go-Based Example Testing Implementation Plan
Implement [example-testing.md](example-testing.md) as one end-to-end sprint.
Each sequential stage below is a single prompt-sized assignment for a Luna
coding agent. The feature roadmap owns intent and scope; this document owns
the implementation steps and local design choices.
## Baseline And Working Rules
The reviewed baseline is `79dfd97afa2acc3cbf1ba6936b3a11b116702cf4`:
Go 1.27.0, module `go.maximumdirect.net/promptkit`, ConfigKit v1.0.0,
ReleaseKit v0.8.0, and three maintained programs under `examples/go-library/`.
There are no Go tests alongside those programs. `scripts/check-examples.py`
executes them and checks JSON results through the sole custom ReleaseKit hook.
`scripts/test-release.py` is a separate development-only diagnostic with a
synthetic replacement for that hook.
This plan replaces the completed ConfigKit migration plan. Its historical
instructions remain available with
`git show 79dfd97:docs/roadmap/implementation.md`; they are not assignments for
this sprint.
Read `docs/development.md` and the applicable agent, architecture, testing,
documentation, and release policies before work. Preserve unrelated changes.
Use the module-compatible local Go toolchain and prepare caches before offline
validation. No dependency, Go-minimum, consumer API, or source/hosted-release
workflow change is needed.
Intermediate stages may be incomplete or temporarily inconsistent: there is no
requirement to introduce compatibility scaffolding, compile every intermediate
state, or stop for human review. Finish all stages before claiming completion.
Run focused checks when useful; if a check depends on later work, record that
dependency rather than claiming it passed. Do not publish or contact providers.
Follow the agent policy for focused local commits.
## Implementation Choices
- Put tests in `main_test.go` with `package main` beside each example. Test
actual workflow and encoding paths directly, not reconstructed engine setups.
- For preparation and execution, extract
`run(ctx context.Context, exampleDir string, out io.Writer) error`.
`exampleDir` is that example's resource directory, not an application-wide
configuration root. Use `filepath.Join` to locate its existing files.
Each `main` passes its existing repository-relative example directory and
`os.Stdout`; tests pass `"."` and a buffer from the package directory.
- Observation already has `run() (summary, error)` and embedded assets. Keep
that workflow and add a small `execute(out io.Writer) error` which calls it
and encodes its result. Both `main` and its test use `execute`.
- Keep stderr reporting and nonzero exit handling in the thin `main` functions.
Workflow helpers return errors instead of exiting. Preserve current successful
summaries and existing example-side semantic checks.
- Use one representative happy-path workflow test per example to carry the
existing useful assertions. Decode JSON independently of production summary
tags: test-local structs or maps are sufficient. Check presence where a
missing field could masquerade as an expected false, zero, or null value.
Do not reject harmless extra fields, require exact JSON formatting, or add
whole-document golden snapshots.
- Use buffers and explicit resource directories rather than redirecting global
stdout or changing the process working directory. These are local design
choices for these examples, not new library contracts.
- No subprocess example test is needed for this baseline: the small extractions
cover the workflow and output without complicating the examples. Verify the
actual `go run` commands once during integration instead of adding redundant
executable tests to every default run.
## Stage 1: Test The Execution Example In Go
Scope: `examples/go-library/run/main.go` and a new `main_test.go` there.
1. Extract its current engine construction, injected deterministic client,
request, and summary encoding into `run(ctx, exampleDir, out)`. Resolve
`prompt.yaml` relative to `exampleDir`. Pass the supplied context to
`engine.Run` instead of creating another background context there.
2. Return construction, execution, and encoding errors. Keep the existing
indented JSON output. Make `main` call this helper with a background context,
`"examples/go-library/run"`, and stdout; retain stderr/exit handling at that
boundary. Remove obsolete error-exit helper code if no longer used.
3. Add `TestRunExample` calling the helper with `t.Context()`, `"."`, and a
`bytes.Buffer`. Decode and assert the current output:
- `output`: `Ada finished the migration review.`;
- `is_valid`: true;
- `validation_status`: `passed`;
- `model`: `offline-model`;
- `total_tokens`: 18.
4. Keep this local to the example: no new shared test framework or public API.
Validation: format changed Go files and run
`go test ./examples/go-library/run`. Confirm the test executes the workflow,
not just the summary type. These expected values describe this fixture only.
Completion: extracted the execution workflow and JSON encoding into a helper
with explicit context, resource directory, and writer. The focused Go test
executes the workflow and checks its summary; it passed on Go 1.27.0.
## Stage 2: Test Preparation With Explicit Resource Context
Scope: `examples/go-library/prepare/main.go` and its new `main_test.go`.
1. Extract the current composition, inspection, validation, preparation, and
summary encoding into the chosen `run(ctx, exampleDir, out)` helper. Replace
the three hard-coded file prefixes with paths under `exampleDir`; retain
the existing collections, source ordering, imports, and later override.
2. Preserve the inline checks for the discovered required `note` input, the
override explanation, and the effective timeout. Return their errors.
Keep those checks readable in the example instead of duplicating their
complete machinery in the test. Use the supplied context for engine calls.
3. Keep `main` callable through the existing repository-root command by passing
`"examples/go-library/prepare"` and stdout. Retain current indented encoding
and error presentation while removing exits from the workflow helper.
4. Add `TestPrepareExample` using the helper, package-local resources, and a
buffer. Assert these retained JSON results:
- `prompt_id`: `support/prepare`; `selected_profile`: `support/fast`;
- `model`: `offline-model`; `message_count` greater than zero;
- `profile_timeout_seconds`: 30;
- `namespaces`: `review`, then `support`;
- `collections`: review with an explicit null `release_version`, followed
by support with `release_version` equal to `1.4.0`;
- `scoped_prompts`: `support/prepare`; `review_prompts`: `review/check`.
Preserve the distinction between an omitted member and the explicit null
currently checked for the review collection.
Validation: format and run `go test ./examples/go-library/prepare`. Ensure
imports and relative content still resolve from their contributing sources;
do not rewrite configuration fixtures merely to bypass path handling.
Completion: the preparation workflow accepts an explicit context, example
directory, and writer while preserving its inline checks and source contexts.
Its Go test verifies the retained summary, including explicit null collection
metadata, and passed on Go 1.27.0.
## Stage 3: Test The Observation Example And Its Output
Scope: `examples/go-library/observe/main.go` and its new `main_test.go`.
1. Retain its existing `run`, embedded files, scripted HTTP transport, event
recorder, and application-owned assessment logic.
2. Extract only the workflow/encoding coordination into `execute(out)` as
specified above. It returns workflow or encoding errors. Make `main` call
it with stdout and retain the current stderr/exit behavior. Do not redesign
the transport or add a new client abstraction.
3. Add `TestObserveExample` calling `execute` with a buffer. Decode and assert:
- `proposal_attempts`: 2; `proposal_transient_retries`: 1;
- `proposal_usage_complete`: false;
- `assessment_attempts`: 1; `assessment_transient_retries`: 0;
- `assessment_outcome`: `accepted`.
Missing false/zero fields must not silently pass. Run the workflow once per
test rather than separately recomputing typed and encoded results.
Validation: format and run `go test ./examples/go-library/observe`, followed
by `go test ./examples/go-library/...`. Existing deterministic collaborators
remain the network boundary; no live provider or credential is needed.
Completion: observation workflow encoding now goes through a writer-based
helper shared by the executable and its Go test. The test checks all retained
summary fields, including false and zero presence; the observation and complete
example package tests passed on Go 1.27.0.
## Stage 4: Remove The Custom Release Hook And Adapt Its Diagnostic
Scope: `scripts/check-examples.py`, `releasekit.json`, and
`scripts/test-release.py`.
1. Delete the Python example runner after the Go tests carry its useful
coverage. Remove the profile's `checks` property and its example-specific
environment overlay. Retain every existing built-in validator, delivery
setting, and token-file protection. Do not add a replacement custom command.
2. In the release-wiring fixture, remove the generated example-check stub,
`FIXTURE_CHECK_MARKER`, `FIXTURE_CHECK_FAILURE`, related environment setup,
and assertions about invocation of that removed hook. Update its docstring
to describe launcher/profile wiring, not product-check dispatch.
3. Retain bootstrap/version/build-info checks, native usage status 2, successful
publication of the intended local fixture candidate without delivery
credentials, selected race/local-Go verification, token-path selection and
ignore checks, and module/worktree preservation.
4. Preserve a small status-1 wrapper check without manufacturing a product
hook: create only the successful version's note in the fixture, then invoke
the publication wrapper with `FAILED_VERSION`, whose note is absent.
Expect status 1 and retain the remote-tag absence check. This exercises
forwarding a native candidate failure, not a new publication algorithm.
5. Keep this diagnostic development-only and in Python. Do not add it to the
default Go suite or the release profile. The native launchers are unchanged.
Validation: run `python3 scripts/test-release.py` using the selected Go on
`PATH` and prepared caches. Its remotes must remain disposable local bare
repositories. Check the profile diff to confirm only the obsolete custom hook
was removed. Do not test by publishing a real tag or hosted page.
Completion: removed the Python example runner and its custom profile check;
the release-wiring diagnostic now tests the native missing-note failure and
preserves its other launcher, suite, publication, and token-path checks. The
diagnostic passed with Go 1.27.0 against disposable local repositories, and the
profile diff retains the existing built-in validators and delivery settings.
## Stage 5: Align Documentation And Verify The Complete Sprint
Scope: the development guide, local testing/release policies, release runbook,
affected example guidance, and integration verification.
1. Update `docs/development.md` to use normal Go testing for example validation.
Offer `go test ./examples/go-library/...` for targeted development without
making it a separate required release pass. Keep the Python release-wiring
command documented as a focused optional diagnostic.
2. Update the Promptkit-specific sections of `docs/policy/testing.md` and
`docs/policy/release.md`: example assertions now run inside the existing Go
suite, not after it as a custom check. Keep the examples' distinct coverage
ownership, all other selected checks, and the publication completion rules.
3. Remove Python from normal release prerequisites in `docs/release.md`.
Preserve Go selection, credentials, source publication, hosted delivery, and
recovery. Update any other current guidance pointing to the deleted runner.
Historical release notes remain unchanged.
4. Update the feature roadmap's completion/baseline text when recording the
implemented result. Remove its now-dead Markdown link to the deleted Python
file, using a plain historical path if still useful. Avoid leaving a broken
link in a completed coordination document. Do not retire unrelated roadmaps.
5. Format changed Go files, review the full diff, and run the integrated checks
below with the same compatible local Go toolchain:
```sh
go test -race -vet=off ./...
go vet ./...
go run ./examples/go-library/prepare
go run ./examples/go-library/run
go run ./examples/go-library/observe
python3 scripts/test-release.py
git diff --check
```
Run the three programs from the repository root. Confirm their outputs
retain the meanings checked by the new tests; JSON spacing is not a new
contract. Reuse successful focused checks for unchanged content rather than
repeating them solely to finish a stage. The separate full vet invocation
above pairs with `-vet=off` to avoid duplicate vet work.
6. As a one-time proof, temporarily make one meaningful expected value wrong in
an example test, run its package test, and confirm failure. Restore the value
and rerun the affected package before final review. Do not commit that probe
or add a permanent self-test of the testing framework.
7. Search current code/configuration/guidance for obsolete runner and hook
references, verify local documentation links, and confirm `go.mod`, `go.sum`,
public library code, and the native release launchers were not changed by
this tooling migration. Commit the focused completed work under agent policy.
Completion requires all three workflows to have real Go test coverage,
successful ordinary suite discovery, preserved useful assertions, removal of
the runner and custom hook, and accurate current documentation. No performance,
coverage-percentage, platform, or release-service gate is added. There are no
unresolved product decisions or upstream publication dependencies for this sprint.
Completion: all three examples have Go workflow and JSON checks, and the
obsolete runner and custom release hook have been removed. Updated the current
development, testing, and release guidance and marked the feature roadmap
complete. On Go 1.27.0, the full race-enabled suite, full vet, all three
repository-root example commands, and the disposable release-wiring diagnostic
passed. A deliberately incorrect example expectation failed before restoration,
then the affected package passed again. Reviewed links, the profile, formatting,
module and launcher changes, and `git diff --check`.