From c7d43181221d56af0da8e0bd4856859362d7fe4a Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Tue, 3 Nov 2026 02:31:35 +0000 Subject: [PATCH] Update guidance for Go example tests --- docs/development.md | 13 +++++----- docs/policy/release.md | 6 ++--- docs/policy/testing.md | 6 ++--- docs/release.md | 4 ++-- docs/roadmap/example-testing.md | 42 ++++++++++++++++----------------- docs/roadmap/implementation.md | 9 +++++++ 6 files changed, 43 insertions(+), 37 deletions(-) diff --git a/docs/development.md b/docs/development.md index 5b0fc8b..4b0e30e 100644 --- a/docs/development.md +++ b/docs/development.md @@ -67,7 +67,6 @@ the maintained examples when they are affected: go test ./... go vet ./... gofmt -w path/to/changed.go -python3 scripts/check-examples.py git diff --check ``` @@ -77,12 +76,12 @@ describes its requirements and recovery. For an optional diagnostic of a committed candidate without publication, run `./scripts/releasekit.sh check VERSION`; this is not a required pre-publication step. -The standalone `python3 scripts/check-examples.py` command checks semantic -expectations for the three offline consumer examples. The focused -`python3 scripts/test-release.py` command exercises the release tooling against -disposable local repositories; use it when changing that tooling, not as a -recurring release gate. Tests and examples require no provider credentials or -live inference calls. +The example workflows are covered by the Go suite; use +`go test ./examples/go-library/...` to run them separately during development. +The focused `python3 scripts/test-release.py` command exercises the release +tooling against disposable local repositories; use it when changing that +tooling, not as a recurring release gate. Tests and examples require no provider +credentials or live inference calls. Ordinary validation can run on intended uncommitted work. Release validation uses committed candidate contents and the selected local Go installation. diff --git a/docs/policy/release.md b/docs/policy/release.md index b54d47e..9814d67 100644 --- a/docs/policy/release.md +++ b/docs/policy/release.md @@ -271,9 +271,9 @@ The [publication command](../../scripts/release.sh) delegates to the released ReleaseKit tool selected in `go.mod`. The committed [profile](../../releasekit.json) selects the race-enabled Go suite, full vet, committed Go formatting, local Markdown links, workspace and filesystem -replacement inspection, and the offline semantic checks for the three -maintained consumer examples. It also rejects the configured local delivery -token path if committed; this narrowly scoped check is not a general credential +replacement inspection. The Go suite includes the offline semantic checks for +the three maintained consumer examples. It also rejects the configured local +delivery token path if committed; this narrowly scoped check is not a general credential scan. These are the selected checks for the final candidate; a separate ordinary test run or standalone build is not an additional release gate. The upstream tool owns candidate, remote, and tag diff --git a/docs/policy/testing.md b/docs/policy/testing.md index 791305a..361e75c 100644 --- a/docs/policy/testing.md +++ b/docs/policy/testing.md @@ -60,10 +60,10 @@ documented validation model. The selected ReleaseKit profile runs the race-enabled package suite and full vet, checks committed Go formatting, local Markdown links, and module -distribution hygiene, and executes semantic assertions for the three +distribution hygiene. The Go suite includes semantic assertions for the three maintained offline consumer examples. ReleaseKit owns this shared validation -machinery and publication workflow. The test run builds the packages; the -examples also exercise ordinary compilation. Separate ordinary tests and a +machinery and publication workflow. The test run builds the packages and +exercises the examples. Separate ordinary tests and a standalone build are not additional release gates. Ordinary tests remain useful during development. The preparation example protects assembled preparation and inspection behavior. The execution example separately protects assembled diff --git a/docs/release.md b/docs/release.md index 656ca7b..767bb58 100644 --- a/docs/release.md +++ b/docs/release.md @@ -8,8 +8,8 @@ delivery separately. Do not wait for CI or Go module availability. ## Prepare The Candidate -Required tools are Git, Go with race-detector support, POSIX shell, and Python -3. Work from the repository root with push access to `origin`. +Required tools are Git, Go with race-detector support, and POSIX shell. Work +from the repository root with push access to `origin`. Use the latest Go installation available locally for the release tool and its child commands. Put it first on `PATH` and keep it selected throughout: diff --git a/docs/roadmap/example-testing.md b/docs/roadmap/example-testing.md index 492b48f..767d926 100644 --- a/docs/roadmap/example-testing.md +++ b/docs/roadmap/example-testing.md @@ -1,5 +1,8 @@ # Go-Based Testing For Maintained Examples +Status: complete. Example assertions now run in the ordinary Go suite; the +custom release check and its Python runner have been removed. + ## Purpose Move the maintained consumer-example checks into Promptkit's ordinary Go test @@ -15,7 +18,7 @@ library API or a new release gate. The companion [implementation plan](implementation.md) defines the staged work for reaching this target. -## Current Baseline +## Previous Baseline The [preparation](../../examples/go-library/prepare/main.go), [execution](../../examples/go-library/run/main.go), and @@ -25,15 +28,14 @@ providers. Preparation and execution use repository-relative configuration paths; observation embeds its assets and already has a workflow helper returning a summary and error. -[scripts/check-examples.py](../../scripts/check-examples.py) launches all three -programs, decodes their JSON summaries, and checks meaningful results. Ordinary -`go test ./...` compiles the example packages but currently has no example tests -to execute. The [ReleaseKit profile](../../releasekit.json) runs the Python -script as its one custom check after shared validation. +The former `scripts/check-examples.py` launched all three programs and checked +their JSON summaries. Ordinary `go test ./...` then had no example tests, and +the [ReleaseKit profile](../../releasekit.json) selected the Python script as a +custom check after shared validation. The separate [release-wiring test](../../scripts/test-release.py) is also Python, but is a development-only diagnostic, not a release gate. Its disposable fixture -currently assumes the example script is a configured product check. +previously assumed the example script was a configured product check. ## Target State @@ -86,25 +88,21 @@ implementation dependencies, consistent with the ## Integration And Documentation Work -Once the Go tests cover the retained behavior, remove `scripts/check-examples.py` -and its custom command and environment overlay from `releasekit.json`. Keep the -existing built-in validation selections and source/hosted publication workflow. -There is no need for new ReleaseKit functionality or a dependency upgrade. +The Go tests cover the retained behavior; the former `scripts/check-examples.py` +and its custom command and environment overlay have been removed. The existing +built-in validation selections and source/hosted publication workflow remain, +without a ReleaseKit dependency upgrade. -Adapt the development-only release-wiring fixture so it no longer expects the -removed script or product-check marker. Retain checks relevant to the actual -launcher and profile; do not manufacture an unused custom hook merely to retain -old fixture assertions. The Python release-wiring diagnostic can remain Python. +The development-only release-wiring fixture no longer expects the removed script +or product-check marker. It retains checks for the actual launcher and profile +without manufacturing an unused custom hook. The diagnostic remains Python. Removing Python from every repository tool is outside this scope. -Update the [development guide](../development.md), +The [development guide](../development.md), [testing policy](../policy/testing.md), [release policy](../policy/release.md), -and [release procedure](../release.md) when implementation lands. Replace the -standalone example command with Go test guidance, describe example coverage as -part of the standard suite, and remove Python as a normal release prerequisite. -Continue documenting Python where the optional release-wiring test needs it. -Check other incoming references to the removed runner and retain the existing -consumer quickstart commands. +and [release procedure](../release.md) describe Go test coverage as part of the +standard suite and retain Python only for the optional release-wiring test. The +existing consumer quickstart commands remain available. ## Verification diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 1d80659..899643d 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -244,3 +244,12 @@ successful ordinary suite discovery, preserved useful assertions, removal of the runner and custom hook, and accurate current documentation. No performance, coverage-percentage, platform, or release-service gate is added. There are no unresolved product decisions or upstream publication dependencies for this sprint. + +Completion: all three examples have Go workflow and JSON checks, and the +obsolete runner and custom release hook have been removed. Updated the current +development, testing, and release guidance and marked the feature roadmap +complete. On Go 1.27.0, the full race-enabled suite, full vet, all three +repository-root example commands, and the disposable release-wiring diagnostic +passed. A deliberately incorrect example expectation failed before restoration, +then the affected package passed again. Reviewed links, the profile, formatting, +module and launcher changes, and `git diff --check`.