Update guidance for Go example tests

This commit is contained in:
2026-11-03 02:31:35 +00:00
parent 175140ae5a
commit c7d4318122
6 changed files with 43 additions and 37 deletions

View File

@@ -67,7 +67,6 @@ the maintained examples when they are affected:
go test ./... go test ./...
go vet ./... go vet ./...
gofmt -w path/to/changed.go gofmt -w path/to/changed.go
python3 scripts/check-examples.py
git diff --check git diff --check
``` ```
@@ -77,12 +76,12 @@ describes its requirements and recovery. For an optional diagnostic of a
committed candidate without publication, run `./scripts/releasekit.sh check committed candidate without publication, run `./scripts/releasekit.sh check
VERSION`; this is not a required pre-publication step. VERSION`; this is not a required pre-publication step.
The standalone `python3 scripts/check-examples.py` command checks semantic The example workflows are covered by the Go suite; use
expectations for the three offline consumer examples. The focused `go test ./examples/go-library/...` to run them separately during development.
`python3 scripts/test-release.py` command exercises the release tooling against The focused `python3 scripts/test-release.py` command exercises the release
disposable local repositories; use it when changing that tooling, not as a tooling against disposable local repositories; use it when changing that
recurring release gate. Tests and examples require no provider credentials or tooling, not as a recurring release gate. Tests and examples require no provider
live inference calls. credentials or live inference calls.
Ordinary validation can run on intended uncommitted work. Release validation Ordinary validation can run on intended uncommitted work. Release validation
uses committed candidate contents and the selected local Go installation. uses committed candidate contents and the selected local Go installation.

View File

@@ -271,9 +271,9 @@ The [publication command](../../scripts/release.sh) delegates to the released
ReleaseKit tool selected in `go.mod`. The committed ReleaseKit tool selected in `go.mod`. The committed
[profile](../../releasekit.json) selects the race-enabled Go suite, full vet, [profile](../../releasekit.json) selects the race-enabled Go suite, full vet,
committed Go formatting, local Markdown links, workspace and filesystem committed Go formatting, local Markdown links, workspace and filesystem
replacement inspection, and the offline semantic checks for the three replacement inspection. The Go suite includes the offline semantic checks for
maintained consumer examples. It also rejects the configured local delivery the three maintained consumer examples. It also rejects the configured local
token path if committed; this narrowly scoped check is not a general credential delivery token path if committed; this narrowly scoped check is not a general credential
scan. These are the selected checks for the final candidate; a separate scan. These are the selected checks for the final candidate; a separate
ordinary test run or standalone build is not an ordinary test run or standalone build is not an
additional release gate. The upstream tool owns candidate, remote, and tag additional release gate. The upstream tool owns candidate, remote, and tag

View File

@@ -60,10 +60,10 @@ documented validation model.
The selected ReleaseKit profile runs the race-enabled package suite and full The selected ReleaseKit profile runs the race-enabled package suite and full
vet, checks committed Go formatting, local Markdown links, and module vet, checks committed Go formatting, local Markdown links, and module
distribution hygiene, and executes semantic assertions for the three distribution hygiene. The Go suite includes semantic assertions for the three
maintained offline consumer examples. ReleaseKit owns this shared validation maintained offline consumer examples. ReleaseKit owns this shared validation
machinery and publication workflow. The test run builds the packages; the machinery and publication workflow. The test run builds the packages and
examples also exercise ordinary compilation. Separate ordinary tests and a exercises the examples. Separate ordinary tests and a
standalone build are not additional release gates. Ordinary tests remain useful standalone build are not additional release gates. Ordinary tests remain useful
during development. The preparation example protects assembled preparation and during development. The preparation example protects assembled preparation and
inspection behavior. The execution example separately protects assembled inspection behavior. The execution example separately protects assembled

View File

@@ -8,8 +8,8 @@ delivery separately. Do not wait for CI or Go module availability.
## Prepare The Candidate ## Prepare The Candidate
Required tools are Git, Go with race-detector support, POSIX shell, and Python Required tools are Git, Go with race-detector support, and POSIX shell. Work
3. Work from the repository root with push access to `origin`. from the repository root with push access to `origin`.
Use the latest Go installation available locally for the release tool and its Use the latest Go installation available locally for the release tool and its
child commands. Put it first on `PATH` and keep it selected throughout: child commands. Put it first on `PATH` and keep it selected throughout:

View File

@@ -1,5 +1,8 @@
# Go-Based Testing For Maintained Examples # Go-Based Testing For Maintained Examples
Status: complete. Example assertions now run in the ordinary Go suite; the
custom release check and its Python runner have been removed.
## Purpose ## Purpose
Move the maintained consumer-example checks into Promptkit's ordinary Go test Move the maintained consumer-example checks into Promptkit's ordinary Go test
@@ -15,7 +18,7 @@ library API or a new release gate.
The companion [implementation plan](implementation.md) defines the staged work The companion [implementation plan](implementation.md) defines the staged work
for reaching this target. for reaching this target.
## Current Baseline ## Previous Baseline
The [preparation](../../examples/go-library/prepare/main.go), The [preparation](../../examples/go-library/prepare/main.go),
[execution](../../examples/go-library/run/main.go), and [execution](../../examples/go-library/run/main.go), and
@@ -25,15 +28,14 @@ providers. Preparation and execution use repository-relative configuration
paths; observation embeds its assets and already has a workflow helper returning paths; observation embeds its assets and already has a workflow helper returning
a summary and error. a summary and error.
[scripts/check-examples.py](../../scripts/check-examples.py) launches all three The former `scripts/check-examples.py` launched all three programs and checked
programs, decodes their JSON summaries, and checks meaningful results. Ordinary their JSON summaries. Ordinary `go test ./...` then had no example tests, and
`go test ./...` compiles the example packages but currently has no example tests the [ReleaseKit profile](../../releasekit.json) selected the Python script as a
to execute. The [ReleaseKit profile](../../releasekit.json) runs the Python custom check after shared validation.
script as its one custom check after shared validation.
The separate [release-wiring test](../../scripts/test-release.py) is also Python, The separate [release-wiring test](../../scripts/test-release.py) is also Python,
but is a development-only diagnostic, not a release gate. Its disposable fixture but is a development-only diagnostic, not a release gate. Its disposable fixture
currently assumes the example script is a configured product check. previously assumed the example script was a configured product check.
## Target State ## Target State
@@ -86,25 +88,21 @@ implementation dependencies, consistent with the
## Integration And Documentation Work ## Integration And Documentation Work
Once the Go tests cover the retained behavior, remove `scripts/check-examples.py` The Go tests cover the retained behavior; the former `scripts/check-examples.py`
and its custom command and environment overlay from `releasekit.json`. Keep the and its custom command and environment overlay have been removed. The existing
existing built-in validation selections and source/hosted publication workflow. built-in validation selections and source/hosted publication workflow remain,
There is no need for new ReleaseKit functionality or a dependency upgrade. without a ReleaseKit dependency upgrade.
Adapt the development-only release-wiring fixture so it no longer expects the The development-only release-wiring fixture no longer expects the removed script
removed script or product-check marker. Retain checks relevant to the actual or product-check marker. It retains checks for the actual launcher and profile
launcher and profile; do not manufacture an unused custom hook merely to retain without manufacturing an unused custom hook. The diagnostic remains Python.
old fixture assertions. The Python release-wiring diagnostic can remain Python.
Removing Python from every repository tool is outside this scope. Removing Python from every repository tool is outside this scope.
Update the [development guide](../development.md), The [development guide](../development.md),
[testing policy](../policy/testing.md), [release policy](../policy/release.md), [testing policy](../policy/testing.md), [release policy](../policy/release.md),
and [release procedure](../release.md) when implementation lands. Replace the and [release procedure](../release.md) describe Go test coverage as part of the
standalone example command with Go test guidance, describe example coverage as standard suite and retain Python only for the optional release-wiring test. The
part of the standard suite, and remove Python as a normal release prerequisite. existing consumer quickstart commands remain available.
Continue documenting Python where the optional release-wiring test needs it.
Check other incoming references to the removed runner and retain the existing
consumer quickstart commands.
## Verification ## Verification

View File

@@ -244,3 +244,12 @@ successful ordinary suite discovery, preserved useful assertions, removal of
the runner and custom hook, and accurate current documentation. No performance, the runner and custom hook, and accurate current documentation. No performance,
coverage-percentage, platform, or release-service gate is added. There are no coverage-percentage, platform, or release-service gate is added. There are no
unresolved product decisions or upstream publication dependencies for this sprint. unresolved product decisions or upstream publication dependencies for this sprint.
Completion: all three examples have Go workflow and JSON checks, and the
obsolete runner and custom release hook have been removed. Updated the current
development, testing, and release guidance and marked the feature roadmap
complete. On Go 1.27.0, the full race-enabled suite, full vet, all three
repository-root example commands, and the disposable release-wiring diagnostic
passed. A deliberately incorrect example expectation failed before restoration,
then the affected package passed again. Reviewed links, the profile, formatting,
module and launcher changes, and `git diff --check`.