base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Update guidance for Go example tests

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-03 02:31:35 +00:00
parent 175140ae5a
commit c7d4318122
6 changed files with 43 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/development.md
View File

@@ -67,7 +67,6 @@ the maintained examples when they are affected:
go test ./...
go vet ./...
gofmt -w path/to/changed.go
python3 scripts/check-examples.py
git diff --check
```
@@ -77,12 +76,12 @@ describes its requirements and recovery. For an optional diagnostic of a
committed candidate without publication, run `./scripts/releasekit.sh check
VERSION`; this is not a required pre-publication step.
The standalone `python3 scripts/check-examples.py` command checks semantic
expectations for the three offline consumer examples. The focused
`python3 scripts/test-release.py` command exercises the release tooling against
disposable local repositories; use it when changing that tooling, not as a
recurring release gate. Tests and examples require no provider credentials or
live inference calls.
The example workflows are covered by the Go suite; use
`go test ./examples/go-library/...` to run them separately during development.
The focused `python3 scripts/test-release.py` command exercises the release
tooling against disposable local repositories; use it when changing that
tooling, not as a recurring release gate. Tests and examples require no provider
credentials or live inference calls.
Ordinary validation can run on intended uncommitted work. Release validation
uses committed candidate contents and the selected local Go installation.

6
docs/policy/release.md
View File

@@ -271,9 +271,9 @@ The [publication command](../../scripts/release.sh) delegates to the released
ReleaseKit tool selected in `go.mod`. The committed
[profile](../../releasekit.json) selects the race-enabled Go suite, full vet,
committed Go formatting, local Markdown links, workspace and filesystem
replacement inspection, and the offline semantic checks for the three
maintained consumer examples. It also rejects the configured local delivery
token path if committed; this narrowly scoped check is not a general credential
replacement inspection. The Go suite includes the offline semantic checks for
the three maintained consumer examples. It also rejects the configured local
delivery token path if committed; this narrowly scoped check is not a general credential
scan. These are the selected checks for the final candidate; a separate
ordinary test run or standalone build is not an
additional release gate. The upstream tool owns candidate, remote, and tag

6
docs/policy/testing.md
View File

@@ -60,10 +60,10 @@ documented validation model.
The selected ReleaseKit profile runs the race-enabled package suite and full
vet, checks committed Go formatting, local Markdown links, and module
distribution hygiene, and executes semantic assertions for the three
distribution hygiene. The Go suite includes semantic assertions for the three
maintained offline consumer examples. ReleaseKit owns this shared validation
machinery and publication workflow. The test run builds the packages; the
examples also exercise ordinary compilation. Separate ordinary tests and a
machinery and publication workflow. The test run builds the packages and
exercises the examples. Separate ordinary tests and a
standalone build are not additional release gates. Ordinary tests remain useful
during development. The preparation example protects assembled preparation and
inspection behavior. The execution example separately protects assembled

4
docs/release.md
View File

@@ -8,8 +8,8 @@ delivery separately. Do not wait for CI or Go module availability.
## Prepare The Candidate
Required tools are Git, Go with race-detector support, POSIX shell, and Python
3. Work from the repository root with push access to `origin`.
Required tools are Git, Go with race-detector support, and POSIX shell. Work
from the repository root with push access to `origin`.
Use the latest Go installation available locally for the release tool and its
child commands. Put it first on `PATH` and keep it selected throughout:

42
docs/roadmap/example-testing.md
View File

@@ -1,5 +1,8 @@
# Go-Based Testing For Maintained Examples
Status: complete. Example assertions now run in the ordinary Go suite; the
custom release check and its Python runner have been removed.
## Purpose
Move the maintained consumer-example checks into Promptkit's ordinary Go test
@@ -15,7 +18,7 @@ library API or a new release gate.
The companion [implementation plan](implementation.md) defines the staged work
for reaching this target.
## Current Baseline
## Previous Baseline
The [preparation](../../examples/go-library/prepare/main.go),
[execution](../../examples/go-library/run/main.go), and
@@ -25,15 +28,14 @@ providers. Preparation and execution use repository-relative configuration
paths; observation embeds its assets and already has a workflow helper returning
a summary and error.
[scripts/check-examples.py](../../scripts/check-examples.py) launches all three
programs, decodes their JSON summaries, and checks meaningful results. Ordinary
`go test ./...` compiles the example packages but currently has no example tests
to execute. The [ReleaseKit profile](../../releasekit.json) runs the Python
script as its one custom check after shared validation.
The former `scripts/check-examples.py` launched all three programs and checked
their JSON summaries. Ordinary `go test ./...` then had no example tests, and
the [ReleaseKit profile](../../releasekit.json) selected the Python script as a
custom check after shared validation.
The separate [release-wiring test](../../scripts/test-release.py) is also Python,
but is a development-only diagnostic, not a release gate. Its disposable fixture
currently assumes the example script is a configured product check.
previously assumed the example script was a configured product check.
## Target State
@@ -86,25 +88,21 @@ implementation dependencies, consistent with the
## Integration And Documentation Work
Once the Go tests cover the retained behavior, remove `scripts/check-examples.py`
and its custom command and environment overlay from `releasekit.json`. Keep the
existing built-in validation selections and source/hosted publication workflow.
There is no need for new ReleaseKit functionality or a dependency upgrade.
The Go tests cover the retained behavior; the former `scripts/check-examples.py`
and its custom command and environment overlay have been removed. The existing
built-in validation selections and source/hosted publication workflow remain,
without a ReleaseKit dependency upgrade.
Adapt the development-only release-wiring fixture so it no longer expects the
removed script or product-check marker. Retain checks relevant to the actual
launcher and profile; do not manufacture an unused custom hook merely to retain
old fixture assertions. The Python release-wiring diagnostic can remain Python.
The development-only release-wiring fixture no longer expects the removed script
or product-check marker. It retains checks for the actual launcher and profile
without manufacturing an unused custom hook. The diagnostic remains Python.
Removing Python from every repository tool is outside this scope.
Update the [development guide](../development.md),
The [development guide](../development.md),
[testing policy](../policy/testing.md), [release policy](../policy/release.md),
and [release procedure](../release.md) when implementation lands. Replace the
standalone example command with Go test guidance, describe example coverage as
part of the standard suite, and remove Python as a normal release prerequisite.
Continue documenting Python where the optional release-wiring test needs it.
Check other incoming references to the removed runner and retain the existing
consumer quickstart commands.
and [release procedure](../release.md) describe Go test coverage as part of the
standard suite and retain Python only for the optional release-wiring test. The
existing consumer quickstart commands remain available.
## Verification

9
docs/roadmap/implementation.md
View File

@@ -244,3 +244,12 @@ successful ordinary suite discovery, preserved useful assertions, removal of
the runner and custom hook, and accurate current documentation. No performance,
coverage-percentage, platform, or release-service gate is added. There are no
unresolved product decisions or upstream publication dependencies for this sprint.
Completion: all three examples have Go workflow and JSON checks, and the
obsolete runner and custom release hook have been removed. Updated the current
development, testing, and release guidance and marked the feature roadmap
complete. On Go 1.27.0, the full race-enabled suite, full vet, all three
repository-root example commands, and the disposable release-wiring diagnostic
passed. A deliberately incorrect example expectation failed before restoration,
then the affected package passed again. Reviewed links, the profile, formatting,
module and launcher changes, and `git diff --check`.