Clarify ownership of upstream integration tests

This commit is contained in:
2026-10-17 15:37:20 +00:00
parent 7b9b945baa
commit 502ea5eca8
2 changed files with 54 additions and 8 deletions

View File

@@ -2,7 +2,7 @@
**Base Policy Version:** 1.3
**Last Updated**: 2026-10-07
**Last Updated**: 2026-10-17
**Upstream Source:** [testing.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/testing.md)
@@ -70,7 +70,7 @@ implemented:
Use external-package consumer tests for facade semantics and real backend
integration where correctness depends on that interaction. Internal package
tests own focused mechanisms. Do not retest the dependencies' complete suites.
tests own focused mechanisms.
Boundary checks protect selected architectural constraints; behavioral evidence
protects invocation semantics and ownership through conversions.
@@ -79,7 +79,7 @@ concurrency guarantees only if deliberately selected. Coverage targets, race
testing, static analysis, example gates, and platform matrices require their own
risk justification; another library's workflow does not establish them here.
To the extent the following features are present in the application, prioritize tests for:
For behavior owned by CommandKit, prioritize tests for:
1. Public and package-level contracts.
2. Domain rules and important invariants.
@@ -106,6 +106,41 @@ application's guarantees. Account for protection already provided at other layer
Do not add tests merely because a function, branch, or line exists. Do not add a test when the same meaningful risk is already adequately protected elsewhere.
## Upstream integration testing ownership
CommandKit tests the interfaces and adapters through which it exposes upstream
APIs. ConfigKit, PromptKit, and other integrated libraries own tests establishing
the correctness of their domain functionality. Trust their public contracts;
CommandKit tests must not independently establish whether those libraries
implement their claimed domain functionality. This applies to partial feature
checks as well as complete suites.
Integration assertions should establish CommandKit-owned behavior:
- CLI choices, typed inputs, configuration selection, and context reach the
appropriate upstream API with the intended values.
- CommandKit performs its selected coordination and preserves returned data,
diagnostics, and supported error identity across adaptation.
- CommandKit's reports and invocation outcomes correctly represent the returned
evidence, including presence, null, empty, and redacted values where applicable.
- Command availability, input/state lifetimes, cancellation handling, and output
delivery follow CommandKit's contracts.
For example, test that a namespace choice becomes the intended native listing
filter, rather than testing which resources PromptKit considers members of a
namespace. Test faithful presentation of ConfigKit's returned source evidence,
rather than its destination resolution, version checking, or sensitivity
classification. Test request forwarding and returned-error handling, rather than
whether PromptKit correctly detects missing artifacts or invalid templates.
A real upstream engine or result can be a useful fixture for a focused
integration test. Its use does not change assertion ownership: the test should
detect incorrect CommandKit adaptation, not independently establish upstream
domain correctness. Use the simplest suitable fixture or test double; this
boundary does not require new mocking interfaces or prohibit real dependencies
in tests. CommandKit's own parsing, registration, coordination, presentation,
and outcome behavior remain within its testing scope.
## Choose the right test boundary
Choose the narrowest stable boundary that expresses the behavior clearly and

View File

@@ -283,11 +283,22 @@ does not require a CommandKit version to mirror any upstream version.
### Evidence for the eventual implementation
The implementation plan should select proportionate evidence for native source
placement/version disclosure, direct prompt selection, absent/empty labels,
preserved requirements behavior, optional catalog availability, scoped/default
and explicit listing filters, errors, and output delivery. Use maintained offline
consumers and existing release-launcher coverage where applicable.
The implementation plan should test CommandKit-owned interfaces and adaptation
under the [upstream testing ownership policy](../policy/testing.md#upstream-integration-testing-ownership).
Verify that command inputs and context reach the intended native APIs, including
ID-only prompt requests and default/explicit listing filters. Verify faithful
representation of upstream-returned source evidence, optional labels, inventories,
diagnostics, and errors, together with CommandKit's optional command availability,
invocation lifetimes, outcome classification, and output delivery.
These checks establish CommandKit's access to the capabilities. They should not
test whether ConfigKit correctly places or composes sources, checks versions,
or classifies sensitivity, nor whether PromptKit correctly resolves namespaces,
discovers requirements, or validates concrete requests. Those domain tests belong
to the upstream libraries. Real upstream fixtures and maintained offline
consumers can exercise CommandKit's boundary without duplicating native feature
test matrices. Existing release-launcher coverage likewise owns CommandKit's
tool invocation, not ReleaseKit's publication algorithms.
Verify all affected modules both in the local workspace and with their own
readonly dependency graphs. Temporary replacements for unpublished CommandKit