Clarify ownership of upstream integration tests
This commit is contained in:
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
**Base Policy Version:** 1.3
|
**Base Policy Version:** 1.3
|
||||||
|
|
||||||
**Last Updated**: 2026-10-07
|
**Last Updated**: 2026-10-17
|
||||||
|
|
||||||
**Upstream Source:** [testing.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/testing.md)
|
**Upstream Source:** [testing.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/testing.md)
|
||||||
|
|
||||||
@@ -70,7 +70,7 @@ implemented:
|
|||||||
|
|
||||||
Use external-package consumer tests for facade semantics and real backend
|
Use external-package consumer tests for facade semantics and real backend
|
||||||
integration where correctness depends on that interaction. Internal package
|
integration where correctness depends on that interaction. Internal package
|
||||||
tests own focused mechanisms. Do not retest the dependencies' complete suites.
|
tests own focused mechanisms.
|
||||||
Boundary checks protect selected architectural constraints; behavioral evidence
|
Boundary checks protect selected architectural constraints; behavioral evidence
|
||||||
protects invocation semantics and ownership through conversions.
|
protects invocation semantics and ownership through conversions.
|
||||||
|
|
||||||
@@ -79,7 +79,7 @@ concurrency guarantees only if deliberately selected. Coverage targets, race
|
|||||||
testing, static analysis, example gates, and platform matrices require their own
|
testing, static analysis, example gates, and platform matrices require their own
|
||||||
risk justification; another library's workflow does not establish them here.
|
risk justification; another library's workflow does not establish them here.
|
||||||
|
|
||||||
To the extent the following features are present in the application, prioritize tests for:
|
For behavior owned by CommandKit, prioritize tests for:
|
||||||
|
|
||||||
1. Public and package-level contracts.
|
1. Public and package-level contracts.
|
||||||
2. Domain rules and important invariants.
|
2. Domain rules and important invariants.
|
||||||
@@ -106,6 +106,41 @@ application's guarantees. Account for protection already provided at other layer
|
|||||||
|
|
||||||
Do not add tests merely because a function, branch, or line exists. Do not add a test when the same meaningful risk is already adequately protected elsewhere.
|
Do not add tests merely because a function, branch, or line exists. Do not add a test when the same meaningful risk is already adequately protected elsewhere.
|
||||||
|
|
||||||
|
## Upstream integration testing ownership
|
||||||
|
|
||||||
|
CommandKit tests the interfaces and adapters through which it exposes upstream
|
||||||
|
APIs. ConfigKit, PromptKit, and other integrated libraries own tests establishing
|
||||||
|
the correctness of their domain functionality. Trust their public contracts;
|
||||||
|
CommandKit tests must not independently establish whether those libraries
|
||||||
|
implement their claimed domain functionality. This applies to partial feature
|
||||||
|
checks as well as complete suites.
|
||||||
|
|
||||||
|
Integration assertions should establish CommandKit-owned behavior:
|
||||||
|
|
||||||
|
- CLI choices, typed inputs, configuration selection, and context reach the
|
||||||
|
appropriate upstream API with the intended values.
|
||||||
|
- CommandKit performs its selected coordination and preserves returned data,
|
||||||
|
diagnostics, and supported error identity across adaptation.
|
||||||
|
- CommandKit's reports and invocation outcomes correctly represent the returned
|
||||||
|
evidence, including presence, null, empty, and redacted values where applicable.
|
||||||
|
- Command availability, input/state lifetimes, cancellation handling, and output
|
||||||
|
delivery follow CommandKit's contracts.
|
||||||
|
|
||||||
|
For example, test that a namespace choice becomes the intended native listing
|
||||||
|
filter, rather than testing which resources PromptKit considers members of a
|
||||||
|
namespace. Test faithful presentation of ConfigKit's returned source evidence,
|
||||||
|
rather than its destination resolution, version checking, or sensitivity
|
||||||
|
classification. Test request forwarding and returned-error handling, rather than
|
||||||
|
whether PromptKit correctly detects missing artifacts or invalid templates.
|
||||||
|
|
||||||
|
A real upstream engine or result can be a useful fixture for a focused
|
||||||
|
integration test. Its use does not change assertion ownership: the test should
|
||||||
|
detect incorrect CommandKit adaptation, not independently establish upstream
|
||||||
|
domain correctness. Use the simplest suitable fixture or test double; this
|
||||||
|
boundary does not require new mocking interfaces or prohibit real dependencies
|
||||||
|
in tests. CommandKit's own parsing, registration, coordination, presentation,
|
||||||
|
and outcome behavior remain within its testing scope.
|
||||||
|
|
||||||
## Choose the right test boundary
|
## Choose the right test boundary
|
||||||
|
|
||||||
Choose the narrowest stable boundary that expresses the behavior clearly and
|
Choose the narrowest stable boundary that expresses the behavior clearly and
|
||||||
|
|||||||
@@ -283,11 +283,22 @@ does not require a CommandKit version to mirror any upstream version.
|
|||||||
|
|
||||||
### Evidence for the eventual implementation
|
### Evidence for the eventual implementation
|
||||||
|
|
||||||
The implementation plan should select proportionate evidence for native source
|
The implementation plan should test CommandKit-owned interfaces and adaptation
|
||||||
placement/version disclosure, direct prompt selection, absent/empty labels,
|
under the [upstream testing ownership policy](../policy/testing.md#upstream-integration-testing-ownership).
|
||||||
preserved requirements behavior, optional catalog availability, scoped/default
|
Verify that command inputs and context reach the intended native APIs, including
|
||||||
and explicit listing filters, errors, and output delivery. Use maintained offline
|
ID-only prompt requests and default/explicit listing filters. Verify faithful
|
||||||
consumers and existing release-launcher coverage where applicable.
|
representation of upstream-returned source evidence, optional labels, inventories,
|
||||||
|
diagnostics, and errors, together with CommandKit's optional command availability,
|
||||||
|
invocation lifetimes, outcome classification, and output delivery.
|
||||||
|
|
||||||
|
These checks establish CommandKit's access to the capabilities. They should not
|
||||||
|
test whether ConfigKit correctly places or composes sources, checks versions,
|
||||||
|
or classifies sensitivity, nor whether PromptKit correctly resolves namespaces,
|
||||||
|
discovers requirements, or validates concrete requests. Those domain tests belong
|
||||||
|
to the upstream libraries. Real upstream fixtures and maintained offline
|
||||||
|
consumers can exercise CommandKit's boundary without duplicating native feature
|
||||||
|
test matrices. Existing release-launcher coverage likewise owns CommandKit's
|
||||||
|
tool invocation, not ReleaseKit's publication algorithms.
|
||||||
|
|
||||||
Verify all affected modules both in the local workspace and with their own
|
Verify all affected modules both in the local workspace and with their own
|
||||||
readonly dependency graphs. Temporary replacements for unpublished CommandKit
|
readonly dependency graphs. Temporary replacements for unpublished CommandKit
|
||||||
|
|||||||
Reference in New Issue
Block a user