Clarify ownership of upstream integration tests
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
**Base Policy Version:** 1.3
|
||||
|
||||
**Last Updated**: 2026-10-07
|
||||
**Last Updated**: 2026-10-17
|
||||
|
||||
**Upstream Source:** [testing.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/testing.md)
|
||||
|
||||
@@ -70,7 +70,7 @@ implemented:
|
||||
|
||||
Use external-package consumer tests for facade semantics and real backend
|
||||
integration where correctness depends on that interaction. Internal package
|
||||
tests own focused mechanisms. Do not retest the dependencies' complete suites.
|
||||
tests own focused mechanisms.
|
||||
Boundary checks protect selected architectural constraints; behavioral evidence
|
||||
protects invocation semantics and ownership through conversions.
|
||||
|
||||
@@ -79,7 +79,7 @@ concurrency guarantees only if deliberately selected. Coverage targets, race
|
||||
testing, static analysis, example gates, and platform matrices require their own
|
||||
risk justification; another library's workflow does not establish them here.
|
||||
|
||||
To the extent the following features are present in the application, prioritize tests for:
|
||||
For behavior owned by CommandKit, prioritize tests for:
|
||||
|
||||
1. Public and package-level contracts.
|
||||
2. Domain rules and important invariants.
|
||||
@@ -106,6 +106,41 @@ application's guarantees. Account for protection already provided at other layer
|
||||
|
||||
Do not add tests merely because a function, branch, or line exists. Do not add a test when the same meaningful risk is already adequately protected elsewhere.
|
||||
|
||||
## Upstream integration testing ownership
|
||||
|
||||
CommandKit tests the interfaces and adapters through which it exposes upstream
|
||||
APIs. ConfigKit, PromptKit, and other integrated libraries own tests establishing
|
||||
the correctness of their domain functionality. Trust their public contracts;
|
||||
CommandKit tests must not independently establish whether those libraries
|
||||
implement their claimed domain functionality. This applies to partial feature
|
||||
checks as well as complete suites.
|
||||
|
||||
Integration assertions should establish CommandKit-owned behavior:
|
||||
|
||||
- CLI choices, typed inputs, configuration selection, and context reach the
|
||||
appropriate upstream API with the intended values.
|
||||
- CommandKit performs its selected coordination and preserves returned data,
|
||||
diagnostics, and supported error identity across adaptation.
|
||||
- CommandKit's reports and invocation outcomes correctly represent the returned
|
||||
evidence, including presence, null, empty, and redacted values where applicable.
|
||||
- Command availability, input/state lifetimes, cancellation handling, and output
|
||||
delivery follow CommandKit's contracts.
|
||||
|
||||
For example, test that a namespace choice becomes the intended native listing
|
||||
filter, rather than testing which resources PromptKit considers members of a
|
||||
namespace. Test faithful presentation of ConfigKit's returned source evidence,
|
||||
rather than its destination resolution, version checking, or sensitivity
|
||||
classification. Test request forwarding and returned-error handling, rather than
|
||||
whether PromptKit correctly detects missing artifacts or invalid templates.
|
||||
|
||||
A real upstream engine or result can be a useful fixture for a focused
|
||||
integration test. Its use does not change assertion ownership: the test should
|
||||
detect incorrect CommandKit adaptation, not independently establish upstream
|
||||
domain correctness. Use the simplest suitable fixture or test double; this
|
||||
boundary does not require new mocking interfaces or prohibit real dependencies
|
||||
in tests. CommandKit's own parsing, registration, coordination, presentation,
|
||||
and outcome behavior remain within its testing scope.
|
||||
|
||||
## Choose the right test boundary
|
||||
|
||||
Choose the narrowest stable boundary that expresses the behavior clearly and
|
||||
|
||||
@@ -283,11 +283,22 @@ does not require a CommandKit version to mirror any upstream version.
|
||||
|
||||
### Evidence for the eventual implementation
|
||||
|
||||
The implementation plan should select proportionate evidence for native source
|
||||
placement/version disclosure, direct prompt selection, absent/empty labels,
|
||||
preserved requirements behavior, optional catalog availability, scoped/default
|
||||
and explicit listing filters, errors, and output delivery. Use maintained offline
|
||||
consumers and existing release-launcher coverage where applicable.
|
||||
The implementation plan should test CommandKit-owned interfaces and adaptation
|
||||
under the [upstream testing ownership policy](../policy/testing.md#upstream-integration-testing-ownership).
|
||||
Verify that command inputs and context reach the intended native APIs, including
|
||||
ID-only prompt requests and default/explicit listing filters. Verify faithful
|
||||
representation of upstream-returned source evidence, optional labels, inventories,
|
||||
diagnostics, and errors, together with CommandKit's optional command availability,
|
||||
invocation lifetimes, outcome classification, and output delivery.
|
||||
|
||||
These checks establish CommandKit's access to the capabilities. They should not
|
||||
test whether ConfigKit correctly places or composes sources, checks versions,
|
||||
or classifies sensitivity, nor whether PromptKit correctly resolves namespaces,
|
||||
discovers requirements, or validates concrete requests. Those domain tests belong
|
||||
to the upstream libraries. Real upstream fixtures and maintained offline
|
||||
consumers can exercise CommandKit's boundary without duplicating native feature
|
||||
test matrices. Existing release-launcher coverage likewise owns CommandKit's
|
||||
tool invocation, not ReleaseKit's publication algorithms.
|
||||
|
||||
Verify all affected modules both in the local workspace and with their own
|
||||
readonly dependency graphs. Temporary replacements for unpublished CommandKit
|
||||
|
||||
Reference in New Issue
Block a user