Plugin conformance
packages/runner/tests/plugins/conformance.test.ts codifies the expectations every supported plugin — built-in or OSS drop-in — must satisfy. External authors should import the same helpers (or copy the scenario list) inside their CI.
What the harness asserts
- Manifest invariants —
id,kind, semver,displayName, compatiblehostCompatibility, ZodconfigSchema. - Lifecycle —
init(valid)resolves,init(invalid)throws,healthcheck()returns structured success,dispose()is safe even withoutinit. - SCM contract — clone URL shape,
matchesRemoteconsistency,pollPrsnapshots,normalizeInboundrejects garbage. - Tracker contract — same lifecycle gates + optional inbound parsing.
- Intelligence contributions — every
intelligence.skills/snippetsrelative path exists underintelligenceRoot().
Executor plugins use a separate suite because their factories need live Settings; SCM/tracker packs stay synchronous for fast feedback.
Scope boundaries
- Hermetic — HTTP clients mock upstream providers.
- Does not replace end-to-end webhook bridge tests (
plugin-webhook-bridge.test.ts). - Does not verify business logic inside individual providers.
Ship a npm test entry that calls runConformance with your plugin factory to catch drift before publishing.