Skip to content

Plugin conformance

packages/runner/tests/plugins/conformance.test.ts codifies the expectations every supported plugin — built-in or OSS drop-in — must satisfy. External authors should import the same helpers (or copy the scenario list) inside their CI.

What the harness asserts

  1. Manifest invariantsid, kind, semver, displayName, compatible hostCompatibility, Zod configSchema.
  2. Lifecycleinit(valid) resolves, init(invalid) throws, healthcheck() returns structured success, dispose() is safe even without init.
  3. SCM contract — clone URL shape, matchesRemote consistency, pollPr snapshots, normalizeInbound rejects garbage.
  4. Tracker contract — same lifecycle gates + optional inbound parsing.
  5. Intelligence contributions — every intelligence.skills / snippets relative path exists under intelligenceRoot().

Executor plugins use a separate suite because their factories need live Settings; SCM/tracker packs stay synchronous for fast feedback.

Scope boundaries

  • Hermetic — HTTP clients mock upstream providers.
  • Does not replace end-to-end webhook bridge tests (plugin-webhook-bridge.test.ts).
  • Does not verify business logic inside individual providers.

Ship a npm test entry that calls runConformance with your plugin factory to catch drift before publishing.