The self-verification Skill — Three Layers of Artifact Review
How Datarim's self-verification skill runs a deterministic shell pipeline, a cross-model peer review, and a parallel multi-agent check against a single artifact — and why three layers are needed instead of one.
Asking the same model that produced an artifact to review it tends to produce confirmation, not critique. The self-verification skill is built around that observation. It runs verification in three layers, each using a different signal source, and only considers the artifact clean when all three come back without high-severity findings.
The skill is invoked manually via /dr-verify {TASK-ID}. It is not an automatic pipeline hook — that path is a deferred future evolution gated by a dogfood threshold. The appropriate target is a completed artifact: a PRD, a plan, a do-output, or an archive. Running it during an active session is the wrong moment.
Layer 1 — deterministic floor
The first layer is a shell pipeline with zero LLM cost. It runs in two to five seconds on a typical task. The checks are heuristic but concrete: every acceptance criterion label in the PRD must have a verification command nearby; every file path mentioned in the plan must resolve in the workspace; shellcheck runs against all shell scripts in the standard directories; and for L3 and L4 tasks, an expectations checklist must be present and must not be in a BLOCKED state.
The floor exits with a count of high-severity findings. A non-zero exit is treated as a BLOCKED floor; the orchestrator may skip Layers 2 and 3 to fail fast, though operators who want only the deterministic check can pass --floor-only to stop there explicitly.
Layer 2 — cross-model peer review
The second layer sends the artifact to an external model via the coworker CLI. The provider is resolved through a six-step chain: an explicit CLI flag first, then per-project config, then per-user config, then the coworker profile default, then a cross-Claude-family subagent, and finally a same-model isolated fallback of last resort. The provider at step four of that chain is whatever the coworker --profile code configuration declares as its recommended provider — there is no hardcoded default. The property that matters for bias reduction is using a different vendor and a different training run than the model that produced the artifact.
The adversarial framing sent to the reviewer is mandatory and specific. The reviewer is told it is an adversarial reviewer whose job is to find weaknesses, not bless the document. It must find at least two substantive concerns and look at acceptance criterion verification commands, logged-but-not-run patterns, reflection coverage, follow-up spawns, reproducibility, and out-of-scope drift. Without this framing, the skill's own dogfood experiments showed the external model returning 29 completion tokens and an empty findings list — a silent false pass.
Layer 3 — parallel dispatch
The third layer spawns three parallel subagents on the Claude Code runtime: a reviewer focused on correctness and completeness, a tester focused on evidence coverage and reproducibility, and a security agent focused on safety category and Appendix A controls. Each subagent is read-only — no Write or Edit tools. Findings from all three are unioned and deduplicated by the tuple of artifact reference, acceptance criterion, and category. When two findings describe the same gap, the one from the earlier layer wins.
On the Codex CLI runtime, Layer 3 degrades to a single-prompt loop. That path hit only 7.7% literal gap-recall on the internal dogfood baseline of 13 tasks, which is why it is marked experimental. The recommended path on Codex is to pass --peer-provider deepseek and rely on Layer 2 for substantive coverage.
Findings schema and verdict
Every finding carries a source layer tag, an artifact reference with line number, an array of acceptance criteria it touches, a severity level, and a category. The category field distinguishes between correctness (a factual claim not supported by evidence), completeness (a required artifact piece missing), consistency (drift between artifacts), and safety (a security or rollback gap). Findings with evidence.type=absent are auto-discarded and do not count toward the verdict.
The verdict follows a simple rule: any non-discarded high-severity finding blocks the artifact; medium findings produce a conditional verdict requiring operator triage; low findings alone produce a pass. The audit log is written with chmod a-w post-write, making it append-only.
Read what Datarim is for the broader picture, or see the security skill to understand how the safety category findings are populated.