/dr-verify

Overview

/dr-verify runs three findings-only layers, cheapest first. Layer 1 is a deterministic shell floor. Layer 2 is an isolated peer review owned by the selected agent runtime. Layer 3 is a native read-only dispatch. External Coworker providers are intentionally invalid for adversarial verification; AC verification, hidden-gap discovery, and semantic judgment stay in the selected runtime.

Usage

/dr-verify {TASK-ID}                          # zero-flag — provider auto-resolves
/dr-verify {TASK-ID} --stage do
/dr-verify {TASK-ID} --floor-only             # Layer 1 only, fast pre-merge gating
/dr-verify {TASK-ID} --peer-provider sonnet  # explicit runtime-owned peer

The Three Layers

1. Layer 1: Deterministic floor — runs dev-tools/dr-verify-floor.sh, a pure shell pipeline performing AC coverage grep, file-touched audit, test-presence parse, and shellcheck on dev-tools/scripts. Zero LLM cost; runs in seconds. Emits JSONL findings with source_layer: "floor" on stdout.
2. Layer 2: Runtime-owned peer review — resolves sonnet, haiku, opus, or none from the CLI, project config, user config, or a bounded runtime fallback. Findings record source_layer, provider, mode, and resolution source. Task-ID propagation remains mandatory for auditable cost and outcome logs.
3. Layer 3: Native runtime dispatch — verification agents on the local runtime:
   • Claude path (canonical): 3 parallel read-only subagents (reviewer + tester + security). Findings union-merged and deduped by tuple (artifact_ref, ac_criteria, category); higher severity wins on collision.
   • Codex path [experimental] fallback: single-prompt loop with canonical adversarial framing. Demoted from canonical — prefer Layer 2 cross-model coverage on Codex too.

Peer Resolution (Zero-Flag UX)

dev-tools/resolve-peer-provider.sh uses a five-step, fail-closed chain:

1. Explicit --peer-provider CLI value.
2. Project datarim/config.yaml.
3. User ~/.config/datarim/config.yaml.
4. Cross-Claude-family Peer Reviewer fallback on Claude runtime.
5. Same-model isolated fallback on Codex, with an explicit warning.

The whitelist is sonnet | haiku | opus | none. Unknown and external values exit non-zero instead of silently changing reviewer ownership.

peer_review_mode Taxonomy

• cross_claude_family — an isolated Claude-family checkpoint reviews output from another checkpoint.
• same_model_isolated — isolated same-model review, used only as the last runtime fallback.

Codex currently uses the experimental same-model isolated path and records that limitation in the audit log.

Findings Schema

Each finding follows a standardized schema (canonical in skills/self-verification/SKILL.md § Findings Schema):

• source_layer — enum: floor, peer_review, dispatch
• severity — enum: high, medium, low
• category — enum: correctness, completeness, consistency, safety
• evidence — {type, source, excerpt} with type in {file_quote, test_output, absent} (absent auto-discards)
• ac_criteria — array of AC labels the finding maps to (may be empty)
• peer_review_provider — optional, Layer 2 only (provider name)
• peer_review_mode — optional, Layer 2 only (cross_claude_family or same_model_isolated)
• peer_review_provider_source_layer — optional, Layer 2 only (cli_flag, per_project_config, per_user_config, fallback_subagent, or fallback_isolated)
• check_name — optional, Layer 1 only (e.g. ac_coverage_grep, shellcheck)

JSONL Emission Discipline (Layer 2 prompts)

Findings are emitted ONLY when a check FAILS or surfaces NEW DRIFT. PASS-as-finding entries (e.g. {"check_name":"F001 cleared, no finding"}) are rejected — confirmations belong in the final-line summary {cleared_iter1: [...], total_new_findings: N}, not in the array. Keeps the audit log signal-dense.

Verdicts

• PASS — only low-severity (or zero) non-discarded findings
• CONDITIONAL — ≥1 medium and zero high
• BLOCKED — ≥1 high (merge/archive blocked until resolved)

--floor-only Flag

The --floor-only flag short-circuits at Layer 1 — ideal for fast pre-merge gating in CI. Completes in seconds with zero LLM cost, blocks only on structural / shellcheck-error issues. Layer 2 and Layer 3 are skipped.

Verification Outcome Tagging

At /dr-archive time the operator fills a verification_outcome block in the archive frontmatter (canonical schema in templates/archive-template.md): caught_by_verify, missed_by_verify, false_positive, n_a, dogfood_window (operator-supplied window-id used as grouping key, not a date range). The aggregator dev-tools/measure-prospective-rate.sh --since <YYYY-MM-DD> walks all archive-*.md files and computes caught_per_5_tasks with a decision_hint for the next pipeline gate.

Example Session (zero-flag UX, no external API key)

> /dr-verify {TASK-ID} --stage all --max-iter 2

Layer 1 — floor: dr-verify-floor.sh --task {TASK-ID} --stage all
  → 2 findings (medium, safety, check_name=shellcheck)
  → exit 0 (no high-severity, proceed)

Layer 2 — peer_review (runtime-owned resolution)
  Step #1 --peer-provider flag    → not set
  Step #2 ./datarim/config.yaml   → not set
  Step #3 ~/.config/datarim/...   → not set
  Step #4 cross-Claude-family     → MATCH (subagent peer-reviewer @ sonnet)
  Dispatching agents/peer-reviewer.md, model=sonnet, isolated context
  → 1 finding (medium, correctness, peer_review_mode=cross_claude_family)

Layer 3 — dispatch runtime=claude
  3 parallel agents: reviewer / tester / security
  → reviewer: 1 finding (completeness)
  → tester: 0 findings
  → security: 0 findings

Aggregate: 4 unique findings post-dedupe
Verdict: CONDITIONAL (0 high, 4 medium)
source_layer_breakdown: {floor: 2, peer_review: 1, dispatch: 1}
peer_review_mode_breakdown: {cross_claude_family: 1}
Audit: datarim/qa/verify-{TASK-ID}-all-1.md (chmod a-w)

Related Commands

• /dr-qa — multi-layer quality verification (different surface; complementary)
• /dr-archive — operator fills verification_outcome block here