The reviewers behind the signature. Qualified, paid, and never charged.
When an invariant fires on a Tier-1 financial agent, a human decides whether it is a defect. That human is a senior analytics engineer who passed a hard battery, is paid a published rate from the first calibration hour, and whose accuracy is continuously measured against hidden gold. This page is the whole deal.
Gated on a benchmark exam, not on years
A three-year elite engineer out-performs a twelve-year one often enough that a tenure proxy is noise. We dropped it. What remains is a 50-item proctored challenge set drawn from the eight validated failure families, with matched clean controls.
| Category | Minimum passing standard |
|---|---|
| Sev-1 critical defect recall | ≥ 98.0% — at most one miss across the 50-item challenge exam |
| Sev-2 / Sev-3 general recall | ≥ 85.0% across relational and performance archetypes |
| False-positive rate | ≤ 5.0% flagging on matched clean controls |
| Calibration | Brier score ≤ 0.15 and Brier skill score > 0 on confidence-rated assessments |
| Continuous gold canary auditing | 5% hidden canaries in production; rolling Bayesian error rate; emergency review suspension upon two consecutive misses |
A 49/50 result is an observed qualifying score on the challenge set. We do not describe it as “98% recall” in the abstract, and we ask you not to either. Expected screening pass rate: about 30% PLANNING ASSUMPTION.
Published per role. Changed with notice, never retroactively.
Qualified reviewer (Tier 3)
Gate: ≥98% Sev-1 recall on the 50-item battery; ≤5% false-positive rate on matched controls
Senior assurance engineer
Gate: Authors Type B/C fixtures; dbt and Snowflake execution-plan fluency
Lead adjudicator (Tier 4)
Gate: Resolves Sev-1 disputes; Brier score ≤0.15 and BSS > 0 over the trailing 90 days
Calibration hours are paid at your rate. The only unpaid step is the 50-item qualification battery itself. Rates are published per role and change with 30 days notice, never retroactively. Competency is a vector (gold accuracy, sample size, recency, dialect, domain, overturn rate, Brier), not a years-of-experience proxy.
Reviewers never pay us. Not for the qualification battery, not for calibration, not to stay on the roster. We make money when you find real defects, not when you sit an exam.
Machines first, you where it matters
Deterministic checks
AST validation and relational invariants triage 70–80% of routine passes with no human at all.
Semantic judge
The Tier-2 model compares query plans against equivalence sets and flags what it cannot resolve.
You: Tier 3
One qualified reviewer on edge cases, invariant exceptions and high-consequence vectors — roughly a fifth to a third of items.
Tier 4 adjudication
Two blinded reviewers plus a lead adjudicator on Sev-1 disputes and contested findings. Your competency signature goes into the bundle.
A 15–25% overlapping sample is rated by more than one reviewer so agreement (α ≥ 0.75, AC1 ≥ 0.80) can be measured and drift caught early. Overlap items are paid like any other.
Usually none
In the default evidence mode you never see a customer’s query or data: the runner executes in their perimeter and you adjudicate on sanitised, pseudonymised evidence. For regulated schemas you work inside the customer’s own locked-down VDI. Your processing location is recorded per engagement, and it matters — a reviewer travelling changes the transfer analysis. Details in the security page and the handbook.
From reviewers
No. Passing the qualification battery is an observed qualifying score on a specific 50-item challenge set — at most one Sev-1 miss — not a claim about your true underlying recall, and not an accredited credential. We put you on the roster, pay you, and keep measuring. We do not sell the exam and we do not issue a badge.
Warehouse SQL at a senior analytics-engineering level: join semantics, three-valued logic, SCD-2 and effective-dated joins, fiscal calendars, and at least one warehouse's execution model (Snowflake micro-partitions and clustering keys first). dbt fluency helps. Years of experience are not a gate; the battery is.
Reviewers never pay us. Not for the qualification battery, not for calibration, not to stay on the roster. We make money when you find real defects, not when you sit an exam.
Honestly: little, today. We are running the first sprints with three design partners. Tier-3 review is dispatched on roughly a fifth to a third of items and Tier-4 adjudication only on Sev-1 disputes. The roster will grow with contracted ARR, not ahead of it.
As a vector, not a score: gold accuracy and its sample size, recency (90-day half-life), dialect competency, domain score, overturn rate, Brier score and Brier skill score. Five percent of your items are hidden gold canaries; two consecutive misses trigger an emergency suspension and a conversation, not a silent drop.
Join the roster
We reply within a week with the battery schedule. The battery is four hours, proctored, and unpaid; everything after it is paid at your role’s rate. We keep your details for 12 months after you leave the roster, as the privacy policy says.
- Senior warehouse SQL — Snowflake or Databricks first
- dbt fluency and effective-dated modelling
- Financial, supply-chain or healthcare domain depth is a plus, not a requirement
Received.
We will write within a week with the battery schedule.