2024-12-18

Synth matrices when humans and models disagree politely

By Haruto Vail

scorecardsgovernance
Synth matrices when humans and models disagree politely

Scorecard Synth Lab builds matrices where human reviewers and assisted checks sit side by side. Each row names a signal, each column names a reviewer pair. Disagreement gets a third column for narrative explanation, not just a tie-break score.

We schedule quarterly recalibration with live examples pulled from production—not sanitized demos. Recalibration adjusts weights publicly so downstream teams trust the math.

Matrices fail when they grow wide. We enforce a hard cap of seven active signals. Retired signals move to an appendix with retirement reasons, preserving organizational memory without cluttering the working scorecard.