We built and run Fact Engineering, a SaaS product that fact-checks documents. It works by refusing to trust any single AI model.
Every claim in an uploaded document is cross-examined in parallel by a panel of frontier models from different providers. An AI judge weighs their answers — not averages them — and issues one consensus verdict. At the highest tier, every claim is re-investigated against the live web and returned with citations, so a human can audit the call.
That architecture came out of a simple observation: when you ask several models the same question, the disagreements are the signal.
Disagreement is the signal
Ask one model whether a claim is true and you get an answer delivered with total fluency, whether it is right or wrong. Ask four models from different providers and you get something more useful: a shape.
Try it below. These are condensed, illustrative panel outputs — the patterns we see in real runs, not live model calls.
Same panel, three claims — pick one
“Goldfish have a three-second memory.”
Three claims, three different shapes. Unanimity means certainty is cheap. A split means the truth has a caveat that a one-line answer would have flattened. And a lone confident dissenter means one model is wrong today — on this claim it was Model B, tomorrow it will be a different model on a different claim.
A single-model system collapses all three shapes into the same thing: an answer, delivered with the same confident tone. The information you actually needed — how much to trust it — is gone.
What we built around that idea
Fact Engineering treats verification as a courtroom, not a chatbot. Scrutiny is dialed to the stakes across three tiers: a fast spelling-and-grammar pass, a panel fact-check against what the models know, and a live-research tier where every claim is re-investigated on the open web and returned with citations — defensible, traceable, audit-ready.
Building it taught us lessons that apply to almost every AI system a business ships.
Consensus is an architecture, not a prompt. Getting models to usefully disagree takes real structure: parallel fan-out across providers, a judge that weighs evidence instead of averaging scores, and sources attached to every verdict. That structure is ordinary engineering — queues, workers, streaming — wrapped around the models. On AWS it runs as Step Functions and Lambda, so a 40-page document fans out into hundreds of model calls without a fleet of servers idling in between.
Make the model layer replaceable. Which models vote, who judges, and what each check costs live in one configuration file, routed through a single provider-agnostic gateway. Models improve monthly; systems that hard-wire one provider age badly. When a better model ships, the product improves by editing one file, not by rebuilding the pipeline.
Stream progress or lose trust. A deep check takes minutes, and a spinner that long reads as broken. Verdicts stream into the browser the moment each one lands, so users watch the panel work claim by claim. The pipeline is asynchronous by design and real-time to watch — that combination is what makes the wait feel like progress instead of silence.
Price the uncertainty up front. Every check is quoted in credits before it runs — per claim, per tier. Metered AI billing that surprises people at the end of the month kills adoption faster than any accuracy problem.
Spend novelty tokens on one thing. Authentication, billing, and storage are managed services — WorkOS, Stripe, S3, Postgres. The engineering effort went into the consensus engine, because that is the part customers pay for. Every hour spent rebuilding a solved problem is an hour not spent on the differentiator.
The question that matters
The broader point for AI adoption: the question is rarely “can a model do this?” It usually can, most of the time. The question is what happens the time it cannot — and whether your system notices before your customer does.
That is verification, and it is a system you build, not a setting you enable.