Same Prompt, 80 Answers: Verifiable Compute’s Determinism Problem

Ask a large language model the same question a thousand times with the sampling temperature set to zero, the setting that is meant to force it to answer identically every time, and you will not get one answer back. In a widely read September 2025 experiment, researchers at Thinking Machines Lab did exactly that and counted 80 distinct completions, with even the most common one showing up in only 78 of the 1,000 runs. The outputs agreed for the first 102 tokens and then split. Nothing was broken. This is simply how modern AI runs on a graphics processor.

That result is a quiet problem for one of the most hyped ideas at the intersection of crypto and AI: verifiable compute, the promise that a network can prove a model produced a given output correctly, so you do not have to trust the company that ran it. Every scheme for doing that, from zero-knowledge proofs to fraud proofs to hardware attestation to slashable stake, rests on one buried assumption, that an honest re-run of the same model on the same input yields the same result. When it does not, the entire machinery of proof starts to slip. This is the story of the least glamorous and most important layer of the verifiable-AI stack in 2026, the one that has to make a GPU agree with itself before anything can be proven at all.

What verifiable compute is supposed to guarantee

The pitch for verifiable compute is easy to state. A server runs a computation and returns not only the result but a cheap-to-check proof that the result is correct, so the verifier never has to redo the work. Trust moves from trust the operator to check the proof. That is different from merely decentralized compute: renting GPUs from a marketplace spreads the work around, but it does not prove anyone ran your model honestly. Ethereum co-founder Vitalik Buterin mapped the design space in his January 2024 essay on crypto and AI, and he warned about the cost. Proving a matrix multiplication in zero knowledge adds well under 4x overhead, but the non-linear operations that make a neural network work can add roughly 200x, and a full zero-knowledge proof of a heavy computation, in his words, can take hours.

Four families of technique now compete to deliver the guarantee, and each buys its trust from a different place. The column that matters for everything below is the last one: in one way or another, every approach depends on being able to reproduce the computation.

Read that table as a set of promises with a shared footnote. zkML gives you math, opML gives you a game, TEEs give you a sealed room, and crypto-economics gives you a bond you can burn. None of those primitives does anything useful if honest parties cannot agree on what the correct output even was.

Why a GPU cannot reproduce itself

The intuition most engineers reach for is that floating-point math is imprecise, and concurrent hardware adds numbers in a random order, so tiny rounding differences pile up. That intuition is only half right, and the correction is the most interesting technical finding in this whole area. The Thinking Machines Lab analysis, written by Horace He and colleagues, showed that the real culprit for production endpoints is not concurrent rounding inside a single request. It is batch invariance. When your request lands on a busy server, it is grouped with other users into a batch, and the size of that batch depends on how much traffic the server is handling at that instant. Different batch sizes push the GPU kernels down different reduction strategies, and those strategies produce numerically different results for your tokens even though your input never changed.

In the team’s own framing, as Horace He put it, the primary reason nearly all LLM inference endpoints are nondeterministic is that the load, and thus the batch size, nondeterministically varies. The forward pass of the model is a deterministic function; the serving system around it is not. That is why the divergence in their test appeared at token 103 rather than token 1. The early tokens were computed identically, and only once accumulated rounding differences crossed a threshold did the sampler pick a different next token, at which point the two answers walked away from each other for good. Floating-point non-associativity is the powder, but the batch scheduler is the spark.

For a consumer chatbot this is invisible and harmless. For a verifiable-compute network it is the ballgame. A proof system exists to let a second party check the first party’s work. If the honest second party, running the identical model on the identical input, gets a different byte string simply because its GPU was under a different load, then there is no fixed thing to prove, no fixed thing to challenge, and no fixed thing to slash against.

The moment the proof slips

Walk the failure through each approach and the same crack appears in a different shape. Start with opML, the optimistic model formalized in the 2024 opML paper and shipped by Ora Protocol. Its whole security argument is a challenge: publish a result, and if any watcher re-executes the step and gets something different, they raise a fraud proof and the dispute is bisected down to a single mismatched operation. But if honest re-execution on a different machine already disagrees, the challenge protocol cannot tell an honest divergence from a cheat. That is precisely why opML implementations run the disputed step inside a constrained, GPU-free virtual machine: they trade away native speed to buy back determinism. The reproducibility problem is not a footnote to opML; it is the reason opML is slow.

zkML has the same problem earlier in the pipeline. To prove an inference in zero knowledge you have to express it as a fixed arithmetic circuit over a finite field, and that circuit has to be deterministic by construction. Native 32-bit floating point does not fit. As Elena Burger, a deal partner at a16z crypto, wrote in the firm’s analysis of machine learning and zero-knowledge proofs, zero-knowledge proofs today cannot represent 32-bit floating point operations in the necessary arithmetic circuit format without massive overheads, which forces builders to quantize the model into integers, producing what she called a crude approximation of the higher-quality initial model. So zkML does not verify the GPU’s floating-point answer at all. It verifies a different, deterministic, integer-only computation and hopes it is close enough.

Trusted execution environments look like they dodge the issue and do not. A TEE, whether Intel TDX, AMD SEV, or a confidential NVIDIA H100, produces a signed attestation that a specific binary ran unmodified on genuine silicon. That is a statement about execution, not about output. Two honest TEEs on two different GPUs, or the same GPU under two different loads, will each attest truthfully while returning different bytes. Attestation proves the room was sealed; it says nothing about whether two sealed rooms would reach the same number. And the crypto-economic approach, restaking and slashing, needs the strongest form of agreement of all: to slash a node for a wrong answer you must define the right answer, and if honest nodes legitimately disagree bit for bit, any slashing rule risks punishing the honest. Every path leads back to the same door.

Determinism turns verification into a byte check

Flip the problem over and the upside becomes obvious. If you can force an honest re-run to produce the exact same bytes every time, verification stops being a statistical argument and becomes a string comparison. This is the thesis of a 2026 paper from Eigen Labs describing EigenAI, a verifiable inference system built on the EigenLayer restaking ecosystem. The EigenAI paper names the three sources of the disease precisely, floating-point non-associativity, kernel scheduling, and variable batching, then argues that curing them changes the economics of proof itself.

The paper’s language is blunt about the payoff. Determinism, the authors write, collapses verification to a simple equality check, because every honest re-execution yields an identical byte string, so the verification predicate reduces to constant-time byte comparisons rather than probabilistic voting, and, as they put it, becomes both sound and complete. In their experiments the claim held up: across 10,000 runs, all hashes matched exactly, with no bit-level divergence observed. You do not need a 200x zero-knowledge circuit or a multi-minute fraud window to check whether two byte strings are equal. You need a hash function. The hard and expensive part of verifiable AI, in this telling, was never the proof. It was making the thing you want to prove reproducible in the first place.

That reframing is why 2026 has seen the center of gravity in verifiable compute shift from proof systems to inference engines. Soubhik Deb of Eigen Labs summarized the posture in the company’s verifiable-cloud writeup with a line that doubles as the sector’s thesis: trust is not a UI feature; it’s infrastructure. If trust is infrastructure, determinism is the concrete it is poured on.

How you actually pin a GPU down

Making a GPU deterministic is unglamorous kernel engineering. The recipe has three main ingredients. First, batch-invariant kernels: rewrite the core operations, matrix multiplication, attention, and normalization, so they use one fixed reduction order regardless of how many requests share the batch, which is the direct fix for the batch-invariance problem. Thinking Machines Lab open-sourced exactly these in its batch_invariant_ops library. Second, canonical reduction orders and fixed math modes so the same additions always happen in the same sequence. Third, version-pinned drivers and libraries, because a silent update to a math library can change results underneath you.

The catch is speed, and here the numbers are honest and instructive. Forcing a single reduction strategy throws away optimizations the hardware normally uses to go fast. The overheads reported across the ecosystem in 2026 look like this.

Read that table skeptically. EigenAI’s near-parity figure comes from its own paper and from tuned, quantized kernels, so it is best treated as a best case rather than a settled industry number. The independent figures from Thinking Machines Lab and the SGLang team, whose deterministic-inference work builds directly on the batch-invariant operators, are the more sober guide: full-precision determinism in a real serving engine still costs somewhere between a quarter and two-thirds of your throughput, though CUDA-graph tricks have pulled that down fast. Either way, the direction is clear. In 2024 determinism at scale was widely assumed to be impractical. In 2026 it is a tunable cost, and the cost is falling.

The catch nobody has fully solved: different silicon

There is an asterisk on every determinism result published this year, and it is a big one. The bit-for-bit matches are demonstrated within a fixed hardware class. EigenAI’s perfect reproduction, for example, is reported for same-host and cross-host tests on identical GPU models. Two of the same accelerator can be made to agree. Two different accelerators, an H100 and an H200, or an NVIDIA part and an AMD part, run different kernels with different numerics, and there is no cheap way to make them produce the same bytes.

That is awkward precisely for the networks that most want verifiable compute. A decentralized GPU marketplace is, by design, a heap of heterogeneous hardware contributed by strangers. Demanding that every prover and every challenger run one blessed GPU model quietly recentralizes the thing crypto set out to decentralize. It also collides with the wider economics of AI silicon, where access to a specific class of accelerator is now a strategic asset that increasingly gets decided in the boardroom rather than the data center. The realistic near-term answer is a canonical hardware target: the network specifies the GPU model, the driver version, and the kernel build, and only results produced on that exact stack are verifiable. Useful, shippable, and a long way from the picture of anonymous idle gaming rigs proving frontier inference. Honest verifiable compute in 2026 is deterministic on purpose and homogeneous by necessity.

This stopped being only a crypto problem

The tell that determinism is a real technical frontier and not a crypto talking point is that mainstream AI infrastructure teams are chasing it independently, for reasons that have nothing to do with blockchains. The Thinking Machines Lab post that opened this article is not a web3 project; it is a research lab building serving infrastructure. The SGLang and LMSYS teams pursued deterministic inference to make reinforcement-learning training reproducible, and they reported the milestone that matters to any scientist: with batch-invariant kernels enabled, two independent training runs produced identical curves, and the run became fully reproducible. Developer-facing writers picked it up too, with Simon Willison walking through the result in his September 2025 note on the work.

This matters for the crypto framing in two ways. It means the determinism problem is real enough that well-funded non-crypto labs spend engineering quarters on it, which retires the objection that verifiable-compute people invented a problem to sell a token. And it means crypto is, for once, early to a primitive the whole industry is converging on. Reproducible inference is becoming table stakes for anyone who needs to debug a model, reproduce a benchmark, satisfy an auditor, or defend an output in court. Verifiable-compute networks did not create the demand for determinism; they are the loudest customers for a capability the rest of AI is starting to want anyway.

Where the proofs actually stand in 2026

Once you fix reproducibility, the proof layer above it has genuinely advanced. Lagrange’s DeepProve system reached what it described as the first proofs of full large-language-model inference end to end, covering GPT-2 and Gemma-3, with Llama-class models still in active development, per the company’s DeepProve writeup. The same effort reports more than ten million proofs generated and verification hundreds of times faster than prior systems. Chief executive Ismael Hishon-Rezaizadeh framed the open-source release in almost civic terms, saying the black box is open and that AI needs a verification layer that everyone can use. Lagrange, citing McKinsey research, has also leaned on a striking demand statistic: that 71% of enterprise executives say they will not scale AI systems without proof of correctness. Whether or not that number survives contact with actual buying behavior, it captures why serious money keeps flowing into a sector whose tokens are a graveyard.

The point for this piece is the ordering. A proof of an inference is only as meaningful as the reproducibility of that inference. DeepProve, like every proof system, proves a specific deterministic computation; the determinism layer is what makes that computation a faithful stand-in for what the model actually did. The proofs got the headlines this year. The plumbing under them, the deterministic kernels that let an honest party reproduce the exact bytes, is what turned the headlines into something you could check. Auditing firms have started to notice the same pattern in adjacent domains, from smart-contract review to the state-backed digital-currency audits that are pulling formal verification into the mainstream.

Determinism does not repeal the trilemma

Fixing reproducibility does not make verifiable compute free. It relocates the cost. A 2026 paper introducing Optimistic TEE-Rollups framed the underlying tension as a verifiability trilemma, arguing that a decentralized inference system cannot simultaneously achieve high computational integrity, low latency, and low cost, and can realistically get two. The OTR paper reports a hybrid, a confidential GPU for fast provisional finality, optimistic fraud proofs for integrity, and stochastic zero-knowledge spot checks for insurance, that claims roughly 99% of centralized throughput at about seven cents of overhead per query, while criticizing pure optimistic schemes for imposing dispute windows so long they prevent real-time interactivity.

Determinism is what makes those hybrids buildable rather than a repeal of the trilemma. If your inference is bit-exact, the optimistic layer can define fraud cleanly, the zero-knowledge spot check has a fixed target to sample, and the TEE’s attestation finally corresponds to an output other nodes will reproduce. The proof marketplaces that shipped in 2025 assume this foundation. When Boundless launched its zero-knowledge compute market, chief executive Shiv Shankar told CoinDesk that for the first time developers on any chain could access abundant zero-knowledge compute to build applications that scale across ecosystems without sacrificing decentralization. Abundant compute still has to be compute over something reproducible, or the abundance proves the wrong thing very efficiently.

The tokens tell a different story

For all the engineering progress, the market has cast a harsh vote. Every token attached to a credible, live verifiable-compute project trades far below its peak, and the gap between shipping technology and token price is the sector’s defining feature in 2026. The figures below are from CoinGecko on 24 September 2026.

The contrast with usage is stark. Phala reports serving tens of billions of confidential model tokens a day across Intel TDX and NVIDIA hardware on its network, and the larger decentralized-AI token, Bittensor’s TAO, still trades around $289 on a multibillion-dollar market cap, while the specialized proving tokens are worth a fraction of that, most under $50 million. Real inference is flowing; the equity value is not following. That divergence sharpens a question builders now ask openly: does verifiable compute need a token at all, or is it infrastructure that will be bought and sold in dollars like every other cloud primitive, with the crypto rails reduced to a settlement detail. The determinism layer, tellingly, has no token of its own. It is pure engineering, and it may be the most valuable part.

What the SEC has said, and what it has not

For a United States reader the regulatory picture is defined mostly by silence. When the Securities and Exchange Commission and the Commodity Futures Trading Commission issued their joint interpretation on the status of digital assets in March 2026, they named major tokens such as Bitcoin, Ether, Solana, and XRP as digital commodities within a broader taxonomy, per the SEC release. They said nothing specific about AI-infrastructure or verifiable-compute tokens, which leaves assets like EIGEN, PROVE, ZKC, and LA in a case-by-case gray zone under the usual Howey analysis rather than in a safe harbor. The same uncertainty that hangs over how digital assets get taxed and classified in Washington hangs over whether a proving-network token is a commodity, a security, or something the current rules never contemplated.

The demand pressure, by contrast, is coming from across the Atlantic. The European Union’s AI Act reached general applicability in August 2026, bringing transparency duties and enforcement of its general-purpose model rules, with the heaviest high-risk obligations phased in through 2027 and 2028, per the European Commission’s regulatory framework. An enforceable duty to document and evaluate model behavior is exactly the kind of pull that could make a cryptographic proof of correct inference commercially valuable rather than merely elegant. For now the pattern is familiar: Brussels is loud and creating obligations, Washington is quiet and creating ambiguity, and the technology is being built as if both will eventually matter.

Why the boring layer decides the exciting ones

The reason to care about determinism is that every flashy application people want from verifiable compute sits directly on top of it. Autonomous agents that hold funds and sign transactions are the clearest case. An agent that trades or moves money has to prove it acted on a real model output and not a manipulated one, and standards for agent identity and validation now assume a cryptographic proof can be attached to an action. That proof is only meaningful if the action’s inputs can be reproduced. The account-level plumbing that lets an agent act on your behalf, from smart-account patterns to the wallet-delegation mechanics of EIP-7702, gives the agent hands; determinism is what lets anyone later check what those hands were told to do. The same holds for the agent frameworks racing to production, including the gaming-focused Eliza agent operating system, where a verifiable action log is only as trustworthy as the reproducibility beneath it.

Verifiable oracles, prediction-market resolution, and on-chain AI games all inherit the same dependency. Each wants to guarantee that a model output committed on-chain is the honest output of a specified model, and each is quietly relying on the ability of a second party to reproduce that output and check it. The proofs, the marketplaces, the restaking guarantees, and the confidential enclaves are the visible half of verifiable compute. The invisible half, the one that decides whether any of it means anything, is a set of kernels rewritten so a graphics processor gives the same answer twice. In 2026 that is where the real fight moved. Same prompt, one answer, is a harder engineering problem than it sounds, and it is the floor everything else is standing on.

Frequently Asked Questions

What is verifiable compute in AI and crypto?

Verifiable compute means running a computation, such as an AI model inference, and returning a cheap-to-check proof that it was done correctly, so a verifier does not have to trust the operator or redo the work. In the AI-crypto context it lets a network guarantee that a specific model produced a specific output, using zero-knowledge proofs, optimistic fraud proofs, hardware attestation, or slashable economic stake.

Why is GPU AI inference non-deterministic?

The main reason production endpoints are non-deterministic is batch invariance. Your request is grouped with others into a batch whose size depends on server load, and different batch sizes make GPU kernels use different reduction orders, which changes the numerical result even at temperature zero. Floating-point non-associativity, kernel scheduling, and variable batching combine so that the same input can yield different outputs on different runs.

How does determinism make AI verifiable?

If an honest re-run always produces the exact same bytes, verification becomes a simple equality check between two hashes rather than a slow proof or a probabilistic vote. Deterministic inference lets fraud proofs define what is wrong, gives zero-knowledge circuits a fixed target, and lets economic slashing punish only genuine cheating, which is why bit-exact reproducibility has become the foundation of verifiable-compute designs in 2026.

Does deterministic inference slow the model down?

Yes, but the cost is shrinking. Independent work in general-purpose engines reported a throughput hit in the range of a quarter to two-thirds when forcing batch-invariant kernels, though pairing them with CUDA graphs cut the average to roughly a third. Vendor benchmarks for tuned, quantized deterministic kernels report far smaller overheads near a couple of percent, so the real figure depends heavily on the setup.

Why are verifiable-compute tokens down so much?

Despite live products and real usage, tokens tied to verifiable-compute projects such as EIGEN, PROVE, ZKC, LA, and PHA trade roughly 86% to 98% below their all-time highs as of late September 2026. The gap reflects a broader question about whether this infrastructure needs a token at all, since the demand is for a service that can be bought in dollars, and the most valuable layer, deterministic inference, has no token of its own.

By Marcus Okafor, HOGE Wire staff writer covering the crossroads of AI and crypto infrastructure.