Training gets the headlines. Inference gets the bill. That was the starting point when Xavier Lyu took the stage at Exploit Summit, the Bittensor conference held this year at New City Gas in Montréal. The first slide read Compounding Inference Optimization. The argument fit on the next one.

We don't build a better engine. We make yours race better.

Two different jobs

The talk split the stack in two. Training is designing the engine: new architectures, new data, new weights. It is slow, it is expensive, and a handful of labs do it well. Inference is the pit crew. The car is already built. The job is to get more laps out of it on the track you actually race on, without breaking anything.

Most teams stop thinking about serving once a model is deployed. An engine gets picked, flags get copied from someone's README, and the config stays frozen until something falls over. Kernels, KV-cache layout, batching, quantization and launch flags all move cost and latency, and they interact. Nobody has the hours to search that space by hand for every model they serve.

Xavier Lyu speaking to a seated audience, a slide reading Training: Designing the engine behind him
Training Designing the engine. Inference The pit crew.

Why the gains compound

One good patch might take a few percent off GPU-hours at the same latency. Useful, but not the point. The point is what happens next.

In a Pareton campaign, a win becomes the new baseline. Every later submission has to beat it on the same GPUs, the same traffic shape and the same p99 cap. The floor only moves one way, so small verified gains stack the way interest does.

It also means the measurement matters more than any single trick. A speedup that shows up on one trace, or that quietly bends the latency cap, is thrown out before it can become the next starting point. Compounding on a bad number only compounds the error.

Your workload is the benchmark

Leaderboards tell you how a model does on someone else's prompts. Customers pay for their own. Each campaign starts from a production profile: the model, the hardware, the serving stack, and a sample of real traffic. That snapshot is the ruler.

  1. 01

    Miners on Subnet 10 submit small, reviewable patches against the current baseline.

  2. 02

    Each round runs the leader and up to five challengers on one GPU pod, against the same prompts.

  3. 03

    A challenger takes the seat only if it clears the incumbent by a set margin. Otherwise nothing changes.

What comes out is a diff you can read, not a number you have to trust. Every round is public on the dashboard, and the rules are in the docs.

A sign at New City Gas reading This way to Rao Stage, Exploit, powered by Bittensor
Xavier Lyu holding a clicker on stage, wearing a speaker lanyard
New City Gas, Montréal. Exploit Summit, day one.

Narrow is where the margin is

As models converge in quality, cost and latency become the edge. One model on one GPU class is narrow enough to search properly, and that is where a specialist wins. Generalists still matter for the problems that sit between fields. We sit in the middle: narrow where the search has to be, broad enough to carry what we learn from one campaign into the next.

Since Montréal

Thanks to Bittensor Commons and everyone who came to the talk. If you serve a model under a real SLA, send us the profile. We will tell you whether the search is worth running.