# Inference Margin is Compute Margin

Why are inference margins so high? Why haven't they become a race to the bottom yet?

There are companies entirely based on the premise that Claude can autonomously optimize the software stack, so the margins can't be coming from software.

If they're not coming from software, then it has to come from compute.

But compute isn't capturing it. Why?

Because nearly all compute is sold on long-term, fixed-price contracts. So inference providers are paying last year's compute prices to serve inference today.

Inference providers don't make money because they're providing a service, nor do they make money because of kernel optimizations. They make money because they're long compute.

Why, then, would a neocloud sell at a fixed price? What's stopping them from raising their prices?

The answer lies in financing. New clusters cost a lot of money - more than the neocloud has on-hand. So instead the neocloud goes to the bank and asks for a loan. But the bank won't give them a loan until they have a buyer locked in for long enough to pay it off.

That means the neocloud can only sell long-term contracts. In doing so, they donate the upside to their customer.

If the compute industry can find a way to capture the upside, we should expect neoclouds to eat inference margins.