The Compute Arbitrage

Every discussion of AI infrastructure begins with training. The cluster is the unit of ambition: how many accelerators, how much interconnect, how many megawatts behind the fence. That framing made sense while capability was the scarce input. It stops making sense the moment a model enters production, because production is not a single event on a single site.

It is millions of small requests, arriving at inconvenient hours, from places the model was never built near. The economics of that shift are unglamorous and decisive. A training run is a capital project with a start and an end. Inference is a utility bill that never stops arriving.

Once inference dominates a provider's cost base, the questions that matter stop being about peak performance and start being about utilisation, latency, and the price of a kilowatt-hour at three in the morning, local time.

Read the report →