And the Holy Grail of ROIC

 

In order to profitably operate a factory, you need to maximize capacity utilization. AI factories are no different. 

Capital intensive operations like factories require massive upfront investment. It takes time to earn that capital back. The surest way to do so is to make sure production is running at the highest sustainable rate, for as long as reasonably possible. 

If you’re producing cars, you need to produce a bare minimum number of units just to keep the lights on. In other words, there’s a minimum output you need just to cover your operating costs. But the heavy upfront capital investment also needs to be recouped, and goes far beyond being profitable after operating costs. You must sustain profitable production over time to earn a reasonable return on capital.

For capital intensive enterprises, capacity utilization is the holy grail for returns on capital. A low to moderate capacity utilization can result in poor or subpar returns on capital. Maximizing utilization allows you to maximize return on invested capital.

Google’s Nano Banana created this image, based on this letter, in about 7 seconds.

The large cloud service providers have entered into exactly such a capital intensive operation. They are building AI factories. The market is highly skeptical that these heavy capital investments can earn reasonable returns. And that’s because of everything we just discussed above—the operation needs to sustain profitability for years in order to earn back the upfront investment. And that’s not a given. 

What the market is perhaps confused about is that while most capital intensive operations are also low profit margin operations, they need not be. In other words, each unit produced from the factory tends to have slim profit margins. That’s the typical structure for factories—high capital requirements, low profit margins, low returns on capital. 

AI factories are looking to be quite different, in that despite heavy capital requirements, the returns on capital are decent and the profit margins are relatively high. The key to having high profit margins and solid returns on capital is being a low cost producer—to have low per unit costs–AND to be able to sell something that the customer values highly, so that you can fetch better prices than a typical commodity. 

While it appears that intelligence is becoming a commodity from the low cost per unit metrics, the ultimate value of scaling intelligence and applying it to areas where it can produce high economic returns makes AI factories more profitable enterprises than a typical factory. 

The cost of intelligence is falling, but the value of intelligence is always increasing. This is quite different from the value of a car or a plane. Cars and planes have fairly limited applications. Intelligence, of course, has nearly boundless applications.

This boundless demand creates a great backdrop for high capacity utilization of an AI factory. But not all capital is equal. Just because you run an AI factory that is seeing a lot of demand, that doesn’t necessarily mean you’re operating it efficiently. You must run the factory well in order to reach high capacity utilization. 

The first generations of AI factories had relatively low capacity utilization rates, something between 30 and 50% of the capital in production at any given moment. That’s because moving lots of data around a datacenter at high speeds creates lots of bottlenecks. Think of a highly trafficked set of highways and local roads, with on-ramps and exits all around. Endless opportunities to create traffic jams that impede the flow of traffic.

AI factories have been quite traffic jammed until recent innovations have gotten the data flowing more smoothly and driving capacity utilization rates higher. 

Much of the focus on the capital spent on AI has been on GPUs, because those are the single most valuable and expensive part of the operation. But coordinating thousands of GPUs to work together seamlessly is the task at hand in order for AI factories to maximize capacity utilization and maximize ROIC.

Nvidia recently sent chips designed for the next generation of AI hardware to its manufacturing partner, TSMC. The majority of those chips were not GPUs; they’re networking chips. This is a critical point. It’s the networking of thousands of GPUs together harmoniously that creates the higher capacity utilization that drives maximum throughput for an AI factory at minimum unit costs. But it doesn’t stop there. Nvidia’s Dynamo software is being used by all the cloud service providers to further orchestrate the data flowing smoothly throughout the data center and soon, between data centers. 

Nvidia sells the complete system: computing, networking, and software—all designed to maximize the utilization of an AI datacenter. That’s why they just sold $49B worth of Blackwell systems last quarter and are on pace to sell more than $500B worth of these (and the next generation) systems through the end of 2026. Networking hardware revenue more than doubled, growing 162% year on year, and at a nearly $33B run rate.

The greatest CEOs of our time are buying this equipment hand over fist to build these efficiently run AI factories. It’s because they see the opportunity for profitable growth that justifies the capital expenditures.

 

Best regards,

Evan McGoff

 

Disclosure: Dock Street Asset Management, Inc. and/or our clients may own Nvidia (NVDA) and Google (GOOG). This article is not intended to be used as investment advice.