AI Demand Fuels Surging GPU Prices: What's Really Going On?

GPU prices are soaring, fueled by an insatiable demand for AI compute power. The reality is, the market isn't cooling down anytime soon. Silicon Data's CEO, Carmen Li, reveals that both older and newer Nvidia GPUs are becoming pricier. Let me break this down.

A Price Surge with No End in Sight

Silicon Data's indexes tell the story. The Neo Cloud H100 index jumped 20% in three months, while the Neo Cloud B200 index saw a 22% rise. Even the Hyperscaler H100 index climbed by 3%. This indicates a demand that far outstrips supply, leaving prices high and climbing.

Traditionally, new chips would debut with high prices, gradually dropping as supply improved. Not here. The B200's pricing remains high, showing that supply constraints are still biting hard. Why aren't prices normalizing? The architecture matters more than the parameter count. AI's hunger for chips remains relentless.

The Cloud Premiums

Hyperscaler environments like Amazon, Microsoft, and Google charge hefty premiums for GPU rentals. Renting an H100 through them can cost almost three times as much as with specialized services like CoreWeave. Why the gap? Customers pay for the convenience and integration these giants offer.

Carmen Li, who founded Silicon Data after leaving Bloomberg, aims for transparency in this opaque market. By normalizing hundreds of thousands of pricing points into indexes, she provides a clearer picture of both spot rentals and long-term values.

Depreciation? Hardly

Here's what the benchmarks actually show: GPU prices aren't just stable, they're rising. Even older GPUs like the H100 aren't depreciating much. Li notes you can sell a refurbished H100 for 85 cents on the dollar in the second year and 84 cents in the third. My car loses value faster than that!

The demand for AI isn't just high. it's off the charts. This impacts infrastructure budgets and the economics of AI products relying on rented GPUs to power their operations. So, what's the takeaway? As long as AI's appetite for processing power remains, expect GPU prices to stay high. For those betting on a price drop, the numbers tell a different story.