In many conversations I’ve had with business leaders and policymakers, the real costs of AI keep cropping up, no doubt triggered by the seemingly large numbers spent by AI companies on chips and power-purchase agreements. Nvidia’s top customer spent $4.2 billion in the last fiscal quarter on Nvidia chips and services, while Microsoft signed a record-breaking estimated $10 billion purchasing power agreement with Brookfield.
These eye-popping numbers are the result of a land grab to build out AI platforms. Demis Hassabis, head of DeepMind, has said that Google will spend over $100 billion on AI over time. Demis’ boss, Sundar Pichai said “The risk of underinvesting is dramatically greater than the risk of over-investing for us here.” Many think that this is money being wasted. Or is it an unsustainable loss leader that will snap back?
Perhaps this money is being wasted, but I’m not sure what the problem is: if a big tech firm is ‘wasting money,’ does it need our sympathy? Its investors can sell their stock and move elsewhere. If their managers mess up, they can be fired. Life goes on.
But anyway, that isn’t what is happening.
The cost of providing AI services (whether LLMs or something less) at a given quality is declining rapidly. And doing so for two reasons:
a. Exponential trends in processing—Huang’s Law is the new Moore’s Law, but faster. Every year, Nvidia expects to deliver chips that are two to three times cheaper at the same performance than the previous year. That is double or faster than the Moore’s law cadence.
