Moore’s Law creates deflation.
Chips get faster, so computing costs less. This extends to everything computing touches. It extends to the components, to the use of software, and to every entity using software. We can all now do more with less.
Huang’s Law is creating inflation.
Each generation of Nvidia hardware costs more than what came before. Each generation uses more energy than what came before. The amount of computing power goes up, but AI software sucks up all the gains.
You’re seeing this now, in your computing bills. Software costs are rising. Cloud costs are rising. Cloud Czars insist their AI software will be worth the cost, but it’s increasingly clear their Large Language Models (LLM) are not delivering that value.
My analogy is to Alzheimer’s. Researchers and drug companies became fixated on one theory of the cause, plaques caused by amyloid beta. They put all their money into solutions for reducing the plaques. But when these finally came to the market, patients didn’t get much benefit. The underlying cause of Alzheimer’s remains a mystery.
How people think also remains a mystery. What LLMs are doing are database lookups. They can tell you what’s known, they can give managers and researchers insights into what to do next. But they can’t think. They will never think, because they’re based on compiling existing data. They can only look backward.
There are some benefits in this approach. AlphaFold2 lets biochemists quickly separate proteins worth their time from those that won’t fit in the chemical locks of DNA. Advertising looks better, and executive summaries are being done faster.
But that’s not Artificial General Intelligence. That’s not intelligence at all. It’s the same thing computers have been doing for almost 80 years.
When Will Computers Think?
Real thinking starts with modeling how brains work, but it only starts there.
We must make better mental models from these new insights.
Those are the table stakes.
Today’s LLMs are still dumber than your cat, although the models they use – the neurons and connections analogous to an LLM’s tokens and parameters, are about the same size. But a cat can reason better than the best LLMs, and your brain is over 100 times bigger than a cat’s. (Don’t tell your cat.) Deep learning techniques can speed up the necessary research, but people still must do the work.
The difference between thinking and an LLM is that an LLM can answer questions. Thinking creatures ask questions.
Meanwhile the cost of building and running LLMs continues to grow. The incremental benefit of each new model declines. The near-term future belongs to smaller LLMs that can prove their value in helping thinking people get more done.
The Cloud Czars are going in the wrong direction. They’re creating inflation while doing so. I hate to be the Cassandra I was in the late 1990s, but we’re still heading for a crash.
