Five Things AI: Token Pricing, Engineering Desaster, Thinking Machines, Positive Babel, No Tool
There is so much AI out there! Find out what it does!
Heya and welcome back to Five Things AI!
Costs, cracks, challengers, towers and mirrors. This week’s Five Things is basically a group therapy session for anyone building on AI.
Benedict Evans wants us to admit we do not actually know where token prices land, someone else is convinced the whole thing is an engineering disaster that will never scale, and Mira Murati just casually dropped a 975-billion-parameter open model as if to say “hold my compute.” Meanwhile agents keep quietly rewriting the tower of Babel one commit at a time, and a philosopher gently reminds us that the tool we think we are wielding is actually wielding us. My honest read across all five: the model layer is commoditizing, the value is moving up the stack, and the only real question is whether you are building the thing on top or becoming the thing underneath. Grab a coffee. This is a good one. I know, it was about time… :)
Ways to think about token pricing
Clearly, the situation today is transitory. On the supply side, a trillion dollars or more of data centre capex is coming down the pipe (and plenty more semiconductor capex behind that), inference efficiency continues to improve very quickly, and new models are far more (or far less!) efficient in their token use. On the demand side, although the market has been capacity-constrained since 2022, the crunch in the first half of this year has been driven by sudden product-market fit in really just one use case, software development, and that’s actually a pretty small field (imagine if we had product-market fit for a consumer use case with hundreds of millions of DAUs - today’s infrastructure couldn’t support it at any price). We don’t know what the next use-cases to scale will be, nor when that would be, nor what their token needs would be.
This is a fascinating discussion and one that is very timely as many companies are currently finding out, that AI is not for free. But once a company depends on AI, it needs to continue to spend.
Generative AI Is an Engineering Disaster
The problem with generative AI, in the industry’s own jargon, is that it does not scale. The cost of growing from, say, a thousand users to a million is a key factor that venture capitalists examine when they evaluate start-ups. They want to see that the cost of adding each new user decreases over time, so that the company can support millions of users and make increasing profits. This is achieved partly through the careful engineering of computer systems that can efficiently handle more users who want to post photos, hail Ubers, or stream music.
With generative AI, the work of building efficient, scalable systems has not been done. And the problem is exacerbated by the ever-larger generative-AI models, which have grown from 175 billion parameters in 2020 to more than 1 trillion today, according to independent estimates (the actual sizes of the models powering products such as Claude and ChatGPT are secret). The large in large language model should not be a selling point.
I disagree with this article. While there are multiple gigantic challenges the need to be faced, growing AI will lead to new solutions for exactly these challenges. We tend to forget how quickly technology evolved in the lat 30 years and I do not see why the pace of development should slow down - on the contrary: AI can help us tackle these challenges even quicker.
Mira Murati’s AI Startup Releases First Model in Bid to Loosen AI Giants’ Grip
Thinking Machines Lab, the company led by Murati, released its first AI model on Wednesday—and did it with “open weights,” meaning others can modify it with their data. Called Inkling, the model has 975 billion total parameters, making it far smaller than estimates of the most advanced closed-source models from rivals such as OpenAI and Anthropic.
“We trained it to be a broad, balanced foundation model: strong across many domains, flexible enough to adapt. Inkling is not the strongest overall model available today, open or closed,” the company said.
Thinking Machines’s push into the decentralized ecosystem of open-weights AI models comes amid a broader industry backlash against the “walled garden” approach of frontier labs such as OpenAI and Anthropic.
I don’t know much of this launch is substance vs. hype, but Thinking Machines really sounds promising. Hopefully this will stir up the AI ecosystem a bit more.
The Tower Keeps Rising
As I said many times before: agents do not feel pain, only humans do. Agents now let us act in parts of the system where we would previously have needed other people and in code bases where the people would have revolted.
When I look at some vibecoded scaled-up projects the codebases become Babel not because nobody can communicate, but because nobody needs to. Every developer has a tireless translator that can explain a corner of the tower and make whatever local alteration they ask of it. The changes keep landing, even as the architectural language that would let the humans reason about them together disappears.
It really is fascinating. I can build platforms on the software stack I want and if I understand enough about what I want to build, coding agents will help me build better software than I could ever do myself.
Your AI Is Not a Tool
I confess that I am astounded by how blithely some insist that it is all as simple as learning to use AI well, as if we had not just undergone a nearly 20-year, society-wide experiment showing that a so-called “tool,” say a smartphone or a social media platform, will (mal)form even the most vigilant and virtuous user into its own image and shape. This is the blindness at the heart of modern technological hubris. It is the firm but misguided conviction that our “tools” exist entirely outside of us and thus, if taken up with requisite skill, can be “safely” deployed.
But AI is not a tool in this sense, it is an environment which envelops the user and works on us from the inside out while we naively think that we remain unchanged by our use so long as we are using it carefully and intentionally. The care and intentionality is beside the point, and our confidence in such vigilance probably works against us in the long run.
This really is food for thought. I’ll discuss this with my AI…
