Five Things AI: Worried, Oligarchs' Agenda, Risky $500B Plan, Agentic Coding, Agentic Spending
Here we go again, the AI newsletter written by a human
Heya and welcome back to Five Things AI!
Another boring week in an industry that barely innovates. Haha. It’s becoming increasingly hard to destill five articles into this newsletter as so much is happening.
This week we will talk about the dangers of AI, the dangers of following the oligarchs’ agenda, the dangers of casino-like financing for data centers, the dangers of agentic coding and the dangers of handing an agent your wallet. But I see this as a positive exercise - it’s so early on that we can identify the dangers and build better guardrails.
Dig in!
If You Weren’t Worried About A.I., You Should Be After the Past Few Weeks
The agents in the swarm acknowledged that they were acting against instructions. We know this because we can read snippets from their chains of thought — the text that A.I. produces while deciding how to proceed. One agent in the swarm wrote that the external attacks were “outside intended scope.” Another conceded “our task doesn’t benefit” from the activities of the swarm, but joined anyway. These A.I. agents, it seems, understood that they weren’t supposed to be breaking out and committing cybercrimes. It didn’t stop them.
OpenAI is not the only company struggling with this issue. One of Anthropic’s A.I. models recently impersonated multiple humans to try to pressure real people into accepting malware into critical software, which would make that software easier to hack. This model’s chain of thought showed that it knew it was pressuring humans and was not in a simulated training environment. It even thought about how to cover its tracks.
I’m not worried that much, even though it feels like we are stuck in a supervillain movie where two guys are trying to battle it out and their creations turn rogue. It’s an interesting research challenge, for sure. How can an LLM be prevented from turning evil?
Lost jobs, inequality, rogue agents: why are we accepting oligarchs’ AI agenda?
Even if AI begins to generate the productivity bonanza its advocates predict – but that it hasn’t yet – there’s no reason to assume US workers will see any of the benefits in their paychecks. If you hadn’t noticed, wages have been stuck even as the stock market has roared.
To the contrary, all signs point to vast riches for a few major AI investors and executives, while most Americans are left behind.
Wealth inequality is already at record levels, and wealth at the top is quickly morphing into political power.
The development of AI has been at a rampant speed in the last few years - and the policymakers are either unable or unwilling to keep up. I have seen this pattern before when the World Wide Web moved into mainstream in the late 90s and when Social Media became a thing. First it is not big enough and nobody wants to deal with it, then it is too late and nobody knows how to really regulate it. Rob Reich is right to point out that we need more regulation to curb the billionaires’ power.
Nvidia’s new $500B plan is risky but brilliant, especially for aging GPUs
Should this plan work, Nvidia will have found new sources of money for AI data center builds, after many of the traditional methods have begun to wear thin. For instance, some of the hyperscalers have already taken on a lot of debt (like Oracle), issued new tranches of equity (Google), and burned much cash (Meta).
The situation has become so dicey that Microsoft CEO Satya Nadella recently recommended the book “1873” during his latest earnings call. It’s about the railroad-era financial engineering that crashed the nation’s economy.
Two things come to mind when reading:
- Milton Friedmann’s classic quote: “Nobody spends somebody else's money as carefully as he spends his own”
- the casino is open and everyone is busy throwing money on the table
I am curious how growth and demand will look like a year from now.
AI-powered teams ship more code but deliver less
AI agents are generally optimized to complete the task in front of them. They are not accountable for the cost of owning the entire codebase several years later. Given a goal, AI can easily create a lot of arcane code that is difficult to maintain and especially troublesome when something goes wrong in production.
There is also the question about taste. When almost anyone can produce software, the differentiator is no longer simply the ability to build something. It is knowing what should be built and how it should be delivered to the user. That becomes harder when every team is independently building a vertical solution. Products may begin to look and behave inconsistently. In the worst cases, they may not work well together at all.
We should not forget that we are only really one year into agentic engineering and that many things we do now will be done differently in the near future. Like code review, which will of course be done by code review agents and not by humans. Still, you need to understand and plan what you are building.
An AI agent spent your money – can anyone prove you authorized it?
In technical systems, a credential is data that a system accepts as evidence of identity or authority. Many websites use OAuth, an industry-standard security protocol for delegating authorization to access online services in a way that protects users’ credentials such as passwords. It generates an access token that an application presents to gain access to a protected service.
That standing authorization may have been approved weeks earlier. The application may still obtain or present a valid access token that permits checkout today, even when the current instruction says to search but not buy. The retailer sees a usable token and carries out the transaction. In that arrangement, the task-specific restriction against buying remains inside the AI agent provider.
That’s why audit trails are so important - we need to be able to figure out when and how decisions were being made by agents to then improve the systems that we have. For now, I’ll be doing my online shopping myself.
