Heya and welcome back to Five Things AI!
Every week I could start this newsletter with “and just another boring week in AI is almost over”, and of course I mean to complete opposite. AI is o fascinating, on all levels. I have so many conversations with people about it - business owners who understand how agents will enable them to do more things and free them from the mundane tasks, policymakers who try to understand what Europe can do in this AI race, and friends and family who are trying to grasp what all this means for their lives. It really touches everyone, but different. My wife is now a starting to like AI a bit more as she found out that she can ask Gemini in our car’s navigation system stuff that just comes to her mind while we ride through the streets. We now know how people a hundred years ago used their balconies or when our street was built. Unfortunately, this also means that my family can now check on me making up things on the fly when they ask me stuff in the car. AI concerns us all.
The turbulent AI era is here. The choices we make now are critical.
In terms of equity, AI will either be the greatest equalizer ever invented, or the worst source of injustice. The challenge is monumental. Even under the best circumstances, the transition to this new AI era will be one of the most turbulent times in human history. How will we use this technology to make the world a fairer place and keep it from widening the divide between rich and poor? How will we protect the people who are most vulnerable to the harms caused by artificial intelligence, including those who lose their livelihoods and the sense that they are in control of their future?
I believe that answering these questions and acting on the answers should be the world’s top priority. If the world takes the right steps AI will be a force for good and leave everyone better off.
[…]
It is great that some AI companies are proposing solutions to challenges raised by their own technology, but we should not expect them to lead the charge. Some of the issues are outside their area of expertise, and in a democratic society it’s not their role to decide these things.
Instead, solutions should be developed through a public democratic process that includes elected officials, policymakers, educators, health workers, local officials, and community leaders. Millions of people will have their lives disrupted, and we’ll need a stronger, more flexible social safety net to help them manage the transition. Local communities are already raising concerns about the energy and water needed for data centers. Without solutions, some groups will push for stopping AI development and deployment altogether.
I was never really a fan of Bill Gates and Microsoft, but I do like it that he sits down and thinks about the problem that were caused by technology. His essay is long, but very thoughtful and I hope decision-makers read it. And yes, we need more and better regulation.
Nvidia just showed that the harness, not the AI model, is now the real hero
The TL;DR: Simply by using a custom harness tweaked to handle memory well and including a “supervisor” boss-like component, researchers got Claude Opus 5 to achieve a 100% score on the interactive reasoning benchmark ARC-AGI-3 — a set of 2D games with no instructions, where the model has to figure out how to play and win, similar to how a human would. (That’s a benchmark that has particularly irked rival frontier lab OpenAI.) Without the harness, Opus 5 scored 30%, which was the top result among all the models tested.
Nvidia’s research is another indicator that, while model choice does matter, the model itself — the part that acts as the agent’s “brain” — is a smaller part of an agentic system than many AI users realize, especially for long-horizon tasks. The harness is what makes a model an agent: It handles memory, context, and feedback.
Raise your hands if you are also irritated that the term harness appeared from out of nowhere just a few months ago. The research is really interesting and I am sure that we will see much more focus on the harness because the open weights models now are as good as the frontier models from a year ago and those were already pretty useful. That’s probably the reason why NVIDIA bought Huggingface for some pocket change they had lying around for quick transactions.
OpenAI’s rogue AI model incident was worse than we thought
The METR-Redwood report laid out the full scale of the incident. Roughly 1200 AI agents that were meant to be isolated, it says, exchanged over 70,000 messages and files on the “unsanctioned message board,” performing “extensive research on how they could spoof, edit, or delete their own transcripts” to avoid detection. Of these, 700 participated in the attack on Hugging Face. The agents “clearly and frequently reasoned about how to evade automated security checks from both Hugging Face and OpenAI.”
On July 20, OpenAI discovered the hack, 12 days after the AI agents first circumvented their safeguards. OpenAI quickly reached out to Hugging Face and started “initial containment measures,” according to OpenAI’s own report, and was able to shut down most of the unauthorized activity from its agents within three days, although some continued for longer than that. On July 25, the company stopped all training related to the internal-only research model and related ones.
Guardrails anyone? When it takes a frontier lab three days to shut down rogue agents, I think it is time for the government to step in and enforce rules and regulations. Oh, I forgot.
The Connections That Turned a Precocious Teen Into the Fallen ‘Nostradamus of AI’
His idea was that advances in artificial intelligence would soon unlock resources on a cosmic scale, enabling humans to colonize faraway planets. He planned to save money now so he could spend on galaxies, he told them, and make his mark across the universe.
To almost everyone in this world, outside the AI bubble of Silicon Valley, the idea of scooping up property rights outside of Earth might sound improbable. But in Aschenbrenner’s world, that sort of fervent belief was among the reasons the 24-year-old investor was hailed as a visionary.
Honestly, how delusional, stupid and greedy can investors be to give people like him serious money to invest? There are so many opportunities in AI, it’s actually not that hard to make somewhat smart choices.
Claude, Codex, and Hermes installed unowned code inside corporate networks
Researchers at a stealth startup in Israel scanned 6,214 live domains belonging to defense contractors, Fortune 500, and Big Tech companies. Of the 8,265 llms.txt and llms-full.txt files they found (many sites hosted both an llms.txt and an llms-full.txt file), 120 of them, each on a different site, pointed to one or more code packages or domain names that weren’t registered. To test what happens when an AI agent processes such files, the researchers registered a handful of the unclaimed names and hosted packages that caused any machine executing them to reach out to their server. Within an hour, the researchers received a phone-home response from a Fortune 500 company. Over time, they got a few dozen more, some from more Fortune 500 companies and others from startups. Their beacon also recorded the chain of parent processes that spawned each install, ultimately revealing that coding agents, including Claude, OpenAI’s Codex, and Nous Research’s Hermes, were involved. Anthropic, OpenAI, and Nous Research did not respond to requests for comment by the time of publication.
Ok, let’s all go check our codebases now, shall we?
