Five Things Tech: European Tech, Bio Research, Space Data Centers, Pokémon Go, Vintage LLM
This is everything you should read about Tech right now.
The big news this week is obviously the SpaceX IPO and while I am happy for many people who put in lots of great work in the last decade or so and who now have their huge payday, I am ever more in favor of curbing the power and influence of individuals. We need more regulation to make sure that we do not get trillionaires. A billionaire should be the very rare exception. This is not healthy. Also, I predict a rocky road ahead for SpaceX - Musk overpromises and underdelivers. The rocjet part of SpaceX itself might be profitable, but all the other companies Musk lumped inside of the shell will continue to lose money. I sincerely hope Musk will hop on a rocket to Mars real soon now.
All the Ways Europe Is Ditching American Technology
The moves are widespread—and growing. Last week, the European Commission launched its official long-term plans to rely less on US technology. The European Parliament has switched the default search engine on its devices from Google to the French alternative Qwant. Thousands of workers in the French government are using its own open-source office software —dubbed LaSuite—as officials aim to “break free” from dependence on American tech firms. An open-source documents offering from more than a dozen European tech companies, called Euro-Office, is due to launch imminently. Cities across the Netherlands, France, and Germany are all moving away from Microsoft Office and Google Docs
It’s not just productivity software, either. The Dutch government is moving its code away from Microsoft-owned Github to its own repository. In a series of decisions, Finland reportedly decided not to move its election data to Amazon’s cloud services, while the organization behind Belgium’s.be top-level domain has said it will move away from AWS. Meanwhile, Eurosky has been spun up as an interoperable alternative to Bluesky on the AT Protocol that underlies both social networks.
Don’t get me wrong, there a lots of really great US tech products. But we need to find our own path in Europe and reduce the dependency on US tech.
AI can design and run thousands of lab experiments without human hands. Humanity isn’t ready for the new risks this brings to biology
Hands-on work in the lab has traditionally been a bottleneck to translating designs into results. Even a brilliant study plan still depends on skilled human hands to carry out. That may not last, as cloud laboratories and robotic automation become cheaper and more accessible, allowing researchers to send AI-generated experimental designs to remote facilities for execution.
AI systems are now able to run experiments autonomously and at scale, but existing regulations were not designed for this. Rules governing biological research do not account for AI-driven automation, and rules governing AI do not specifically address its use in biology.
We need so many new ways to define guardrails, otherwise we cannot keep up with AI.
The Real Cost Of Cooling GPUs In Space Might Shock You
Proponents tout the many wonders of computing in space: abundant solar energy, free cooling, and freedom from Earth-based disturbances like earthquakes, floods, and protesters. But a sober look at the physics of space-based computing paints a much more nuanced picture.
Free cooling is perhaps the biggest misconception. Space is cold, but it also has no atmosphere. That means the best heat-removal mechanisms, conduction and convection, are off the table. The only option is radiation. To prevent a chip from overheating in space, a large, costly surface area is required to dissipate the energy and then radiate it.
Solar energy is abundant, but collecting it with functional solar panels that maintain perfect alignment toward the sun is a complex task requiring extensive attitude control systems. On top of that, ionizing radiation in space from cosmic rays and other sources poses a unique challenge, degrading the solar panels, the radiative coolers, and the chips themselves. Because regular maintenance in space is difficult, redundancy has to be built in at launch, and cost estimates have to account for efficiency degradation over time.
I really don’t see why people remain so optimistic about datacenters in space. So much hype, just for one IPO…
Pokémon Go data trained AI that could assist military drones in war zones
Niantic Spatial – a spin-off company from Niantic – announced its partnership with Vantor, a company that specialises in spatial detection software for drones, including those used by some militaries, in December.
The agreement is designed to allow drones to navigate and coordinate precisely in areas where GPS is not available.
Pokemon generated so much data and this is now being used to train drones that could get deployed across our cities. What a time to be alive.
Making a vintage LLM from scratch
In short, to build an LLM you need 4 things:
the data -- an LLM has no discernment or understanding. It will learn from anything you tell it to, good or bad. This is the longest process.
tokenization -- the Tokenizer is a little program that converts words or letters into numbers (tokens). LLMs don’t understand words, they only understand numbers.
pre-training -- it’s a confusing expression and it means “base-training”, where the LLM learns to autocomplete text. If you’re going for a 300m+ params, this is the most expensive process.
fine-tuning -- where the LLM learns how to chat in turns, question & answer.
This is so cool. Not really useful, but I guess if you really want to understand how an LLM works, you have to build it yourself.
That’s all for now! Thanks for reading! If you missed last week’s Five Things Tech, you can find it here:
🤖
— Nico