Mama Mia!

Sometimes you’re not even allowed to even suggest that AI might be a thing that exists

It’s healthy to change mind in light of new evidence. AI has improved since GPT 3 days - in particular in coding tasks.

“This is a BIG merge window,” he wrote in his weekly state of the kernel update, as it contains almost 20,000 commits compared to the usual ~15,000 that developers submit for a new cut of the kernel.

Great to see so much momentum!

Anti-intellectuals think science is like in the movies: write gibberish on a chalkboard and time travel is suddenly invented. “Who needs experiments? My equation is correct!”

Wake me up when their AI can propose and conduct novel scientific experiments.

Maths is an “easy” target for AI because it’s self contained. It’s easier to brute force training data in mathematics when there’s no external forces.

Can’t have that cheat code in science, unless they let their models study the behavior of real ant colonies as part of their reinforcement learning process.

I prefer Java style:

entity.position = entity.position.add(entity.velocity.scale(dt));

Not hard to read.

Operator overloads is something that sounds like a good idea until you experience them firsthand. Just properly name your functions for fuck’s sake.

There’s a difference between pre-training and reinforcement learning.

Pre-training is when the models are trained on scraped data. The models basically learn how to parrot the Internet. Usually they have a cutoff point to avoid training on slop.

Reinforcement learning is where most of the training is happening. They’re given tasks (for example, make a calculator app), and then judged by how well they complete the tasks. This is more resistant against model collapse because they’re not judged by how well they parrot slop.

As part of the reinforcement learning process they give the model access to tools like searching the internet so the newest programming language features can be accessed while the tasks are performed. This is where OpenAI is shifting to a cached version of the Internet the models can interact with instead.

Unlikely. They’ll need to train the models so they can perform their tasks even if the internet is full of slop.

There’s so much AI slop out there that the frontier labs risk model collapse if they continue to train on it.

That’s why they’re resorting more and more on reinforcement learning. Give the AI a task. Quantify how well the AI performed the task. Less parroting what’s said on the Internet.

They’ll still scrape the internet. The difference is that the models will only have access to a cached version of the internet during the reinforcement learning process (for example, check the documentation of a programming framework). I’m surprised it took them this long to realize it’s a bad idea to do reinforcement learning with live internet access.

Translation: they’re still going to scrape internet. They’re just no longer going to perform reinforcement learning with access to live internet (instead they’ll use a cached version of the internet).

They’re going to use AI to install the AI.

Recursive Self Installation

Wordpress is still commonly used for e-commerce. You’re probably visiting more PHP sites than you realize.

There might be “better” languages, but PHP is still used by ~70% of the web.

四

I know of some powerful people I would want to write the name of. I’m not sure if it will end the wars and suffering they’ve caused, but I believe the world would be a better place without them.

The entire Internet went into panic mode yesterday because someone used 4 as their password.

Most of Internet is built upon the principle ”A fella I trust is trusting some other guy, so I will do so too”.

Turns out anyone can easily impersonate as your trusted fella, so now that trust is meaningless. Now you need to find all IoT devices out there so you can tell them not to trust this fella anymore.

In short, everything is fucked.

Oh, I thought this meme was somehow related to the remote code execution bug in PHP, but turns out I missed the other calamity event yesterday.