humanity only has one engineering project, building better engineers than humans. After that, the thing we built can do the engineering.

clips have been making the rounds on Twitter about the “bishop guy” in a chess engine, or a “cone guy” in a self-driving car. these engineering ideas look ever more ridiculous.

how cool would it be if we could make a machine that could drive cars like a human. obviously there’s no reference to traffic cones inside human DNA, they learn about them from data. So there shouldn’t be any reference to traffic cones in this codebase.

Rich Sutton stated this most iconically in 2019.

One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

but then where does it stop? why draw the line at DNA? evolution is clearly a search and optimization process. why is hard coding stuff that’s in the human DNA okay, why not evolve a driving agent? why not evolve life?

the concept of a Seed AI is very captivating. build a minimum viable self improving AI, and allow it to bootstrap its way to human and beyond. this is clearly possible, evolution did it (though with an ungodly amount of compute).

but remember, our goal is just to build something superhuman, not go beyond. unlike a self-driving car, if you were building a train driving agent, learning like a human is probably not the right choice. it’s simple enough to code and test. you should have a train_signal.py

i say the bitter lesson stops at human DNA. while we have to avoid cargo culting, i think that many of the pieces of the human brain are starting to be buildable with today’s technology. the brain isn’t some hyper elegant machine that captures the essence of learning, it’s a bunch of hacks, and hacks that we can replicate.

having a cone guy in your self-driving car is still ridiculous, because the only working implementation of a driving agent doesn’t have a cone guy, it learns cones from data. but the only working implementation of engineers do have a neocortex, a hippocampus, a basal ganglia, an amygdala, and a thalamus.

i would be fine with human agent software having directories for each one of those pieces. transformers as a neocortex, some way better RAG as a hippocampus, actually working TD-learning as a basal ganglia, an amygdala to prevent the robot destroying itself, and a thalamus to coordinate the system and search.

we should just try to build knock-off humans, not solve life. they can do that.