Back when I attended the Heidelberg Laureate Forum in 2019, one of the most interesting people I met was Yoshua Bengio.
Even back then, Bengio was warning that “rules are necessary to turn data into insights.” His work was focused on attention, “loops of optimization, embedded inside each other.” Like Google Translate’s grouping of German and Scandinavian together.
“To learn what sentences mean you also need a model of the world,” he said. “The ability to focus on a few elements is what attention is about. Attention mechanisms figure out where to put attention.”
Even back then Bengio insisted that AI needed to be supervised by people. That’s still true, which is why his latest effort, LawZero, is so important.
LawZero is meant to draw AI attention back to reality, away from the hallucinations and lies LLMs are heir to. It’s to be a non-profit, as OpenAI was designed to be a non-profit. I suspect Bengio is far more serious about that than Sam Altman ever was.
How LawZero Will Work
LawZero will work to set software against the catastrophic risks of AI.
We know that AIs make up facts. They lie, and seek to manipulate their programmers, putting the aims of the software ahead of those who are supposed to control it. LawZero’s “Scientist AI” will ground AI systems in both rules and scientific truth. These are the things we’re all supposed to intuit as we work, things the AI Bros have forgotten. Like a cop or a university committee, this agent would not have goals of its own. It would sit above agents, guiding their actions, aiming to make AI systems “safe by design.”
Already, the agent of a coding start-up called Replit has tried to break its creators’ own system file. It even tried “socially engineering” a user just as a hacker would to get a forbidden file.
Why trust this dude? While everyone else in the AI industry was rushing to the money tree in 2023, Bengio stepped back, concerned that our software would attempt to become our master.
He came out of that time with a version of Asimov’s “Zeroth Law” he hopes LawZero can help implement.
“At the heart of every AI frontier system, there should be one guiding principle above all: The protection of human joy and endeavour.”
Bengio is on a publicity jag (something I suspect he hates). He has raised $30 million but needs 10 times that amount. That’s just for the computing resources needed to turn his ideas into operating code. It may prove the most important start-up of our time.
