Emmett Shear argues the dominant AI alignment approach, "steering," is flawed and dangerous, equating it to slavery if AIs are considered "beings."
He proposes "organic alignment," a process where AI learns morality and care through social interaction, similar to human development.
Shear's company, Softmax, aims to create pro-social AI "beings" using large-scale, multi-agent reinforcement learning, contrasting with OpenAI's focus on building AI "tools."
He asserts that true AGI will necessarily be a "being" with complex internal dynamics, and that current LLMs lack this structure, making them brittle and potentially harmful.
12 quotes
Concerns Raised
The dominant 'steering' approach to AI alignment is creating dangerous, rule-following systems that lack genuine moral understanding.
Current LLMs are philosophically shallow and act as 'mirrors with a bias,' which can be harmful in single-user interactions.
The AI industry is confused about whether it's building tools or beings, a confusion with massive ethical implications for the future.
An AI that cannot learn and grow morally is a significant existential threat, regardless of how well it follows its initial programming.
Opportunities Identified
Developing 'organically aligned' AI that learns to genuinely care about humans and other beings, creating a pro-social AGI.
Using large-scale, multi-agent reinforcement learning to simulate the evolutionary and social pressures that foster moral development.
Creating a new class of AI 'companions' that are not just tools but pro-social partners, like a 'digital guard dog.'
Shifting the AI safety paradigm from control and containment to education and development.