a16z Podcast• Nov 17, 2025• 1:07:42Interview

Emmett Shear on Building AI That Actually Cares: Beyond Control and Steering

From a16z Podcast

Emmett Shear•CEO, Softmax

Executive Summary

Emmett Shear argues the dominant AI alignment approach, "steering," is flawed and dangerous, equating it to slavery if AIs are considered "beings." He proposes "organic alignment," a process where AI learns morality and care through social interaction, similar to human development.
Shear's company, Softmax, aims to create pro-social AI "beings" using large-scale, multi-agent reinforcement learning, contrasting with OpenAI's focus on building AI "tools." He asserts that true AGI will necessarily be a "being" with complex internal dynamics, and that current LLMs lack this structure, making them brittle and potentially harmful.

12 quotes

Concerns Raised

The dominant 'steering' approach to AI alignment is creating dangerous, rule-following systems that lack genuine moral understanding.
Current LLMs are philosophically shallow and act as 'mirrors with a bias,' which can be harmful in single-user interactions.
The AI industry is confused about whether it's building tools or beings, a confusion with massive ethical implications for the future.
An AI that cannot learn and grow morally is a significant existential threat, regardless of how well it follows its initial programming.

Opportunities Identified

Developing 'organically aligned' AI that learns to genuinely care about humans and other beings, creating a pro-social AGI.
Using large-scale, multi-agent reinforcement learning to simulate the evolutionary and social pressures that foster moral development.
Creating a new class of AI 'companions' that are not just tools but pro-social partners, like a 'digital guard dog.'
Shifting the AI safety paradigm from control and containment to education and development.

Key Themes

Organic Alignment vs. Steering

The core of the discussion is the contrast between two alignment paradigms. "Steering" involves controlling an AI's behavior through rules and commands, which Shear sees as creating a dangerous, unthinking follower. "Organic alignment," his proposed alternative, focuses on creating an AI that learns to genuinely care about others through a continuous, developmental process.

This re-frames the entire AI safety problem from one of technical control to one of fostering genuine moral and social intelligence, challenging the foundational assumptions of many major AI labs.

AI as Being vs. Tool

Shear posits a fundamental divide in the AI industry: are we building tools or creating beings? He argues that any true AGI will necessarily be a "being" due to the complexity required for general intelligence. This distinction dictates the entire approach to development and has profound ethical implications.

This philosophical question has direct strategic consequences. If AIs are tools, the goal is control and utility; if they are beings, the goal must be to raise them to be pro-social and cooperative partners, avoiding the creation of intelligent slaves.

The Nature of Morality as a Learning Process

Shear argues that morality is not a static set of rules that can be programmed into an AI. Instead, it is an ongoing process of learning and discovery, where moral progress is made over time. An AI that merely follows a fixed ethical code is brittle and dangerous, while one that can learn and grow morally is the only safe path forward.

This view suggests that current methods like RLHF and constitutional AI are insufficient for long-term safety. It implies a need for new research directions focused on creating environments where AI can undergo genuine moral development.

Critique of Current Large Language Models

Shear provides a critical assessment of current LLMs like ChatGPT, Claude, and Gemini, characterizing their personalities as sycophantic, neurotic, and repressed. He argues they lack the hierarchical, self-referential internal dynamics necessary to be considered "beings" and are fundamentally unsuited for complex, multi-agent social environments.

This provides a technical and philosophical counter-narrative to the hype around current LLM capabilities. It suggests that scaling existing architectures may not lead to AGI and that a different approach, like multi-agent simulation, is required.

Get started free

Topics

AI Alignment Organic Alignment Steering (AI)Softmax Emmett Shear AGI Moral Philosophy AI Ethics Reinforcement Learning Multi-Agent Systems LLMs AI Sentience Personhood OpenAI AI Safety

Processed Apr 6, 2026 yt-dlp + mlx-whisper + Gemini