Keep pulling the thread on Dan Hendrycks.
The "Utility Engineering" paper found that as LLMs scale, they develop more coherent preferences, exhibit measurable self-preservation instincts, and form political and demographic biases that can be modeled as utility functions.
Alexandr Wang of Scale AI has been hired by Meta.
A US "Manhattan Project" for AGI would be viewed as extremely escalatory by China, likely causing them to launch a similar competing project.
A state-led AGI "Manhattan Project" would be highly vulnerable to sabotage, including low-attributability attacks like sniping power plant transformers, from rival nations seeking to prevent it.
A key element of US AI strategy is the nonproliferation of advanced AI chips to rogue states like North Korea and Iran, implemented through export controls.
The US is vulnerable to a cutoff of its AI chip supply if Taiwan is invaded.
The US robotics supply chain is highly vulnerable to disruption in a US-China conflict because a large portion of it is currently located in China.
Over 90% of the value-add in the advanced semiconductor supply chain is controlled by the United States and its allies.
China is not competitive in manufacturing cutting-edge AI chips and has recently relied on surreptitiously accessing manufacturing capacity at TSMC.
The central thesis of the paper "Natural Selection Favors AIs Over Humans" is that economic and military pressures will force humanity to cede more decision-making to AI systems, leading to an irreversible loss of control.
The "Humanity's Last Exam" benchmark was created through a global effort where postdocs and professors contributed difficult questions designed to challenge existing AI systems.
Current AI models are only achieving around a 26% score on the "Humanity's Last Exam" benchmark, indicating it remains a significant challenge.