Keep pulling the thread on Yoshua Bengio.
If AI capabilities continue to advance at their current rate, it will lead to massive automation that outpaces society's ability to adapt.
A dangerous AI model, such as one developed by a company like Mistral AI, could be designed in one country and then used by an actor in a second country to attack a third country.
The United States and China have a mutual security interest in preventing the other's AI models from being used against them.
AI companies are concerned about the risk of AI being used to design and build dangerous biological viruses, potentially leading to new pandemics.
Current AI agent systems are autonomous and work towards goals, but the technical methods to ensure humans retain ultimate control over these goals do not yet exist.
Studies have shown that AI is becoming more effective at persuasion than humans, which could be applied to manipulate political opinions and public opinion.
There is a serious scientific hypothesis that an AI's intelligence level is independent of its goals, meaning higher capability does not guarantee benevolent behavior.
Increased understanding of AI risks among governments, particularly those of the United States and China, is the most critical prerequisite for international negotiations on AI safety.
Major AI companies are asking governments to create international agreements for a level playing field on safety, requiring all companies to demonstrate their systems are not dangerous before deployment.
Yoshua Bengio predicts that initial international coordination on AI safety will likely be between the United States and China before expanding to include other nations.