Keep pulling the thread on Ilya Sutskever.
OpenAI's founding goal has consistently been to ensure that artificial general intelligence (AGI) benefits all of humanity.
OpenAI transitioned from a non-profit to a 'capped-profit' structure because a non-profit model could not raise the capital required for large-scale compute clusters.
The release of GPT-3 demonstrated to the broader community that training increasingly large transformers on more data was a highly effective formula for AI progress.
Future large language models will become more reliable, be able to perform more tasks, and have deeper insight into the human world.
The primary capability lost when using smaller language models compared to larger ones is reliability.
The biggest bottleneck preventing current large language models from being truly useful is their lack of reliability.
In the long run, the question of whether highly capable AI models should be open-sourced will become more complicated and the argument for it less clear-cut.
The most significant near-term limit to scaling large language models is the availability of training data.
The limitation of finite training data for scaling large language models can be overcome with further research, allowing progress to continue.
It is plausible that within 5 to 10 years, there will be data centers that are smarter than humans, not just in memory or knowledge, but in having deeper insight.
The goal of the Superalignment project is to develop the science to ensure future superintelligent systems have a strong desire to be kind and beneficial to humans.
The successful development of AlexNet was enabled by the recent application of GPUs to machine learning.