Keep pulling the thread on Dr. Terry Sejnowski.
Geoff Hinton's argument for language model understanding is that the ability to accurately predict the next word in a sequence requires the model to have developed a sophisticated internal model of semantics.
A fundamental limitation of ChatGPT is its lack of inherent, evolutionarily-derived goals like survival and reproduction, which are core to biological intelligence.
A critical difference between large language models and human brains is that models like ChatGPT are completely inert and have no self-generating activity between user prompts.
A significant limitation of the current ChatGPT architecture is its inability to learn continuously from user interactions, as the model is static after its initial training phase.
The development of the Boltzmann Machine led to the introduction of the backpropagation algorithm by Geoff Hinton and David Rumelhart.
The deep learning revolution began with the ImageNet competition at the NeurIPS conference around 2012-2013, which demonstrated a breakthrough in object recognition in images.
ChatGPT's ability to adopt various personas stems from its training on a massive dataset that includes thousands of books and numerous articles.
The AlphaGo model defeated Go champion Lee Sedol in a match in Seoul in 2016.
In its 2016 match against Lee Sedol, AlphaGo made a pivotal and unorthodox move, known as Move 37, which ultimately led to its victory.
Dr. Terry Sejnowski views hallucinations in large language models as a form of creativity, rather than simply an error or untrustworthy output.
The key to AlphaGo's success was its use of self-play, which enabled it to discover novel strategies beyond the scope of conventional human play.
The NetTalk project, developed with graduate student Charlie Rosenberg, used a neural network with 200 units and 20,000 connections to successfully learn the complex rules and exceptions of English pronunciation.