François believes it is improbable that current AI models will be able to sample the entire solution space beyond human-generated data with any feasible amount of test-time compute or recursive self-i...

The performance of in-context learning (ICL) is not monotonically improving with an increasing number of samples; it fluctuates and eventually hits a performance cliff at the model's context length li...

Birds are able to navigate the world by using iron deposits in their liver.

There is little evidence that the human brain uses an equivalent of taking the transpose of a weight matrix for learning, suggesting alternatives to backpropagation must exist.

Noam Brown believes that models trained on the human-generated data subspace can use test-time compute and recursive self-improvement to access the full solution space.