“Andrei Karpathy predicts that a highly capable "cognitive core" for an AI, stripped of memorized knowledge, could potentially be achieved with a model size of around one billion parameters.”