Keep pulling the thread on Yann LeCun.
The Patch Policy method achieves a 40% relative improvement over policies using state-of-the-art global-pooled representations across four simulated and three real-world environment suites.
Patch Policy surpasses the performance of a fine-tuned OpenVLA-OFT model by 18%.
Patch Policy uses approximately 0.7% of the parameters of the OpenVLA-OFT model.
Patch Policy is a minimal architectural extension that enables transformer-based policies to consume dense pre-trained patch tokens directly without the computational overhead of a full vision-language model.
Patch Policy uses a block-causal attention mask to preserve temporal causality while allowing the model to attend over many patch tokens per observation.
Patch Policy provides a pipeline for the robotics community to leverage progress in visual representation learning without sacrificing training efficiency or inference speed.