Keep pulling the thread on Jack Parker-Holder & Shlomi Fruchter.
A key capability of Gini3 is 'spatial memory,' which allows for object persistence in the generated world even when the user looks away and then looks back.
A primary goal for the Gini3 project was to achieve over one minute of spatial memory while also operating in real-time and at a higher resolution.
Gini3 achieves world consistency and spatial memory by generating frame-by-frame, deliberately avoiding explicit 3D representations like NeRFs or Gaussian splatting.
Jack from Google DeepMind believes that developing interactive world models like Gini3 is the fastest path to achieving embodied AGI agents that can operate in the real world.
Google DeepMind has an agent called Sima that can interact with environments generated by Gini3, demonstrating the model's composability.
Gini3 aims to bridge the 'sim-to-real' gap in robotics by combining a real-world, data-driven approach with the ability for agents to learn from experience in a simulated environment.
Google DeepMind's Gini3 model is perceived by non-experts as being able to generate video that looks real.
The Gini3 model is a significant advancement in video generation as it introduces an interactive element, unlike previous models which were limited to generating short, non-interactive video clips.
Google DeepMind's Genie 2 project focused on generating 3D environments but did not achieve the same quality as state-of-the-art video models like VO2.
The Gini3 project combined insights and technologies from separate internal Google DeepMind projects, including Genie 2, VO2, and Game & Gen.
The original motivation for the Genie project, started in 2022, was to solve the problem in reinforcement learning of needing unlimited, diverse environments for training agents.
The Genie 2 model had early signs of spatial memory, with the ability to remember objects for a few seconds, but this capability was not widely noticed at its release.