Keep pulling the thread on Meta Ray-Ban.
Kyber's goal is to achieve 4 milliseconds of glass-to-glass latency for remote control applications.
Kyber has achieved a glass-to-glass latency of 7 milliseconds in tests running from Windows to Windows or Windows to Mac.
Kyber is a new software development kit (SDK) and platform designed for ultra-low latency, end-to-end control of remote machines like robots, drones, and computers.
When Waymo autonomous vehicles encounter a situation they cannot handle, a human operator takes remote control, an event estimated to occur 1% of the time.
Kyber uses a single QUIC protocol connection over UDP to transmit multiple synchronized streams, including audio, video, and control commands.
Kyber's platform includes a re-timestamping mechanism to account for clock drift between multiple sensors on a robot, ensuring data coherence for AI model training.
Devices like the Meta Ray-Ban glasses will require streaming from a GPU, either in the cloud or on a connected phone, due to limited on-device processing power.
Kyber is working to optimize video encoders to achieve a frame encoding time of 4 milliseconds.
To improve reliability over the internet, Kyber uses forward error correction to reconstruct lost packets without adding re-transmission latency.
Kyber demonstrated its technology at CES by remotely controlling a 3D-printed rover in Las Vegas from an operator located in France.
In Kyber's 7-millisecond latency stack, the NVIDIA hardware encoder contributes approximately 3.5 milliseconds and the Intel decoder contributes about 2 milliseconds.
Kyber is designed to be compatible with low-power compute hardware used in robotics, such as the NVIDIA Jetson Nano or an N150.