“The Reversible Networks (RevNets) architecture enables significant memory savings during training by recomputing activations on-the-fly during the backward pass, thus avoiding the need to store them in HBM and trading higher compute for lower memory usage.”