“The minimum possible latency for an inference step on a given hardware configuration is limited by the time it takes to read all of the model's total parameters from memory into the chips.”