“A drawback of pipeline parallelism is that it requires using micro-batches, which prevents the full amortization of weight-loading costs across a large global batch.”