More complex methods for KV-cache compression, such as vector quantization and Hessian-aware quan..., Sonic AI
“More complex methods for KV-cache compression, such as vector quantization and Hessian-aware quantization, provide only marginal additional gains over SAW-INT4 once serving compatibility is considered.”