Moonshot AI ships FlashKDA: FlashKDA: high-performance Kimi Delta Attention kernels

Executive Summary

Continue your research

Keep pulling the thread on Yang Zhilin.

6 quotes

Requires modern, high-end GPUs (SM90+), limiting accessibility for users with older hardware.
Specific implementation constraints, such as fixed key/value dimensions of 128, may limit flexibility for some use cases.

Significant performance improvement for models using Kimi Delta Attention by leveraging a highly optimized CUTLASS-based kernel.
Provides a more efficient inference backend for the popular `flash-linear-attention` library, reducing latency and computational costs.

The correctness tests for FlashKDA verify an exact match against a PyTorch reference implementation.

The `flash_kda.fwd` kernel API currently requires the key (K) and value (V) dimensions to both be 128.

Moonshot AI has released FlashKDA, a set of high-performance Kimi Delta Attention kernels built on the CUTLASS library.

FlashKDA requires a GPU with SM90 architecture or higher.

Once installed, FlashKDA is automatically dispatched from the `chunk_kda` function within the `flash-linear-attention` library.

Using FlashKDA as a backend requires `flash-linear-attention` version 0.5.0.

Processed Jul 28, 2026Daily intelligence brief → yt-dlp + mlx-whisper + Gemini