Keep pulling the thread on Liang Wenfeng.
Most kernels in the TileKernels library approach the hardware performance limit for compute intensity and memory bandwidth.
TileKernels requires an NVIDIA GPU with SM90 or SM100 architecture.
DeepSeek has released TileKernels, a library of optimized GPU kernels for LLM operations built with TileLang.
TileLang is a domain-specific language designed for expressing high-performance GPU kernels in Python.
Some kernels from the TileKernels library are already used in DeepSeek's internal training and inference scenarios.
The current version of TileKernels does not represent best practices, and DeepSeek is actively working to improve its code quality and documentation.
TileKernels includes kernels for Top-k expert selection and scoring for Mixture of Experts (MoE) routing.
TileKernels provides kernels for Mixture of Experts routing that handle token-to-expert mapping, fused expansion/reduction, and weight normalization.
TileKernels supports per-token, per-block, and per-channel quantization to FP8, FP4, and E5M6 formats.
TileKernels provides Engram gating kernels which include fused RMSNorm, support for forward/backward passes, and weight gradient reduction.
TileKernels includes Manifold HyperConnection kernels that perform Sinkhorn normalization and mix splitting/application.
TileKernels offers high-level PyTorch autograd function wrappers to compose its low-level kernels into trainable layers.