Blog
Technical articles by Haoran Zhang on machine learning systems, linear attention, and efficient deep learning.
-
Linear Attention, Visualized
An interactive guide to linear attention, deriving recurrent and chunkwise forms for RetNet, Mamba-2, DeltaNet, GDN, GDP, GLA, and KDA.