NVIDIA AI Introduces Fast-dLLM: A Training-Free Framework That Brings KV Caching and Parallel Decoding to Diffusion LLMs

Diffusion-based large language models (LLMs) offer the potential for faster, multi-token generation through bidirectional attention mechanisms but face practical challenges in achieving competitive inference speeds. Their lack of key-value caching and difficulties in maintaining generation quality during parallel decoding limit their real-world applicability compared to traditional autoregressive models.