Does a Better Source Distribution Really Help Flow Matching?
Some thinking on the source distribution of flow matching.
Exploring the principles behind learning and intelligence, and sharing what I learn along the way.
Some thinking on the source distribution of flow matching.
Lessons from trying to diffuse compressed language representations.
A personal recap of ICML 2026 diffusion language model papers, with trends, caveats, and a selective reading list.
A note on self-conditioning.
A note on why continuous diffusion language models may express global consistency more naturally than discrete masked diffusion.
A note on diffusion language models as a signal-design problem: if diffusion is a refinement process, what representation should it refine?
A note on why masked diffusion models pay a harder training objective in exchange for inference-time flexibility, and why token ordering matters most when the best decoding order is instance-dependent.
A hardware-efficiency perspective on why diffusion models can sometimes generate faster than autoregressive models despite worse theoretical complexity.
A note on policy improvement for LLM post-training, from policy ratios and advantages to PPO clipping, GRPO group baselines, and DPO preference optimization.
Offline RL is about reliable policy improvement under dataset support constraints.
Some notes on attention mechanisms.