FVD: Fleming-Viot Resampling Boosts Diffusion Model Diversity and Speed

The performance of diffusion models, a cornerstone of modern generative AI, has been significantly constrained by issues such as 'diversity collapse' in Sequential Monte Carlo (SMC) based samplers. Existing resampling schemes, including multinomial resampling, often lead to a reduction in output diversity and lineage collapse under strong selection pressures, limiting the creative breadth and fidelity of generated content. This bottleneck has been a persistent challenge for researchers aiming to push the boundaries of image and media generation.

Fleming-Viot Diffusion (FVD) emerges as a critical inference-time alignment method designed to directly address this diversity collapse. Inspired by Fleming-Viot population dynamics, FVD innovatively replaces conventional multinomial resampling with a specialized birth-death mechanism. This mechanism is engineered to handle scenarios where rewards are only approximately available, integrating independent reward-based survival decisions with stochastic rebirth noise. This unique combination yields flexible population dynamics that effectively preserve broader trajectory support while efficiently exploring reward-tilted distributions, critically avoiding the need for costly value function approximation or extensive rollouts.

The empirical evidence strongly supports FVD's efficacy, demonstrating substantial gains across various settings. On the DrawBench benchmark, FVD outperforms prior methods by a notable 7% in ImageReward. Furthermore, on class-conditional tasks, it achieves an impressive 14-20% improvement in FID (Fréchet Inception Distance) over strong baselines. Crucially, FVD also boasts a significant speed advantage, being up to 66 times faster than value-based approaches, while maintaining full parallelizability and efficient scaling with inference compute. This breakthrough not only elevates the quality and diversity of generative AI outputs but also drastically improves the computational efficiency, paving the way for more powerful and accessible creative AI tools.