Genesis: Evolved AVX-512 Kernels Accelerate LLM Inference

Genesis is a novel approach to accelerating NF4 LLM inference using evolved x86 AVX-512 kernels. It fuses weight dequantization with the dot product in a single pass, eliminating the need for an intermediate matrix and significantly reducing data transfer. This is particularly effective for MoE CPU offload on a single GPU, where it achieves substantial speedups compared to traditional methods. The kernels in Genesis were not written by hand but were discovered through a genetic evolution process. This process involved representing the kernel's inner loop as a sequence of x86 operations, applying random mutations, benchmarking the mutants on real hardware, and selecting the fastest variants.

Over 25 evolutionary runs, the system evaluated thousands of mutations, leading to kernels that outperform hand-tuned baselines by up to 19.25%. The evolved instruction orderings exploit Zen 4 microarchitectural properties, such as NOP alignment, early scale broadcast, reverse activation loading, and interleaved computation. These optimizations would be difficult to discover manually, highlighting the power of evolutionary algorithms for kernel optimization. Genesis offers a significant performance boost for local LLM inference, particularly for MoE models, enabling faster and more efficient processing on CPUs without relying on CUDA. This could democratize access to large language models by enabling efficient CPU-based inference on a wider range of hardware.