Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass

Prime Intellect released Prime Mega MoE, a training extension of DeepSeek's fused Mega MoE kernel that adds an overlapping backward pass, making the kernel usable for training rather than inference only. It reports 32% to 34% faster full training steps than the prime-rl baseline and 48% to 55% faster than TorchTitan on Qwen3-30B-A3B and Qwen3-235B-A22B, with a best case of 3.11x faster forward pass on a Qwen3-30B-A3B MoE layer. It ships in prime-rl as the mega_moe dispatch backend, with kernels in a prebuilt DeepGEMM wheel from prime-kernels.