Yes, absolutely in deep learning. Custom fused CUDA kernels everywhere.
Yep. MoE, FlashAttention, or sparse retrieval architectures for example.
alt Hacker News
Yep. MoE, FlashAttention, or sparse retrieval architectures for example.