Support FlashMLA backend

[sleepcoo]

Motivation

Integrate flashmla for decoding, and the accuracy test is currently okay. The current implementation is quite simple, directly integrating flashmla as the backend. Later, we need to abstract a fastmla_backend, using fa3 for prefill and flashmla for decode

Modifications

• FlashMLABackend inherits from FlashInferMLAAttnBackend, using FlashInferMLAAttnBackend for prefill and FlashMLABackend for decoding.
• Add the create_flashmla_kv_indices_triton function to be compatible with the block table format of flashmla.

command

 python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3 --trust-remote --tp 8 --page-size 64 --disable-cuda-graph --enable-flashmla

Todo

• Support FlashMLA decode with cudagraph
• Enable speculative sampling in FlashMLA
• Add unit test
• Performance Analysis and Optimization
• Integrate FA3 prefill

[merrymercy]

automatically set this

[merrymercy]

this metadata is the same for all layers, so we can process them only once in init_forward_metadata

[sleepcoo]

I will fix the issue in the PR of CUDA graph.

[lishicheng1996]

Hi, I test --enable-flashmla on H20*16, and it is slower than normal version. Do you have data about E2E inference speed gain with flashmla? Thanks very much!

[sleepcoo]

Hi, I test --enable-flashmla on H20*16, and it become slower than normal version. Do you have data about E2E inference speed gain with flashmla? Thanks very much!

Wait for my cudagraph implementation