[Feature]: Factor out zero_expert_num from FusedMoE #28152
Description
🚀 The feature, motivation and pitch
We have many special cases in FusedMoE for zero_expert_num
This parameter is used exclusively for LongCatFlash. We should factor this out of FusedMoe and put the complexity into the model file.
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.