whn09 3c38fd5829 Neuron MoE: skip unrouted experts inside the fp4 kernels at decode sizes (DEEPINFER_NEURON_GATED_EXPERTS, default on) ... The fp4 expert GEMMs dequantize the whole weight to bf16 in HBM on every call (device profile: the MoE graph reads 756 MB a layer), and a rank computes all 6 of its experts while a 6-row step routes to about one. fp4_linear_2core_gated takes a per-expert flag (any row routed to it) and runs the dequant and GEMM in an nl.fori_loop bounded by it, so an unrouted expert costs a zero fill. Decode sizes only (<= 16 rows); prompt chunks route to every expert anyway. bench70 (vllm-sonnet 512, TP=64, nextn=5): TPOT 258.9 -> 248.4 ms, step 738 -> 711 ms, tokens per step identical. Tiny cp=2 nextn=5 on device: 9/9 golden tokens, drafts unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>		2026-10-26 06:36:48 +08:00
..
__init__.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
blockquant.py	Neuron MoE: skip unrouted experts inside the fp4 kernels at decode sizes (DEEPINFER_NEURON_GATED_EXPERTS, default on)	2026-10-26 06:36:48 +08:00
compress_pool.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
hc_attn_out.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
hc_collapse.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
hc_expand.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
hc_mixes.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
index_score.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
rmsnorm.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
sparse_attn.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
swiglu.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00