whn09 c24574bfd6 Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25) ... - Layer-major KV / state layout so per-layer views are contiguous graph inputs (the executor copied non-contiguous inputs every call). - One compiled entry per (class, method), rank as a registered buffer, no layer_idx guard: every layer reuses the same graphs and loaded NEFFs. - fp8 weight rescale on the host (numpy LUT, ~2 min instead of 35+ min of eager per-shape compiles); engram sf packing on the host. - int16 ue8m0 packing in fp32 (int32 interleaved slices narrowed to int16 crash neuronx-cc, NCC_IBCG902). - Token embedding and the text model's vision tower stay in host memory on Neuron (2.5 GiB of HBM a rank). - offline example: --max_num_seqs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>		2026-10-23 01:39:22 +08:00
..
test_fp8_e4m3.py	Neuron: static decode graphs, static-shape MoE, trn2 e4m3 fp8 (issues #9, #12, #24)	2026-10-22 11:59:37 +08:00
test_fp8_legalize.py	Neuron: tiny-model decode region graphs compile and run on a NeuronCore (issues #9, #25)	2026-10-22 18:38:01 +08:00
test_nki_attention.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
test_nki_hc.py	Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25)	2026-10-23 01:39:22 +08:00