whn09 02e099fff8 Neuron: static decode graphs, static-shape MoE, trn2 e4m3 fp8 (issues #9, #12, #24) ... - NeuronGraphRunner: the CudaGraphRunner contract over torch.compile(dynamic=False), one graph per batch size (DEEPINFER_ENFORCE_EAGER=0, backend DEEPINFER_NEURON_COMPILE_BACKEND). Layer cache views are re-sliced from their root allocations inside the graph, so AOT never regenerates aliased mutated inputs with as_strided. - Decode MoE with fixed shapes: packed all-gather, dense local experts, fp32 reduce-scatter (neuron_ops/static_moe.py, deep_gemm.m_grouped_fp8_fp4_gemm_nt_shared_a). - Graph-break and compile fixes: hide torch.cuda during compile, torch.device equality for FX codegen, hash-based force_random routing, Engram hash/fetch on the host (exact int64), env vars resolved before tracing, Movable annotation, no .item()/nonzero/as_strided writes on the decode path, KV stores route padding slots to a scratch block kept past the pool, native-dtype gather under compile. - fp8: trn2 reads float8_e4m3fn as IEEE e4m3 (max 240). Activations/KV quantize to 224 (DEEPINFER_NEURON_FP8_MAX), checkpoint fp8 weights are rescaled at load (bytes / 2, ue8m0 + 1), and neuronx-cc gets --experimental-unsafe-fp8e4m3fn-as-fp8e4m3 via --internal-hlo2tensorizer-options. The uint8-carrier backend (neuron_ops/neuron_compile.py, "deepinfer_neuron") stays as a fallback. - Tests: tests/neuron/compile; run_all.sh runs kernel unit tests at the e4m3fn range (448) and the end-to-end stages at 224. Docs: vllm-neuron PR #71 analysis and compile notes. Host: tiny model 9/9 vs golden (all prompts) eager, compiled path (aot_eager) matches eager; CPU suite green. Device: the whole decode graph still crashes neuronx-cc (NCC_IINL902, being bisected). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>		2026-10-22 11:59:37 +08:00
..
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
attn.py	initial commit	2026-10-20 12:09:42 +08:00
builder.py	initial commit	2026-10-20 12:09:42 +08:00
compress.py	initial commit	2026-10-20 12:09:42 +08:00
containers.py	initial commit	2026-10-20 12:09:42 +08:00
cross_layer.py	initial commit	2026-10-20 12:09:42 +08:00
engram_metadata.py	initial commit	2026-10-20 12:09:42 +08:00
engram_parameters.py	initial commit	2026-10-20 12:09:42 +08:00
engram.py	Neuron: full DeepSeek-V4.1-Flash loads and prefills on 64 NeuronCores (EP=64)	2026-10-21 14:57:51 +08:00
hc_blocks.py	initial commit	2026-10-20 12:09:42 +08:00
hc_lazy.py	initial commit	2026-10-20 12:09:42 +08:00
hc.py	initial commit	2026-10-20 12:09:42 +08:00
heads.py	initial commit	2026-10-20 12:09:42 +08:00
index_attn.py	initial commit	2026-10-20 12:09:42 +08:00
linear.py	initial commit	2026-10-20 12:09:42 +08:00
mhc_modeling.py	initial commit	2026-10-20 12:09:42 +08:00
mhc_ops.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
mla_attn.py	initial commit	2026-10-20 12:09:42 +08:00
mlp.py	Neuron: static decode graphs, static-shape MoE, trn2 e4m3 fp8 (issues #9, #12, #24)	2026-10-22 11:59:37 +08:00
moe.py	initial commit	2026-10-20 12:09:42 +08:00
norm.py	initial commit	2026-10-20 12:09:42 +08:00
rope.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	Neuron: tiny model on a NeuronCore matches the reference (issues #1, #2, #4)	2026-10-21 13:22:59 +08:00
yoco.py	initial commit	2026-10-20 12:09:42 +08:00