whn09 c24574bfd6 Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25) ... - Layer-major KV / state layout so per-layer views are contiguous graph inputs (the executor copied non-contiguous inputs every call). - One compiled entry per (class, method), rank as a registered buffer, no layer_idx guard: every layer reuses the same graphs and loaded NEFFs. - fp8 weight rescale on the host (numpy LUT, ~2 min instead of 35+ min of eager per-shape compiles); engram sf packing on the host. - int16 ue8m0 packing in fp32 (int32 interleaved slices narrowed to int16 crash neuronx-cc, NCC_IBCG902). - Token embedding and the text model's vision tower stay in host memory on Neuron (2.5 GiB of HBM a rank). - offline example: --max_num_seqs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>		2026-10-23 01:39:22 +08:00
..
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
base.py	Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25)	2026-10-23 01:39:22 +08:00
cache_engine.py	Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25)	2026-10-23 01:39:22 +08:00
cuda_graph_runner.py	initial commit	2026-10-20 12:09:42 +08:00
decode_perf_cache.py	initial commit	2026-10-20 12:09:42 +08:00
dflash_cuda_graph_runner.py	initial commit	2026-10-20 12:09:42 +08:00
model_runner.py	Neuron: static decode graphs, static-shape MoE, trn2 e4m3 fp8 (issues #9, #12, #24)	2026-10-22 11:59:37 +08:00
neuron_graph_runner.py	Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25)	2026-10-23 01:39:22 +08:00
runtime_config.py	initial commit	2026-10-20 12:09:42 +08:00
worker.py	initial commit	2026-10-20 12:09:42 +08:00
zos_status.py	initial commit	2026-10-20 12:09:42 +08:00