mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-10-28 23:07:09 +08:00
45e1e44444
Fixes found by reducing neuronx-cc NCC_IINL902 crashes to minimal reproducers (crash rates over repeated compiles, the compiler is nondeterministic on these graphs): - uint8 byte assembly of narrowed values (KV records, e2m1/e4m3 codes, ue8m0 exponents) crashed or silently corrupted bytes: producers keep int32 codes under compile and stores narrow once (fused_transform, kv_quant, store_kv_cache; neuron_compile.encode_codes). - int16 packed scales written through a uint8 view and read back in the same graph: the Neuron kernel now receives the int16 storage and packs byte pairs arithmetically (_alloc_quant, return_quant). - fp8 legalization pass treated plain float->bf16 conversions as fp8 carriers (wrong values + crash). - Projection stage split further (nullcontext instead of a disabled CP session). Device (tiny model, regions mode): every decode graph compiles; argmax vs golden short 9/9, medium 8/9 (step 6 near-tie, margin 0.065), long 9/9, cos 0.997-0.9995. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| compile | ||
| deep_ep | ||
| deep_gemm | ||
| device | ||
| flash_mla | ||
| sampling | ||
| tile_kernels | ||
| nie_skip.py | ||
| run_all.sh | ||
| setup_trn2.sh | ||