loopyt 7c6fdc7c6e fix(sm90): expand expert scales on the contraction axis only ... `requant_expert_fp4` expanded the UE8M0 scale with `_expand_blocks`, which repeats *both* axes. That is right for dense weights, whose scales are `(n_out/32, k/32)`, but expert scales are already per-output-channel: `(n_out, k/32)`. Expanding both axes therefore treated `(2304, 160)` as `(72, 5)` and handed every one of the 2304 output channels the scales of the first 72 rows, scrambling each expert's weights. This is what the layer comparison was pointing at: `shared_expert` matched the reference (cos 0.9987) because it goes through `requant_dense_fp8`, which uses `_expand_blocks` correctly, while `combined` -- routed experts plus shared -- was off by 3.6e-1. Verified against the authoritative implementation, `inference/convert.py`'s `cast_e2m1fn_to_e4m3fn`, on layer 0 / expert 370: w1 cos 0.8936 -> 1.0000 rel_err 5.35e-1 -> 2.84e-2 w3 cos 0.8960 -> 0.9997 rel_err 5.26e-1 -> 2.69e-2 w2 cos 0.8964 -> 0.9996 rel_err 5.20e-1 -> 2.65e-2 The residual ~2.7% is the per-32 -> per-128 requantization loss, not a defect; the reference is lossless because it folds the relative per-32 offset into the FP8 values. End to end on the 11-token chat prompt: before: [17607, 41175, 389, 666] "中国的首都是**" after: [17607, 2793, 3139, 666, 6127, 666, 320, 1] "中国的首都是**北京**。" ref: [17607, 2793, 3139, 6127, 320] "中国的首都是北京。" The answer is now correct and the first three tokens match the reference exactly. The remaining difference is markdown emphasis (`**`), so some numerical drift still moves the top-1 on the occasional position. Also adds two checks that would have caught this immediately: `scripts/h20_expert_weight_check.py` diffs our requantization against DeepSeek's own converter, and `scripts/h20_quant_check.py` pins the E2M1 table, nibble order and UE8M0 bias. The nibble order and table turn out to match the reference; only the scale broadcast was wrong.		2026-10-21 16:50:31 +08:00
..
_C	initial commit	2026-10-20 12:09:42 +08:00
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	initial commit	2026-10-20 12:09:42 +08:00
entrypoints	initial commit	2026-10-20 12:09:42 +08:00
h20_ops	fix(sm90): expand expert scales on the contraction axis only	2026-10-21 16:50:31 +08:00
hugging_face	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
model_executor	fix(sm90): expand expert scales on the contraction axis only	2026-10-21 16:50:31 +08:00
npu_ops	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
structure_output	initial commit	2026-10-20 12:09:42 +08:00
ve	initial commit	2026-10-20 12:09:42 +08:00
worker	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
attrdict.py	initial commit	2026-10-20 12:09:42 +08:00
config.py	initial commit	2026-10-20 12:09:42 +08:00
environ.py	initial commit	2026-10-20 12:09:42 +08:00
load_class.py	initial commit	2026-10-20 12:09:42 +08:00
logger.py	initial commit	2026-10-20 12:09:42 +08:00
model_config_loader.py	initial commit	2026-10-20 12:09:42 +08:00
multimodal_tokens.py	initial commit	2026-10-20 12:09:42 +08:00
platform.py	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
py.typed	initial commit	2026-10-20 12:09:42 +08:00
sampling_params.py	initial commit	2026-10-20 12:09:42 +08:00
sequence.py	initial commit	2026-10-20 12:09:42 +08:00
shared_memory.py	initial commit	2026-10-20 12:09:42 +08:00
testing.py	initial commit	2026-10-20 12:09:42 +08:00
timer.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	initial commit	2026-10-20 12:09:42 +08:00