loopyt 2ea88a3431 test(sm90): rule out the KV cache and the decode indexer ... Two closed-loop checks, both clean, which finally retires the two standing suspects. P4.3 KV cache. The repo's own store/load tests pass for the fp32-scale index layout SM90 uses (6 each). Running every layout gives 48 passed / 12 failed, and every failure is a ue8m0 index layout -- `FP4_INDEX_KV_FORMAT` or `MXFP8_INDEX_KV_FORMAT`, neither of which SM90 selects. Those failures are in the test's reference, not the kernel: it reinterprets ue8m0 bytes as `PLATFORM.fp8_sf_type`, which is float32 here, so `index_copy_` hits a dtype mismatch. `scripts/h20_kv_roundtrip.py` then closes the loop without any reference at all -- write known BF16 through the paged cache, read it back, compare. Both SM90 layouts, block 32 and 64, 1/233/ 1029 tokens: V41_FORMAT (main/swa KV) rel 0.0266-0.0295 cos 0.9996 FP8_FLOAT32_SF_INDEX_KV_FORMAT (132B) rel 0.0266-0.0327 cos 0.9996 That is the fp8 floor, the same order as the 2.7e-2 on weights, so the cache is not where the decode error comes from and the 132-byte layout question from P3.9 is closed. One trap: the first version of the script compared raw fp8 values against BF16 and reported rel=126. Index layouts are deliberately *not* dequantized by `load_paged_kv_cache` -- the consumer is a kernel that wants bytes plus scales. Multiplying by the scale gives 0.027. The 127x was the test, not the implementation. P4.4 the decode indexer. `_decode_p_sum_sm90` reimplements DeepGEMM's paged MQA logits in torch because Hopper's kernel requires 64-token pages; nothing else exercised it. Compared against a float64 reference that shares no code -- explicit per-token offsets instead of strided views -- it agrees to rel~1e-7 / cos 1.0 across page sizes 32 and 64, boundary-crossing lengths, multi-page contexts and 32 heads. The `q_sf[..., :1]` dequantization is also sound: head_dim is 128, equal to the granularity, so one scale per (row, head) is correct. What this changes. `logits_prompt` from the max_tokens=1 run puts the prefill error at cos 0.9846 with an absolute error of 3.30 on the winning token, against a reference margin of 3.112: the margin is thin because the error is the same size as the margin, not by coincidence. The reference's own prefill-to-decode transition also costs about 2x margin (3.112 -> 1.638) purely from the quantized KV round trip. Our 0.256 halved would still be positive, yet it flips, so roughly 0.5-1.0 of logit-level error in decode remains unexplained -- now narrowed to attention over the cached main KV and the SWA-window / compressed-position mapping. Also corrects an earlier overstatement: "the difference is introduced by decode" does not mean decode has a discrete bug, and P4.3/P4.4 show its two main suspects do not. One more dump trap worth recording: with max_tokens > 1, `logits_prompt` is contaminated too. mysteps6 (max_tokens=7) gives cos 0.68 and a 16.5 top-1 error for the same input where mylogits3 (max_tokens=1) gives 0.9846 and 3.30. Prefill logits are only trustworthy from a max_tokens=1 run.		2026-10-21 20:02:19 +08:00
..
build_vendor.sh	initial commit	2026-10-20 12:09:42 +08:00
build.sh	initial commit	2026-10-20 12:09:42 +08:00
compare_layers.py	fix(sm90): expand expert scales on the contraction axis only	2026-10-21 16:50:31 +08:00
format_csrc.sh	initial commit	2026-10-20 12:09:42 +08:00
gen_stub.sh	initial commit	2026-10-20 12:09:42 +08:00
h20_ask.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_attn_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_engram_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_expert_weight_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_index_decode_check.py	test(sm90): rule out the KV cache and the decode indexer	2026-10-21 20:02:19 +08:00
h20_kv_roundtrip.py	test(sm90): rule out the KV cache and the decode indexer	2026-10-21 20:02:19 +08:00
h20_quant_check.py	test(sm90): decode FP4 by table lookup in the quant check	2026-10-21 17:15:16 +08:00
h20_real_requant_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_recipe_probe.py	debug: capture final logits, and keep the profile forward out of the dumps	2026-10-21 17:44:21 +08:00
h20_smoke_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_topk_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
ref_layer_dump.py	debug: capture final logits, and keep the profile forward out of the dumps	2026-10-21 17:44:21 +08:00
ref_next_token.py	debug: per-step decode dumps, and prove the flip is decode-only	2026-10-21 19:07:49 +08:00