loopyt c6a65fe6ad feat(h20): FlagGems' Triton sparse MLA matches the TileLang kernel at V4.1's shape ... The remaining blocker for a TileLang-free path was sparse MLA: `h20_ops/sparse_mla_sm90.py` is a 231-line TileLang program with no Triton counterpart in this repo. FlagGems already has one, and it turns out to be drop-in. `flagos-ai/FlagGems` -> `src/flag_gems/fused/flashmla_sparse.py` -> `flash_mla_sparse_fwd`. Pure Triton. At V4.1's real dims (h_q=64, h_kv=1, d_qk=512, topk=512) it compiles and runs on H20, and on 7 shapes it agrees with the TileLang kernel within 2 bf16 ULP -- including `topk_length` and `attn_sink`, the two optional arguments. The pairing is closer than expected, which is why it needed no adaptation: prefill_attn_varlen_sparse(query, kv, index_topk, softmax_scale, attn_sink, index_length) flash_mla_sparse_fwd(q, kv, indices, sm_scale, attn_sink, topk_length) Causality matches too, and for the same reason. `attn_sm90.py` passes `is_causal=False` because the indexer only scores positions before each query, so the indices already carry causality -- bounding on the row index instead collapses the layer to NaN, which already happened once and is documented at that call site. The FlagGems kernel has no causal parameter either. Recorded as *not* the answer: `DSA/sparse_mla.py` in the same repo. It hardcodes causal and needs the V3.2 576-wide split layout; V4.1 folds RoPE inside a single 512-wide latent, so TD = 0 and it fails to compile on `tl.arange(0, 0)`. Worth writing down because the name looks like the better match. What remains, all much smaller than writing a kernel: wire it in behind the existing `patch_imports` hook; test the decode path separately (it gathers to bf16 with identity indices, a different call shape than the 7 cases); and convert the return, since the kernel gives (out, max_logits, lse) with lse in natural log where DeepInfer expects (out, lse) with lse in log2 -- a 1.4427x scale error if dropped.		2026-10-25 08:52:03 +08:00
..
simulate_no_tilelang	feat(h20): make the pure-torch kernel references usable without TileLang	2026-10-22 16:08:46 +08:00
sparse_mla_port	feat(h20): FlagGems' Triton sparse MLA matches the TileLang kernel at V4.1's shape	2026-10-25 08:52:03 +08:00
tile_kernels_patch	fix(h20): let the reference backend sync, and stop the reference raising under the guard	2026-10-24 19:10:49 +08:00
ab_cases.py	test(h20): catch a factory that returns nothing before eight ranks do	2026-10-22 14:16:41 +08:00
build_vendor.sh	initial commit	2026-10-20 12:09:42 +08:00
build.sh	initial commit	2026-10-20 12:09:42 +08:00
compare_layers.py	fix(sm90): expand expert scales on the contraction axis only	2026-10-21 16:50:31 +08:00
copy_site_packages.py	feat(h20): make the FlagTree image setup reproducible instead of a shell history	2026-10-22 13:20:03 +08:00
enable_torch_kernel_refs.sh	feat(h20): make the pure-torch kernel references usable without TileLang	2026-10-22 16:08:46 +08:00
flagtree_triton_probe.py	docs(h20): record what a FlagTree/FlagOS image can and cannot do	2026-10-22 10:53:47 +08:00
format_csrc.sh	initial commit	2026-10-20 12:09:42 +08:00
gen_stub.sh	initial commit	2026-10-20 12:09:42 +08:00
h20_ask.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_attn_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_e2e_report.sh	docs(h20): point at the setup script, and let the report script carry its own defaults	2026-10-22 13:20:08 +08:00
h20_engram_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_expert_weight_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_index_decode_check.py	test(sm90): rule out the KV cache and the decode indexer	2026-10-21 20:02:19 +08:00
h20_kv_roundtrip.py	test(sm90): rule out the KV cache and the decode indexer	2026-10-21 20:02:19 +08:00
h20_ops_ab.py	test(h20): catch a factory that returns nothing before eight ranks do	2026-10-22 14:16:41 +08:00
h20_philox_check.py	feat(triton): reproduce cuRAND's Philox stream, so the sampler can be ported	2026-10-22 08:37:08 +08:00
h20_quant_check.py	test(sm90): decode FP4 by table lookup in the quant check	2026-10-21 17:15:16 +08:00
h20_real_requant_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_recipe_probe.py	debug: capture final logits, and keep the profile forward out of the dumps	2026-10-21 17:44:21 +08:00
h20_rng_dump.py	feat(triton): reproduce cuRAND's Philox stream, so the sampler can be ported	2026-10-22 08:37:08 +08:00
h20_smoke_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_state_check.py	test(h20): check what the model actually says, in a file rather than on the command line	2026-10-22 10:53:01 +08:00
h20_topk_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
ref_layer_dump.py	debug: capture final logits, and keep the profile forward out of the dumps	2026-10-21 17:44:21 +08:00
ref_next_token.py	debug: per-step decode dumps, and prove the flip is decode-only	2026-10-21 19:07:49 +08:00
setup_flagtree_image.sh	feat(h20): make the FlagTree image setup reproducible instead of a shell history	2026-10-22 13:20:03 +08:00
verify_tile_kernels_patch.py	feat(h20): make the pure-torch kernel references usable without TileLang	2026-10-22 16:08:46 +08:00