loopyt dcf00aca7c feat(h20): enable FlagGems end to end, from an allow-list of ops proven equal to PyTorch ... `DEEPINFER_ENABLE_FLAGGEMS=1` turns it on. Verified on all 8 DP ranks: exit=0, 8/8 ranks agreeing, no kernel fallbacks, `hard failures: none`, correct answers. 19 ops actually dispatched, 152 call sites, 43 registrations from 37 requested names. The default is `flag_gems.only_enable(include=[...])`, not `enable()`, and that is the whole design: * allow-list -- every intermediate state is safe (unproven ops stay on PyTorch, which is correct by construction); an incomplete list costs throughput, not correctness. * deny-list -- I tried it first. Five full engine runs, one new defect each: `cat` (e8m0 dtype), `mul` (broadcast shape), `topk` (compile blowup), `rms_norm` (signature), `mm` (compile assert). The method requires being bitten before it can exclude, and every state along the way is unsafe. That matches the instruction this was built to: use what works, leave the rest. Where it is called is not incidental. `import flag_gems` initialises a CUDA context -- measured, `torch.cuda.is_initialized()` flips on the import alone, before `enable()`. A `sitecustomize` injector therefore put all 8 ranks on device 0 and NCCL refused with "Multiple Ranks are using the same GPU". It now runs inside `create_llm`, after `setup_cuda_visible_devices`, which is the same hazard already documented there for the profiler. Defects found and documented rather than worked around, because each one is worth knowing: * `mul` returns the *first operand's shape* when broadcasting expands it: (1,64,8)*(4,1,8) gives (1,64,8), and (64,1)*(1,128) gives (64,1). Silent, no error, fewer elements computed. It made `engram_hash_ref` disagree with PyTorch during a real run. add/sub/div/maximum/minimum all broadcast correctly, so it is mul's own bug. * `cumsum` drifts by up to 1.0 over a 512-element bf16 row (tolerance ~0.17) -- bf16 accumulation. * `topk`'s compile time explodes with k: 4.8 s at k=8, 19 s at k=64, >180 s at k=512, and the sampler asks for 4096. py-spy put all 8 ranks at 100% CPU inside `make_llir` with every GPU idle. * `cat` raises `KeyError: 'float8_e8m0fnu'` on the model's UE8M0 scale tensors during weight load. * `mm` passes every isolated probe yet fails inside the engine at `general_mm`; trigger unknown, so GEMMs stay on cuBLAS, which is faster anyway. Two methodological errors in my own measurements, both fixed and both recorded: 1. After `enable()`, `torch.mul` *is* `flag_gems.mul`, so the first version of the mul A/B compared FlagGems with itself and reported a perfect score. All references are now computed before enable. 2. "ok" did not mean FlagGems ran. `torch.bitwise_xor` has no FlagGems registration at all, so that "match" was PyTorch against PyTorch. `screen_ops.py` now checks registration first and reports such cases as `skip` rather than pass. Name mismatch is its own trap: FlagGems registers `add` bare but `eq.Scalar`, `where.self`, `repeat_interleave.self_int`, `_softmax` (not `softmax`), `div.true_divide`. Asking for `softmax` silently registers nothing. The bootstrap now expands names against `flag_gems._FULL_CONFIG` and warns on anything unresolved -- which is how `softmax` was found to need `_softmax`. Output is not token-identical to the no-FlagGems build (diverges at the first token), but two runs of the same FlagGems config are bit-identical, so the difference is Triton-versus-eager accumulation order, not nondeterminism. Answers stay correct and coherent.		2026-10-26 12:53:47 +08:00
..
flaggems_port	feat(h20): enable FlagGems end to end, from an allow-list of ops proven equal to PyTorch	2026-10-26 12:53:47 +08:00
simulate_no_tilelang	feat(h20): make the pure-torch kernel references usable without TileLang	2026-10-22 16:08:46 +08:00
sparse_mla_port	docs: correct what importing flag_gems actually does	2026-10-25 14:37:59 +08:00
tile_kernels_patch	fix(h20): let the reference backend sync, and stop the reference raising under the guard	2026-10-24 19:10:49 +08:00
ab_cases.py	test(h20): catch a factory that returns nothing before eight ranks do	2026-10-22 14:16:41 +08:00
build_vendor.sh	initial commit	2026-10-20 12:09:42 +08:00
build.sh	initial commit	2026-10-20 12:09:42 +08:00
compare_layers.py	fix(sm90): expand expert scales on the contraction axis only	2026-10-21 16:50:31 +08:00
copy_site_packages.py	feat(h20): make the FlagTree image setup reproducible instead of a shell history	2026-10-22 13:20:03 +08:00
enable_torch_kernel_refs.sh	feat(h20): make the pure-torch kernel references usable without TileLang	2026-10-22 16:08:46 +08:00
flagtree_triton_probe.py	docs(h20): record what a FlagTree/FlagOS image can and cannot do	2026-10-22 10:53:47 +08:00
format_csrc.sh	initial commit	2026-10-20 12:09:42 +08:00
gen_stub.sh	initial commit	2026-10-20 12:09:42 +08:00
h20_ask.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_attn_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_e2e_report.sh	docs(h20): point at the setup script, and let the report script carry its own defaults	2026-10-22 13:20:08 +08:00
h20_engram_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_expert_weight_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_index_decode_check.py	test(sm90): rule out the KV cache and the decode indexer	2026-10-21 20:02:19 +08:00
h20_kv_roundtrip.py	test(sm90): rule out the KV cache and the decode indexer	2026-10-21 20:02:19 +08:00
h20_ops_ab.py	test(h20): catch a factory that returns nothing before eight ranks do	2026-10-22 14:16:41 +08:00
h20_philox_check.py	feat(triton): reproduce cuRAND's Philox stream, so the sampler can be ported	2026-10-22 08:37:08 +08:00
h20_quant_check.py	test(sm90): decode FP4 by table lookup in the quant check	2026-10-21 17:15:16 +08:00
h20_real_requant_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_recipe_probe.py	debug: capture final logits, and keep the profile forward out of the dumps	2026-10-21 17:44:21 +08:00
h20_rng_dump.py	feat(triton): reproduce cuRAND's Philox stream, so the sampler can be ported	2026-10-22 08:37:08 +08:00
h20_smoke_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
h20_state_check.py	test(h20): check what the model actually says, in a file rather than on the command line	2026-10-22 10:53:01 +08:00
h20_topk_check.py	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
ref_layer_dump.py	debug: capture final logits, and keep the profile forward out of the dumps	2026-10-21 17:44:21 +08:00
ref_next_token.py	debug: per-step decode dumps, and prove the flip is decode-only	2026-10-21 19:07:49 +08:00
setup_flagtree_image.sh	feat(h20): make the FlagTree image setup reproducible instead of a shell history	2026-10-22 13:20:03 +08:00
verify_tile_kernels_patch.py	feat(h20): make the pure-torch kernel references usable without TileLang	2026-10-22 16:08:46 +08:00