loopyt dcf00aca7c feat(h20): enable FlagGems end to end, from an allow-list of ops proven equal to PyTorch ... `DEEPINFER_ENABLE_FLAGGEMS=1` turns it on. Verified on all 8 DP ranks: exit=0, 8/8 ranks agreeing, no kernel fallbacks, `hard failures: none`, correct answers. 19 ops actually dispatched, 152 call sites, 43 registrations from 37 requested names. The default is `flag_gems.only_enable(include=[...])`, not `enable()`, and that is the whole design: * allow-list -- every intermediate state is safe (unproven ops stay on PyTorch, which is correct by construction); an incomplete list costs throughput, not correctness. * deny-list -- I tried it first. Five full engine runs, one new defect each: `cat` (e8m0 dtype), `mul` (broadcast shape), `topk` (compile blowup), `rms_norm` (signature), `mm` (compile assert). The method requires being bitten before it can exclude, and every state along the way is unsafe. That matches the instruction this was built to: use what works, leave the rest. Where it is called is not incidental. `import flag_gems` initialises a CUDA context -- measured, `torch.cuda.is_initialized()` flips on the import alone, before `enable()`. A `sitecustomize` injector therefore put all 8 ranks on device 0 and NCCL refused with "Multiple Ranks are using the same GPU". It now runs inside `create_llm`, after `setup_cuda_visible_devices`, which is the same hazard already documented there for the profiler. Defects found and documented rather than worked around, because each one is worth knowing: * `mul` returns the *first operand's shape* when broadcasting expands it: (1,64,8)*(4,1,8) gives (1,64,8), and (64,1)*(1,128) gives (64,1). Silent, no error, fewer elements computed. It made `engram_hash_ref` disagree with PyTorch during a real run. add/sub/div/maximum/minimum all broadcast correctly, so it is mul's own bug. * `cumsum` drifts by up to 1.0 over a 512-element bf16 row (tolerance ~0.17) -- bf16 accumulation. * `topk`'s compile time explodes with k: 4.8 s at k=8, 19 s at k=64, >180 s at k=512, and the sampler asks for 4096. py-spy put all 8 ranks at 100% CPU inside `make_llir` with every GPU idle. * `cat` raises `KeyError: 'float8_e8m0fnu'` on the model's UE8M0 scale tensors during weight load. * `mm` passes every isolated probe yet fails inside the engine at `general_mm`; trigger unknown, so GEMMs stay on cuBLAS, which is faster anyway. Two methodological errors in my own measurements, both fixed and both recorded: 1. After `enable()`, `torch.mul` *is* `flag_gems.mul`, so the first version of the mul A/B compared FlagGems with itself and reported a perfect score. All references are now computed before enable. 2. "ok" did not mean FlagGems ran. `torch.bitwise_xor` has no FlagGems registration at all, so that "match" was PyTorch against PyTorch. `screen_ops.py` now checks registration first and reports such cases as `skip` rather than pass. Name mismatch is its own trap: FlagGems registers `add` bare but `eq.Scalar`, `where.self`, `repeat_interleave.self_int`, `_softmax` (not `softmax`), `div.true_divide`. Asking for `softmax` silently registers nothing. The bootstrap now expands names against `flag_gems._FULL_CONFIG` and warns on anything unresolved -- which is how `softmax` was found to need `_softmax`. Output is not token-identical to the no-FlagGems build (diverges at the first token), but two runs of the same FlagGems config are bit-identical, so the difference is Triton-versus-eager accumulation order, not nondeterminism. Answers stay correct and coherent.		2026-10-26 12:53:47 +08:00
..
_C	initial commit	2026-10-20 12:09:42 +08:00
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	initial commit	2026-10-20 12:09:42 +08:00
entrypoints	initial commit	2026-10-20 12:09:42 +08:00
h20_ops	docs: correct what importing flag_gems actually does	2026-10-25 14:37:59 +08:00
hugging_face	feat(h20): route tile_kernels calls through a compat layer, so the model runs without TileLang	2026-10-22 18:10:14 +08:00
model_executor	feat(h20): route tile_kernels calls through a compat layer, so the model runs without TileLang	2026-10-22 18:10:14 +08:00
npu_ops	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
structure_output	initial commit	2026-10-20 12:09:42 +08:00
tk_compat	fix(h20): let the reference backend sync, and stop the reference raising under the guard	2026-10-24 19:10:49 +08:00
ve	initial commit	2026-10-20 12:09:42 +08:00
worker	feat(h20): route tile_kernels calls through a compat layer, so the model runs without TileLang	2026-10-22 18:10:14 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
attrdict.py	initial commit	2026-10-20 12:09:42 +08:00
config.py	initial commit	2026-10-20 12:09:42 +08:00
debug_dump.py	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
environ.py	initial commit	2026-10-20 12:09:42 +08:00
flaggems_bootstrap.py	feat(h20): enable FlagGems end to end, from an allow-list of ops proven equal to PyTorch	2026-10-26 12:53:47 +08:00
load_class.py	initial commit	2026-10-20 12:09:42 +08:00
logger.py	initial commit	2026-10-20 12:09:42 +08:00
model_config_loader.py	initial commit	2026-10-20 12:09:42 +08:00
multimodal_tokens.py	initial commit	2026-10-20 12:09:42 +08:00
platform.py	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
py.typed	initial commit	2026-10-20 12:09:42 +08:00
sampling_params.py	initial commit	2026-10-20 12:09:42 +08:00
sequence.py	initial commit	2026-10-20 12:09:42 +08:00
shared_memory.py	initial commit	2026-10-20 12:09:42 +08:00
testing.py	initial commit	2026-10-20 12:09:42 +08:00
timer.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	initial commit	2026-10-20 12:09:42 +08:00