loopyt 39b086d781 feat(h20): route tile_kernels calls through a compat layer, so the model runs without TileLang ... The references were importable and numerically validated, but nothing pointed the model at them: the forward pass still called the compiled kernels, which raise ModuleNotFoundError on a host without TileLang. This adds the routing layer and swaps the call sites. `deepinfer/tk_compat/` exposes the *compiled* signatures -- out-buffers, in-place conventions, argument order -- because that is what the 25 call sites are written against, and adapts to the reference's shape internally. Selection is `DEEPINFER_KERNELS=auto|cuda|ref`; `auto` follows whether TileLang imports. The adapters are not pass-throughs. Each difference was found by running both backends: * `per_token_cast` takes `num_per_channels=n`, the reference takes `block_size=(1, n)`. * `apply_rotary` takes a concatenated cos|sin cache and rotates in place returning None; the reference takes cos and sin separately and returns a new tensor. The adapter splits, calls, and copies back, because the call site relies on the in-place contract. * `engram_gate` takes the pre-multiplied weight and `(eps, clamp_value)`; the reference needs the two weights separately and `(clamp_value, eps)`. Since a product cannot be un-multiplied, `fused_weight` returns the pair on the reference path and the gate unpacks it. * `mhc_pre_big_fuse_fwd` has no single reference: its five steps are composed from the reference's own components. * `mhc_reduce_partials_and_rmsnorm_fwd` likewise; rebuilt from the reference's formula. `moe_topk_gate(force_random=...)` raises rather than silently ignoring the flag. The reference has no equivalent, and dropping it would route tokens differently while looking healthy. Verification, all on the real call signatures: 18 cases compared between `cuda` and `ref` modes, `ALL_MATCH`, 12 bitwise identical including every quantised byte and every routing index; plus a smoke run that executes all 23 APIs with TileLang blocked, `ALL_CALLS_OK`. End to end still has not completed, and the reason is not this change: six GPUs on the host are held by other containers (82 GiB of 139 each), so the 8-way EP run OOMs in allocate_userbuffer and the surviving ranks wait at a barrier. Recorded in the doc along with the command to rerun it.		2026-10-22 18:10:14 +08:00
..
tilelang_ops	feat(h20): port the four MTP kernels, so speculative decoding no longer needs TileLang	2026-10-22 14:16:33 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
attention_backend.py	initial commit	2026-10-20 12:09:42 +08:00
layernorm.py	initial commit	2026-10-20 12:09:42 +08:00
linear.py	feat(h20): route tile_kernels calls through a compat layer, so the model runs without TileLang	2026-10-22 18:10:14 +08:00
linear.py.import-orig	feat(h20): route tile_kernels calls through a compat layer, so the model runs without TileLang	2026-10-22 18:10:14 +08:00
mtp_info_pool.py	initial commit	2026-10-20 12:09:42 +08:00
rotary_embedding.py	feat(h20): route tile_kernels calls through a compat layer, so the model runs without TileLang	2026-10-22 18:10:14 +08:00
rotary_embedding.py.import-orig	feat(h20): route tile_kernels calls through a compat layer, so the model runs without TileLang	2026-10-22 18:10:14 +08:00
sampler_utils.py	initial commit	2026-10-20 12:09:42 +08:00
sampler.py	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
vocab_parallel_embedding.py	initial commit	2026-10-20 12:09:42 +08:00