loopyt 8ae7365cbc feat(h20): port the four MTP kernels, so speculative decoding no longer needs TileLang ... `DEEPINFER_USE_NEXTN_DECODE` defaults to 5, so speculative decoding is on unless someone turns it off, and these four ran on every decode step. On a host without TileLang the first step failed; on the H20 they were the only kernels still answering from the other backend during a normal run. The interesting one is `remap_draft_topk`. Its TileLang form builds a vocabulary-windowed lookup table in shared memory and reads the logprobs back out of it. Triton has no equivalent primitive, so the table is gone and each target index is matched against the draft indices directly: draft indices are unique per token id, so at most one candidate can match and a max over the block is well defined. That is O(topk^2) instead of O(vocab/pool_size * topk), which would be the wrong trade at a large `force_topk` -- but `force_topk` defaults to 0, so this kernel is off the default path and correctness comes first. Worth revisiting if a deployment turns it on. The other three adapt rather than change shape: * `get_accept_result`'s prefix scan becomes a bounded loop over the widest span in the block; lanes with a shorter span hold their value on the extra iterations. MTP packs at most num_steps+1 rows per request, so the bound stays small. * `finalize_verification` reduces `batch_confidence_error` within the program and issues one atomic instead of one per row, because a scalar pointer with a vector value is not valid in this Triton version. Accumulating in a different order changes the last bits, so its cases compare with a tolerance and say why. * `get_fused_dflash_get_input` rounds its column loop up. `padded_kv_len` is a multiple of 64 but not necessarily of the tile width, and flooring leaves the tail of every row unwritten. 33 new A/B cases across the four, covering each constexpr variant the engine builds. The full run is green and an end-to-end pass is token-for-token identical to the run before the change, with the MTP fallback notices gone.		2026-10-22 14:16:33 +08:00
..
_C	initial commit	2026-10-20 12:09:42 +08:00
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	initial commit	2026-10-20 12:09:42 +08:00
entrypoints	initial commit	2026-10-20 12:09:42 +08:00
h20_ops	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
hugging_face	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
model_executor	feat(h20): port the four MTP kernels, so speculative decoding no longer needs TileLang	2026-10-22 14:16:33 +08:00
npu_ops	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
structure_output	initial commit	2026-10-20 12:09:42 +08:00
ve	initial commit	2026-10-20 12:09:42 +08:00
worker	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
attrdict.py	initial commit	2026-10-20 12:09:42 +08:00
config.py	initial commit	2026-10-20 12:09:42 +08:00
debug_dump.py	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
environ.py	initial commit	2026-10-20 12:09:42 +08:00
load_class.py	initial commit	2026-10-20 12:09:42 +08:00
logger.py	initial commit	2026-10-20 12:09:42 +08:00
model_config_loader.py	initial commit	2026-10-20 12:09:42 +08:00
multimodal_tokens.py	initial commit	2026-10-20 12:09:42 +08:00
platform.py	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
py.typed	initial commit	2026-10-20 12:09:42 +08:00
sampling_params.py	initial commit	2026-10-20 12:09:42 +08:00
sequence.py	initial commit	2026-10-20 12:09:42 +08:00
shared_memory.py	initial commit	2026-10-20 12:09:42 +08:00
testing.py	initial commit	2026-10-20 12:09:42 +08:00
timer.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	initial commit	2026-10-20 12:09:42 +08:00