loopyt 8ae7365cbc feat(h20): port the four MTP kernels, so speculative decoding no longer needs TileLang ... `DEEPINFER_USE_NEXTN_DECODE` defaults to 5, so speculative decoding is on unless someone turns it off, and these four ran on every decode step. On a host without TileLang the first step failed; on the H20 they were the only kernels still answering from the other backend during a normal run. The interesting one is `remap_draft_topk`. Its TileLang form builds a vocabulary-windowed lookup table in shared memory and reads the logprobs back out of it. Triton has no equivalent primitive, so the table is gone and each target index is matched against the draft indices directly: draft indices are unique per token id, so at most one candidate can match and a max over the block is well defined. That is O(topk^2) instead of O(vocab/pool_size * topk), which would be the wrong trade at a large `force_topk` -- but `force_topk` defaults to 0, so this kernel is off the default path and correctness comes first. Worth revisiting if a deployment turns it on. The other three adapt rather than change shape: * `get_accept_result`'s prefix scan becomes a bounded loop over the widest span in the block; lanes with a shorter span hold their value on the extra iterations. MTP packs at most num_steps+1 rows per request, so the bound stays small. * `finalize_verification` reduces `batch_confidence_error` within the program and issues one atomic instead of one per row, because a scalar pointer with a vector value is not valid in this Triton version. Accumulating in a different order changes the last bits, so its cases compare with a tolerance and say why. * `get_fused_dflash_get_input` rounds its column loop up. `padded_kv_len` is a multiple of 64 but not necessarily of the tile width, and flooring leaves the tail of every row unwritten. 33 new A/B cases across the four, covering each constexpr variant the engine builds. The full run is green and an end-to-end pass is token-for-token identical to the run before the change, with the MTP fallback notices gone.		2026-10-22 14:16:33 +08:00
..
asc	initial commit	2026-10-20 12:09:42 +08:00
cuda	initial commit	2026-10-20 12:09:42 +08:00
triton	feat(h20): port the four MTP kernels, so speculative decoding no longer needs TileLang	2026-10-22 14:16:33 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
cache_common.py	feat(ops): let the package import on a host that has only Triton	2026-10-22 10:53:30 +08:00
candidate_blocks.py	initial commit	2026-10-20 12:09:42 +08:00
common.py	initial commit	2026-10-20 12:09:42 +08:00
compress.py	feat(ops): let the package import on a host that has only Triton	2026-10-22 10:53:30 +08:00
compute_seqlens_k_zos.py	initial commit	2026-10-20 12:09:42 +08:00
conftest.py	initial commit	2026-10-20 12:09:42 +08:00
dflash_copy.py	initial commit	2026-10-20 12:09:42 +08:00
dflash.py	initial commit	2026-10-20 12:09:42 +08:00
flash_mla_utils.py	initial commit	2026-10-20 12:09:42 +08:00
fused_sample.py	feat(ops): report fallbacks at use time, and record why the sampler stays put	2026-10-22 08:05:05 +08:00
fused_transform.py	feat(ops): let the package import on a host that has only Triton	2026-10-22 10:53:30 +08:00
gather_engram_tokens_zos.py	initial commit	2026-10-20 12:09:42 +08:00
gather_input_ids_zos.py	initial commit	2026-10-20 12:09:42 +08:00
gather.py	initial commit	2026-10-20 12:09:42 +08:00
get_kv_index.py	initial commit	2026-10-20 12:09:42 +08:00
image_embed_ops.py	initial commit	2026-10-20 12:09:42 +08:00
kv_index_cat.py	initial commit	2026-10-20 12:09:42 +08:00
load_paged_kv_cache.py	feat(ops): let the package import on a host that has only Triton	2026-10-22 10:53:30 +08:00
norm.py	feat(ops): let the package import on a host that has only Triton	2026-10-22 10:53:30 +08:00
ops.py	feat(ops): let the package import on a host that has only Triton	2026-10-22 10:53:30 +08:00
outlier_detection.py	initial commit	2026-10-20 12:09:42 +08:00
prepare_cp_mirror.py	initial commit	2026-10-20 12:09:42 +08:00
prepare_v4_decode_metadata.py	initial commit	2026-10-20 12:09:42 +08:00
random_sampling.py	initial commit	2026-10-20 12:09:42 +08:00
sampler.py	initial commit	2026-10-20 12:09:42 +08:00
schedule_mtp_varlen.py	initial commit	2026-10-20 12:09:42 +08:00
state_cache.py	initial commit	2026-10-20 12:09:42 +08:00
store_kv_cache.py	initial commit	2026-10-20 12:09:42 +08:00
store_mtp_info.py	initial commit	2026-10-20 12:09:42 +08:00
structure_output.py	initial commit	2026-10-20 12:09:42 +08:00
sum_smaller_probs.py	initial commit	2026-10-20 12:09:42 +08:00
swa_index.py	initial commit	2026-10-20 12:09:42 +08:00