loopyt df9559e965 fix(triton): the grid for prepare_verification_metadata, and first matching end-to-end run ... The illegal memory access was not in `expand_sampling_params` at all, though that is where `CUDA_LAUNCH_BLOCKING` pointed. It was the metadata kernel leaving its output unwritten. `prepare_verification_metadata_tl` sized its second grid dimension from `num_steps`, i.e. `max_MTP_value`. With speculative decoding off that value is 0, and a zero-sized grid dimension launches nothing, so `row_to_req` -- a `torch.empty` buffer -- kept whatever the allocator handed out. `expand_sampling_params` then used those values as request indices, and the reads went off the end of the parameter arrays. The kernel now launches one program per request and loops the tiles at runtime, mirroring the TileLang version, so the row count comes from device memory and no host read is needed (which would have been rejected as a synchronising operation anyway). `num_steps` is accepted and ignored. The A/B case for this kernel is also fixed, in two ways that matter: - it used a fixed `total_q` while the request lengths were random, so it allocated more rows than were ever written and the comparison was meaningless; - it pre-filled the outputs with a sentinel but never required them to be overwritten, so both backends could write nothing and still agree with each other. The case now asserts every row was written, and covers `num_steps=0` plus a request longer than one tile. With that, `DEEPINFER_OPS_BACKEND=triton` runs end to end and produces exactly the same tokens as the pure TileLang run, `[17607, 2793, 3139, 666, 6127, 666, 320, 1]`, identical on all eight ranks. Fifteen of the required kernels are Triton; the remainder still come from TileLang through the fallback while they are ported.		2026-10-22 07:26:09 +08:00
..
_C	initial commit	2026-10-20 12:09:42 +08:00
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	initial commit	2026-10-20 12:09:42 +08:00
entrypoints	initial commit	2026-10-20 12:09:42 +08:00
h20_ops	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
hugging_face	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
model_executor	fix(triton): the grid for prepare_verification_metadata, and first matching end-to-end run	2026-10-22 07:26:09 +08:00
npu_ops	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
structure_output	initial commit	2026-10-20 12:09:42 +08:00
ve	initial commit	2026-10-20 12:09:42 +08:00
worker	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
attrdict.py	initial commit	2026-10-20 12:09:42 +08:00
config.py	initial commit	2026-10-20 12:09:42 +08:00
debug_dump.py	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
environ.py	initial commit	2026-10-20 12:09:42 +08:00
load_class.py	initial commit	2026-10-20 12:09:42 +08:00
logger.py	initial commit	2026-10-20 12:09:42 +08:00
model_config_loader.py	initial commit	2026-10-20 12:09:42 +08:00
multimodal_tokens.py	initial commit	2026-10-20 12:09:42 +08:00
platform.py	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
py.typed	initial commit	2026-10-20 12:09:42 +08:00
sampling_params.py	initial commit	2026-10-20 12:09:42 +08:00
sequence.py	initial commit	2026-10-20 12:09:42 +08:00
shared_memory.py	initial commit	2026-10-20 12:09:42 +08:00
testing.py	initial commit	2026-10-20 12:09:42 +08:00
timer.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	initial commit	2026-10-20 12:09:42 +08:00