loopyt 0d144cbce4 feat(ops): report fallbacks at use time, and record why the sampler stays put ... The import-time line only counts kernels the selected backend does not export, which says nothing about whether a running configuration reaches them. `_FallbackNotice` wraps each inherited kernel and prints once, on first call, so an end-to-end run states the answer itself. On 8xH20 with speculative decoding off that answer is a single line: [ops] kernel 'fused_sample_tl' used from 'cuda': 'triton' does not implement it Eighteen of the nineteen kernels on that path are Triton and produce exactly the same tokens as the pure TileLang run. The nineteenth is left there on purpose. A probe of the engine's own sampling parameters (`DEEPINFER_PROBE_SAMPLING=1`, which stages an async D2H copy and reports at exit so it does not synchronise inside the forward) settles what `fused_sample` is being asked to do: temperatures: [1.0, 1.0] top_ps: [0.99, 0.99] top_ks: [-1, -1] That is a real nucleus sample over 4096 candidates, not an argmax, so the draw matters. Two things stand in the way of reproducing it bit for bit, both internal to TileLang: `T.rng_init`/`T.rng_rand` are a Philox stream implemented in its C++ codegen, so a Triton kernel draws a different stream and every sampled token at temperature > 0 would change; and the baseline pins the layout of its LSE accumulator precisely to fix the summation order, after which probabilities are quantised into a 2^31-scaled uint32 CDF where a one-ulp difference can move a column across a boundary. Swapping it anyway would be a behaviour change reported as a kernel swap. The per-kernel fallback exists for exactly this case, so the sampler stays on TileLang and the rest of the path does not. `triton/__init__.py` records this so the gap is not mistaken for an oversight, along with the order in which porting it later would have to proceed: a Philox that matches the stream exactly, then the radix filter, then the fixed-point CDF and the two-level search.		2026-10-22 08:05:05 +08:00
..
_C	initial commit	2026-10-20 12:09:42 +08:00
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	initial commit	2026-10-20 12:09:42 +08:00
entrypoints	initial commit	2026-10-20 12:09:42 +08:00
h20_ops	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
hugging_face	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
model_executor	feat(ops): report fallbacks at use time, and record why the sampler stays put	2026-10-22 08:05:05 +08:00
npu_ops	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
structure_output	initial commit	2026-10-20 12:09:42 +08:00
ve	initial commit	2026-10-20 12:09:42 +08:00
worker	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
attrdict.py	initial commit	2026-10-20 12:09:42 +08:00
config.py	initial commit	2026-10-20 12:09:42 +08:00
debug_dump.py	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
environ.py	initial commit	2026-10-20 12:09:42 +08:00
load_class.py	initial commit	2026-10-20 12:09:42 +08:00
logger.py	initial commit	2026-10-20 12:09:42 +08:00
model_config_loader.py	initial commit	2026-10-20 12:09:42 +08:00
multimodal_tokens.py	initial commit	2026-10-20 12:09:42 +08:00
platform.py	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
py.typed	initial commit	2026-10-20 12:09:42 +08:00
sampling_params.py	initial commit	2026-10-20 12:09:42 +08:00
sequence.py	initial commit	2026-10-20 12:09:42 +08:00
shared_memory.py	initial commit	2026-10-20 12:09:42 +08:00
testing.py	initial commit	2026-10-20 12:09:42 +08:00
timer.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	initial commit	2026-10-20 12:09:42 +08:00