mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-11-01 01:06:23 +08:00
compute_seqlens_k_zos, confidence_head_fused and dflash_input_store, all identical to the baseline: 15 cases across the three. `compute_seqlens_k_zos` masks the remap lookup rather than clamping it: `-1` marks a fresh request, and an unmasked `PREV_ACCEPTED + (-1)` would read the element before the buffer. `confidence_head_fused` reads the weight as one flat vector and splits it at `n_embed`, so the hidden and Markov halves are one accumulation instead of two passes. Without a Markov embedding the second pointer is filled with the first tensor rather than left null, since the read is compiled out but the argument still has to be a pointer. The scaled logit is clamped to +-60 before the sigmoid, as the baseline does, so a small temperature cannot produce inf through `exp`. `dflash_input_store` accumulates in fp32 even though both sides are bf16: the baseline sums into an fp32 fragment, and a bf16 accumulator would round differently for hc_mult > 1. One trap: `tl.next_power_of_2` inside a kernel body is not a compile-time constant, so the Markov block size is resolved in the factory and passed as a constexpr. |
||
|---|---|---|
| .. | ||
| tilelang_ops | ||
| __init__.py | ||
| attention_backend.py | ||
| layernorm.py | ||
| linear.py | ||
| mtp_info_pool.py | ||
| rotary_embedding.py | ||
| sampler_utils.py | ||
| sampler.py | ||
| vocab_parallel_embedding.py | ||