coffee/DeepInfer
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepInfer.git synced 2026-10-29 23:37:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs(mtp_varlen): state zero-rate participation at the ranking site

Browse Source
This commit is contained in:
Zhiren Xiao 2026-10-26 16:17:33 +08:00
parent 553741557a
commit 06cfc90597
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
deepinfer/model_executor/layers/tilelang_ops/schedule_mtp_varlen.py
View File

@ -69,6 +69,11 @@ def compute_seqlens_q_by_confidence_rates_ref(
# Phase 2: rank-based selection
# Flatten all (seq, pos) rates and find the top-k by rate (higher wins, tie-break by flat index ascending)
# Cells with cumprod 0.0 take part in this ranking too: remap == -1 (fresh seq) and a seq whose draft
# confidences were masked to 0 after emitting a stop token (model_executor/layers/sampler.py) both
# contribute all-zero rows. Those rows are reached once extra_budget exceeds the number of positive
# cells, and are then filled in flat-index order, so which zero-confidence seq is verified beyond its
# base token depends on batch position rather than on confidence.
extra_budget = total_q - num_seqs # number of extra tokens to distribute
flat_rates = cum_rates.reshape(-1) # [num_seqs * nextn]