mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-10-28 14:57:09 +08:00
- fused_sample reject mode for greedy rows (accept iff the draft is the argmax), get_accept_result / finalize_verification (static final rows), store_mtp_info (static pool writes, slot 0 kept), confidence_head_fused, dflash_input_store, get_fused_dflash_get_input: the frontend refs with the kernels' argument lists. - CP followers run the draft forward (its MoE has collectives); the driver's accepted offsets reach them through one all-reduce on Neuron (no eager broadcast), the request first rows through the graph metadata. - run_deepinfer_tiny.py --nextn: free-running greedy with drafts. Host tiny, nextn=5, eager: short 9/9 golden tokens; long matches up to its known near tie at step 5. - bench17: the two-level MoE all-reduce does not move TPOT (433 ms). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| mtp | ||
| v4_modules | ||
| vision_encoder | ||
| __init__.py | ||
| decoder_layer.py | ||
| deepseek_vl.py | ||
| deepseek.py | ||