coffee/DeepInfer
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepInfer.git synced 2026-10-28 14:57:09 +08:00
Code Issues Packages Projects Releases Wiki Activity
DeepInfer/deepinfer/model_executor/layers/tilelang_ops/neuron
History
whn09 b2d94e16b2 Neuron DSpark (#15): greedy verification and MTP ops, CP followers run the drafts 
- fused_sample reject mode for greedy rows (accept iff the draft is the
  argmax), get_accept_result / finalize_verification (static final rows),
  store_mtp_info (static pool writes, slot 0 kept), confidence_head_fused,
  dflash_input_store, get_fused_dflash_get_input: the frontend refs with the
  kernels' argument lists.
- CP followers run the draft forward (its MoE has collectives); the driver's
  accepted offsets reach them through one all-reduce on Neuron (no eager
  broadcast), the request first rows through the graph metadata.
- run_deepinfer_tiny.py --nextn: free-running greedy with drafts. Host tiny,
  nextn=5, eager: short 9/9 golden tokens; long matches up to its known near
  tie at step 5.
- bench17: the two-level MoE all-reduce does not move TPOT (433 ms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 11:31:59 +08:00
..
__init__.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
_unsupported.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
candidate_blocks.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
compress.py Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25) 2026-10-22 17:36:54 +08:00
compute_seqlens_k_zos.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
dflash_copy.py Neuron basic mode — part 2: deep_ep, sampling kernels, fixes 2026-10-20 18:57:27 +08:00
dflash.py Neuron DSpark (#15): greedy verification and MTP ops, CP followers run the drafts 2026-10-24 11:31:59 +08:00
flash_mla_utils.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
fused_sample.py Neuron DSpark (#15): greedy verification and MTP ops, CP followers run the drafts 2026-10-24 11:31:59 +08:00
fused_transform.py Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25) 2026-10-23 01:39:22 +08:00
gather_engram_tokens_zos.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
gather_input_ids_zos.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
gather.py Neuron: static decode graphs, static-shape MoE, trn2 e4m3 fp8 (issues #9, #12, #24) 2026-10-22 11:59:37 +08:00
get_kv_index.py Neuron DSpark (#15): greedy verification and MTP ops, CP followers run the drafts 2026-10-24 11:31:59 +08:00
image_embed_ops.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
kv_index_cat.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
kv_quant.py Neuron: tiny-model decode region graphs compile and run on a NeuronCore (issues #9, #25) 2026-10-22 18:38:01 +08:00
load_paged_kv_cache.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
norm.py Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25) 2026-10-23 01:39:22 +08:00
prepare_cp_mirror.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
prepare_v4_decode_metadata.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
random_sampling.py Neuron: integer mod without torch.remainder in the sampler; sampler cProfile (#27) 2026-10-23 08:10:16 +08:00
rstd_1ulp.py Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25) 2026-10-22 17:36:54 +08:00
sampler.py Neuron DSpark (#15): greedy verification and MTP ops, CP followers run the drafts 2026-10-24 11:31:59 +08:00
schedule_mtp_varlen.py Neuron basic mode — part 2: deep_ep, sampling kernels, fixes 2026-10-20 18:57:27 +08:00
state_cache.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00
store_kv_cache.py Neuron: tiny-model decode region graphs compile and run on a NeuronCore (issues #9, #25) 2026-10-22 18:38:01 +08:00
store_mtp_info.py Neuron DSpark (#15): greedy verification and MTP ops, CP followers run the drafts 2026-10-24 11:31:59 +08:00
structure_output.py Neuron basic mode — part 2: deep_ep, sampling kernels, fixes 2026-10-20 18:57:27 +08:00
sum_smaller_probs.py Neuron basic mode — part 2: deep_ep, sampling kernels, fixes 2026-10-20 18:57:27 +08:00
swa_index.py Add AWS Trainium (Neuron) platform, basic mode — part 1 2026-10-20 18:34:51 +08:00