coffee/DeepInfer
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepInfer.git synced 2026-10-28 14:57:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

Neuron: whole-layer region graphs by default (TPOT 447.8 -> 430.3 ms at TP=64)

Browse Source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
whn09 2026-10-24 10:15:43 +08:00
parent 3a8ef041db
commit 5e89a12997
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
deepinfer/environ.py
View File

@ -115,7 +115,7 @@ class envs:
# Neuron only: cut the region graphs further at the stage boundaries marked by neuron_ops.regions.split().
DEEPINFER_NEURON_REGION_SPLIT = EnvVar.bool(default=False)
# Neuron only: one region graph per decoder layer (attention half + MoE) instead of two.
DEEPINFER_NEURON_REGION_LAYER = EnvVar.bool(default=False)
DEEPINFER_NEURON_REGION_LAYER = EnvVar.bool(default=True)
# Neuron only: split the region graphs of prefill chunks (steps of more than one row) regardless.
DEEPINFER_NEURON_REGION_SPLIT_PREFILL = EnvVar.bool(default=True)
# Neuron only: synchronize around every region graph call and log per-kind times each decode step (profiling).