coffee/DeepInfer
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepInfer.git synced 2026-10-31 00:36:20 +08:00
Code Issues Packages Projects Releases Wiki Activity

serve/pd_server: drop the redundant expert slots when EPLB is off, as openai_server.py does

Browse Source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YNgkNrJL5oR1pZqVYzZMHW
This commit is contained in:
Jiejing 2026-10-25 21:36:15 +00:00
parent 754bcb5b43
commit 9e455f55d9
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
serve/pd_server.py
View File

@ -107,6 +107,9 @@ def worker_main(args: argparse.Namespace, dp_rank: int, in_q: mp.Queue, out_q: m
os.environ["DEEPINFER_PREFILL_STORE_EXTRA_STATE"] = "1"
os.environ.setdefault("DEEPINFER_PREFILL_STORE_EXTRA_SIZE", "512") # identical state layout on PE and CE
os.environ.setdefault("DEEPINFER_PREFILL_ANY_ATTEMPT_INTERVAL", str(args.prefill_any_interval))
if args.eplb_interval == 0:
# Same as openai_server.py: without EPLB the redundant expert slots only add expert weight per rank.
os.environ.setdefault("DEEPINFER_OVERRIDE_NUM_EXTRA_EXPERTS", "0")
setup_common_env(model_name=args.model, sampler_force_topk=1024, set_override_experts=True, enable_eplb=False)
if args.eplb_interval > 0:
os.environ["DEEPINFER_EXPERT_REARRANGEMENT_STEP_INTERVAL"] = str(args.eplb_interval)