zhaozhuoliang 0e36d808b5 perf: decouple the Engram buffer size from max_num_batched_tokens ... The number of RDMA QPs DeepEP allocates for the Engram buffer is derived from its max_num_tokens, which in turn comes from max_num_batched_tokens (model_loader.py). Raising the QP count therefore required raising the global batched-token budget, which also inflates the EP/MoE/CP buffers. Add DEEPINFER_ENGRAM_MIN_PREFILLING_TOKENS as a lower bound applied only to the Engram buffer, so a deployment can give Engram more QPs without resizing the other buffers. Default 0 keeps the current behavior.		2026-10-27 20:37:03 +08:00
..
eplb	initial commit	2026-10-20 12:09:42 +08:00
input_metadata	initial commit	2026-10-20 12:09:42 +08:00
layers	initial commit	2026-10-20 12:09:42 +08:00
models	perf: decouple the Engram buffer size from max_num_batched_tokens	2026-10-27 20:37:03 +08:00
parallel_utils	initial commit	2026-10-20 12:09:42 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
layer_utils.py	initial commit	2026-10-20 12:09:42 +08:00
model_loader.py	initial commit	2026-10-20 12:09:42 +08:00
model_metadata_utils.py	initial commit	2026-10-20 12:09:42 +08:00
model_output.py	initial commit	2026-10-20 12:09:42 +08:00
sampling_inputs.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	initial commit	2026-10-20 12:09:42 +08:00