7 Commits

Author SHA1 Message Date
Yijun Zhao
a3a6d7e186
Merge 479d7f787358baf1cb0c0704733e099210a382a2 into 553741557a57df44a46fb3ab72b6aafeaf945337 2026-10-26 13:48:15 +08:00
Inference_
553741557a
fix: validate boolean environment variables with strict parsing (#218) 2026-10-25 19:59:27 +08:00
Shaoyuan CHEN
9bd9cf243f
Bump DeepEP to f6aa38fe and drop the SSH->HTTPS rewrite (#157)
DeepEP now records an HTTPS DeepJIT URL, so build_vendor no longer needs the git@github.com rewrite.

Co-authored-by: Shaoyuan Chen <chensy@deepseek.com>
2026-10-21 12:55:43 +08:00
Yijun Zhao
479d7f7873 engine: refactor Scheduler for readability without changing scheduling behavior
- Turn Scheduler into a class: only waiting/running/decode_step (bound by
  pybind) stay public; internals become private.
- Move immutable knobs into SchedulerConfig::capture(), alongside the
  constructor-time checks.
- Drop dead state: dp_size, dp_rank, truncate_len, schedule_counter,
  last_step_*, tokens_chunked_prefill_remaining, and the max_num_prompts
  alias. The unused truncate_len constructor argument is removed from the
  binding, the stub and llm_engine.
- Split _schedule into _schedule_normal / _schedule_prefill /
  _schedule_decode; decode is further split into _admit_prompted and
  _select_decode_batch.
- Share the forked-KV detach logic between prefill and decode
  (_detach_forked_cache) instead of two hand-written copies.
- prompted no longer carries a tuple<seq, bool>: the flag was always true.
- process_outputs: the returned buffers become per-instance members instead
  of function-level statics shared by all Scheduler instances. The binding
  returns them as zero-copy numpy arrays whose base is the Scheduler, so
  holding an array keeps the storage alive.
- SchedulerOutputs is filled incrementally instead of via a 10-arg ctor.
- Fix the LCFS comment: preemption victims are the longest-output seqs.
- block_manager.hpp now includes <sys/time.h> itself for gettimeofday.

Verified with a differential harness that drives the old and new
Scheduler through identical randomized workloads (NORMAL/PROMPT/COMPLETION,
chunked prefill, MTP varlen, forks, preemption, aborts, multimodal chunk
boundaries, simulated multi-rank votes) and compares full per-step state:
120/120 traces identical on macOS/clang and Linux/GCC 15; a Python smoke
test through the real pybind bindings is also identical old vs new.
2026-10-20 20:42:29 +08:00
hkeee21
774ce244b8 update Q&A EN doc 2026-10-20 19:03:13 +08:00
Jiashi Li
8b16149741 update Q&A documentation 2026-10-20 17:06:57 +08:00
Jiashi Li
d585e6542c initial commit 2026-10-20 12:09:42 +08:00