Jahidul Hossain Mekat b6ddc56a0c Release the KV blocks and state slot of requests aborted while waiting ... A request that is aborted (or fails multimodal processing / structured output setup) while it sits in `waiting` is popped by the scheduler and handed back as an ignored sequence. Nothing released what it still held. A long prompt that is prefilled in chunks is moved back to `waiting` after each chunk with the KV blocks of the whole prompt (and its state slot) already allocated, so aborting it between two chunks leaked those blocks for the lifetime of the engine, and so did a fork that pinned its parent's blocks. Every such abort shrinks the cache until the engine can no longer admit new requests. _schedule_prefill and the waiting sweep in _schedule_decode now release the sequence's resources when they drop it. free_seq_nolock also reset nothing after freeing the state slot, so releasing a sequence a second time (a preempted request that is aborted had already given its slot back) freed the slot twice and tripped the block manager's "ref_count corrupted" check. It now clears state_id, which makes the release idempotent like the block table already was. Tests drive the real scheduler through schedule / free / process_outputs on a CPU process group. A decode step in the middle of a prefill is forced through decode_step, which is what a data-parallel rank sees when the other ranks do not agree to a prompt step.		2026-10-20 14:49:40 +00:00
..
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	Release the KV blocks and state slot of requests aborted while waiting	2026-10-20 14:49:40 +00:00
models	initial commit	2026-10-20 12:09:42 +08:00
ve	initial commit	2026-10-20 12:09:42 +08:00
worker	initial commit	2026-10-20 12:09:42 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
conftest.py	initial commit	2026-10-20 12:09:42 +08:00
test_cuda_sync_guards.py	initial commit	2026-10-20 12:09:42 +08:00