A request that is preempted for recompute was pushed to the front of `waiting`.
_schedule_prefill takes `waiting` in order and stops at the first sequence it
cannot allocate, and a preempted request usually does not fit right away: it was
preempted because it needed a block more than it had, and recomputing it takes
that block as well. If a chunked prefill is in progress at that moment (it is
back at the front of the queue holding the KV blocks of its whole prompt), the
preempted request lands in front of it and ends the scan before the prefill is
reached. The prefill never gets its next chunk, so it never finishes and never
releases the blocks the preempted request is waiting for, and the engine stops
making progress (or keeps recomputing the requests that are left, which are
preempted again for the same reason).
Decode steps between two chunks of a prompt are all it takes. They happen when
the cache usage is above the prefill watermark, and on a data-parallel rank
whenever the other ranks do not agree to a prompt step.
Queue the preempted request ahead of the fresh requests, as before, but behind
the sequences that already hold blocks. They only need compute to finish. A
sequence that cannot be scheduled yet (a multimodal request that is still being
preprocessed, but already owns its blocks as in examples/benchmark.py) is not
one of them: _schedule_prefill stops at it, so the preempted request stays in
front of it.
A request that is aborted (or fails multimodal processing / structured output
setup) while it sits in `waiting` is popped by the scheduler and handed back as
an ignored sequence. Nothing released what it still held. A long prompt that is
prefilled in chunks is moved back to `waiting` after each chunk with the KV
blocks of the whole prompt (and its state slot) already allocated, so aborting
it between two chunks leaked those blocks for the lifetime of the engine, and
so did a fork that pinned its parent's blocks. Every such abort shrinks the
cache until the engine can no longer admit new requests.
_schedule_prefill and the waiting sweep in _schedule_decode now release the
sequence's resources when they drop it.
free_seq_nolock also reset nothing after freeing the state slot, so releasing a
sequence a second time (a preempted request that is aborted had already given
its slot back) freed the slot twice and tripped the block manager's
"ref_count corrupted" check. It now clears state_id, which makes the release
idempotent like the block table already was.
Tests drive the real scheduler through schedule / free / process_outputs on a
CPU process group. A decode step in the middle of a prefill is forced through
decode_step, which is what a data-parallel rank sees when the other ranks do
not agree to a prompt step.
