whn09 d54fe25993 serve: Mooncake Transfer Engine KV transport for the prefill/decode split (--kv-transport mooncake) ... Stacked on #197 (pd_server.py / pd_transfer.py), which moves the prefilled KV blocks and SWA state PE -> CE as pinned host buffers over TCP. --kv-transport mooncake keeps that flow and control socket, and replaces the data plane with a Mooncake TransferEngine: - deepinfer/hicache/pd_mooncake.py: GpuArena (one registered device buffer per rank, first-fit regions; cuMemAlloc like the KV pools, since expandable_segments memory has no cudaIpc handle) and MooncakeLink (TransferEngine with P2PHANDSHAKE metadata, register_memory, transfer_sync_write). - PDTransfer.export_into / admit accept device buffers: the PE exports a sequence straight into an arena region (state included, no extra staging) and the CE admits from its arena. - pd_server.py: MC_MAGIC header -> CE reserves an arena region and replies {addr, segment} (or busy -> retry) -> one contiguous write -> go/ok. Writes are retried, then the region is sent over the TCP path, so a failing peer cannot strand a request. TCP is unchanged and is also the fallback when the PE arena is full. - --mooncake-protocol nvlink_intra (default, one host) or rdma/efa across hosts. - serve/README.md: usage; containers must share IPC and PID namespaces (--pid=host). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>		2026-10-26 15:09:01 +08:00
..
_C	engine: expose Scheduler.add_prompted_sequence and free_finished_seqs	2026-10-21 22:26:29 +00:00
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	initial commit	2026-10-20 12:09:42 +08:00
entrypoints	initial commit	2026-10-20 12:09:42 +08:00
hicache	serve: Mooncake Transfer Engine KV transport for the prefill/decode split (--kv-transport mooncake)	2026-10-26 15:09:01 +08:00
hugging_face	initial commit	2026-10-20 12:09:42 +08:00
model_executor	initial commit	2026-10-20 12:09:42 +08:00
npu_ops	initial commit	2026-10-20 12:09:42 +08:00
structure_output	initial commit	2026-10-20 12:09:42 +08:00
ve	initial commit	2026-10-20 12:09:42 +08:00
worker	initial commit	2026-10-20 12:09:42 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
attrdict.py	initial commit	2026-10-20 12:09:42 +08:00
config.py	initial commit	2026-10-20 12:09:42 +08:00
environ.py	initial commit	2026-10-20 12:09:42 +08:00
load_class.py	initial commit	2026-10-20 12:09:42 +08:00
logger.py	initial commit	2026-10-20 12:09:42 +08:00
model_config_loader.py	initial commit	2026-10-20 12:09:42 +08:00
multimodal_tokens.py	initial commit	2026-10-20 12:09:42 +08:00
platform.py	initial commit	2026-10-20 12:09:42 +08:00
py.typed	initial commit	2026-10-20 12:09:42 +08:00
sampling_params.py	initial commit	2026-10-20 12:09:42 +08:00
sequence.py	initial commit	2026-10-20 12:09:42 +08:00
shared_memory.py	initial commit	2026-10-20 12:09:42 +08:00
testing.py	initial commit	2026-10-20 12:09:42 +08:00
timer.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	initial commit	2026-10-20 12:09:42 +08:00