whn09 2ff094fddc hicache: Mooncake Store as an L3 tier under HostPrefixCache (DEEPINFER_HICACHE_MOONCAKE_STORE) ... Stored page groups (DEEPINFER_HICACHE_L3_GROUP_PAGES consecutive pages, default 4) and SWA states are written behind to a Mooncake Store straight from the registered pinned pools (batch_put_from_multi_buffers on a background thread; host slots stay pinned until the put returns). A host miss is extended from the Store: batch_is_exist finds the longest group run and the deepest boundary with a state (host or Store), batch_get_into_multi_buffers fills fresh host slots, and the existing H2D path runs unchanged. Keys are namespaced and carry a versioned layout fingerprint (pages under dsv4kv/<fp>/g<G>/, states under state/<fp>/, as discussed in kvcache-ai/Mooncake#4803). Pages are rank-independent, so prefixes are shared across DP ranks, instances and restarts. Any Store error falls back to the host-only path. tests/hicache/test_host_cache_l3.py: a second cache hits from L3 with exact bytes, no re-puts, an object lost between exists() and get() falls back cleanly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>		2026-10-23 15:52:12 +08:00
..
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	initial commit	2026-10-20 12:09:42 +08:00
hicache	hicache: Mooncake Store as an L3 tier under HostPrefixCache (DEEPINFER_HICACHE_MOONCAKE_STORE)	2026-10-23 15:52:12 +08:00
models	initial commit	2026-10-20 12:09:42 +08:00
ve	initial commit	2026-10-20 12:09:42 +08:00
worker	initial commit	2026-10-20 12:09:42 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
conftest.py	initial commit	2026-10-20 12:09:42 +08:00
test_cuda_sync_guards.py	initial commit	2026-10-20 12:09:42 +08:00