loopyt 29fed050cb feat(triton): port the paged KV cache read and write ... store_kv_cache and load_paged_kv_cache, both byte-for-byte identical to the TileLang baseline on the layouts SM90 uses: main/SWA KV (e4m3 values, ue8m0 scale bytes, 528 B per token) and index KV (128 fp8 + one fp32 scale, 132 B per token). The store also covers fp32 scales, because the index layout uses them. FP4 caches and the packed-ue8m0 index layout are refused instead of approximated. Store cases compare the whole cache buffer, including the `-1` invalid-slot path and a block size (134) that is not a power of two. Load cases cover dequantising to BF16, the index passthrough, and holes expressed as `src_block_id == -1`. Two traps worth recording: The load kernel's `other=` default has to be a float literal. With an fp8 source pointer, `other=0` fails to type-check with a message about casting an int32 tile to fp8e4nv, which points at the wrong line. The load cases originally filled the cache with random bytes. Random bytes decode as fp8 NaN and as NaN scale factors, and NaN compares unequal to itself, so the harness reported mismatches that said nothing about the kernel. The cases now build the cache with `store_kv_cache` and compare index outputs as raw bytes, where a NaN bit pattern is just a byte pattern. 15 kernels and 62 cases agree.		2026-10-22 00:33:33 +08:00
..
_C	initial commit	2026-10-20 12:09:42 +08:00
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	initial commit	2026-10-20 12:09:42 +08:00
entrypoints	initial commit	2026-10-20 12:09:42 +08:00
h20_ops	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
hugging_face	style(sm90): satisfy ruff and add the missing type annotations	2026-10-21 17:09:08 +08:00
model_executor	feat(triton): port the paged KV cache read and write	2026-10-22 00:33:33 +08:00
npu_ops	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
structure_output	initial commit	2026-10-20 12:09:42 +08:00
ve	initial commit	2026-10-20 12:09:42 +08:00
worker	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
__init__.py	initial commit	2026-10-20 12:09:42 +08:00
attrdict.py	initial commit	2026-10-20 12:09:42 +08:00
config.py	initial commit	2026-10-20 12:09:42 +08:00
debug_dump.py	fix(debug): move debug_dump out of h20_ops so other platforms can import it	2026-10-21 22:37:49 +08:00
environ.py	initial commit	2026-10-20 12:09:42 +08:00
load_class.py	initial commit	2026-10-20 12:09:42 +08:00
logger.py	initial commit	2026-10-20 12:09:42 +08:00
model_config_loader.py	initial commit	2026-10-20 12:09:42 +08:00
multimodal_tokens.py	initial commit	2026-10-20 12:09:42 +08:00
platform.py	feat: add SM90 (Hopper) support for DeepSeek-V4.1 inference	2026-10-21 15:18:56 +08:00
py.typed	initial commit	2026-10-20 12:09:42 +08:00
sampling_params.py	initial commit	2026-10-20 12:09:42 +08:00
sequence.py	initial commit	2026-10-20 12:09:42 +08:00
shared_memory.py	initial commit	2026-10-20 12:09:42 +08:00
testing.py	initial commit	2026-10-20 12:09:42 +08:00
timer.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	initial commit	2026-10-20 12:09:42 +08:00