whn09 6c653df08c neuron: drafter forward as one region, compiled dflash metadata fill; verification + drafts regions on by default ... vllm-sonnet 512 (Trn2-Hy4): decode step 660 -> 591 ms, TPOT 221.7 -> 206.5 ms (p50), TTFT 4.02 -> 3.93 s, tokens/step identical (3.02/2.82/2.65). Tiny golden 9/9. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>		2026-10-27 00:14:32 +08:00
..
__init__.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
_unsupported.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
candidate_blocks.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
compress.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
compute_seqlens_k_zos.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
dflash_copy.py	Neuron basic mode — part 2: deep_ep, sampling kernels, fixes	2026-10-20 18:57:27 +08:00
dflash.py	Neuron DSpark (#15): greedy verification and MTP ops, CP followers run the drafts	2026-10-24 11:31:59 +08:00
flash_mla_utils.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
fused_sample.py	neuron: compiled decode verification behind DEEPINFER_NEURON_COMPILE_VERIFY (default off)	2026-10-26 21:33:34 +08:00
fused_transform.py	Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25)	2026-10-23 01:39:22 +08:00
gather_engram_tokens_zos.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
gather_input_ids_zos.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
gather.py	Neuron: eager embedding/gather without dtype casts or host scalar operands	2026-10-25 06:55:43 +08:00
get_kv_index.py	neuron: drafter forward as one region, compiled dflash metadata fill; verification + drafts regions on by default	2026-10-27 00:14:32 +08:00
image_embed_ops.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
kv_index_cat.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
kv_quant.py	Neuron: tiny-model decode region graphs compile and run on a NeuronCore (issues #9, #25)	2026-10-22 18:38:01 +08:00
load_paged_kv_cache.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
norm.py	Neuron: full DeepSeek-V4.1-Flash load + decode region graphs at EP=64 (issues #21, #25)	2026-10-23 01:39:22 +08:00
prepare_cp_mirror.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
prepare_v4_decode_metadata.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
random_sampling.py	Neuron: integer mod without torch.remainder in the sampler; sampler cProfile (#27)	2026-10-23 08:10:16 +08:00
rstd_1ulp.py	Neuron: per-layer region graphs + vllm-neuron#71 NKI kernels (issues #10, #11, #13, #25)	2026-10-22 17:36:54 +08:00
sampler.py	Neuron DSpark (#15): greedy verification and MTP ops, CP followers run the drafts	2026-10-24 11:31:59 +08:00
schedule_mtp_varlen.py	Neuron basic mode — part 2: deep_ep, sampling kernels, fixes	2026-10-20 18:57:27 +08:00
state_cache.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00
store_kv_cache.py	Neuron: tiny-model decode region graphs compile and run on a NeuronCore (issues #9, #25)	2026-10-22 18:38:01 +08:00
store_mtp_info.py	Neuron DSpark (#15): greedy verification and MTP ops, CP followers run the drafts	2026-10-24 11:31:59 +08:00
structure_output.py	Neuron basic mode — part 2: deep_ep, sampling kernels, fixes	2026-10-20 18:57:27 +08:00
sum_smaller_probs.py	Neuron basic mode — part 2: deep_ep, sampling kernels, fixes	2026-10-20 18:57:27 +08:00
swa_index.py	Add AWS Trainium (Neuron) platform, basic mode — part 1	2026-10-20 18:34:51 +08:00