- bench_neuron.py --dataset vllm-sonnet: vLLM's SonnetDataset as #71's bench
builds it for the completions backend ("Pick as many lines..." + random poem
lines, chat-formatted; --chat_tokenizer for a template the checkpoint lacks).
--dataset sonnet: a contiguous text window. --dump: prompt/output token ids.
- reference_dspark.py: the official inference/model.py forward_spec on the host
CPU (tiny checkpoint, teacher-forced on reference_golden.py tokens): each
decode step's 5 drafts and their logits.
- run_deepinfer_tiny.py RUN_TINY_DRAFTS=1 prints every draft block.
DeepInfer's drafts match the reference on host eager, on device and with
attention TP: every divergence is a near tie (top-1/top-2 gap 0.001-0.055).
With #71's prompt format the acceptance is #71's: 2.8 tokens a step at 512
(first position 65-79%); a raw text window gave 1.4 (continuations not in the
context), random token ids 3.85 (repetitive outputs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
