coffee/DeepInfer
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepInfer.git synced 2026-10-29 23:37:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge 0f48b6130e4c8d4dd6857b2af0d18ff78c608c49 into 553741557a57df44a46fb3ab72b6aafeaf945337

Browse Source
This commit is contained in:
ledd 2026-10-26 16:27:46 +08:00 committed by GitHub
commit 8136c32b97
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -82,6 +82,8 @@ See [examples](examples/README.md) for details.
Measured on 4 SM100 GPUs for prefill and 64 for decode — each within one scale-up domain. To reproduce the benchmarks, see [run_sweep.sh](examples/sweep/run_sweep.sh).
These are offline engine sweeps, not end-to-end serving measurements. Prefill throughput counts only non-cached input tokens. The decode plot measures steps per second, then converts both axes to tokens per second using an assumed draft acceptance of 75% with `nextn=3` (`1 + 3 × 0.75 = 3.25` tokens per step per request). The 75% value is supplied to the plotting script, not inferred from the sweep. See [benchmark methodology](examples/README.md#benchmark) for the measurement windows and token accounting.
## Evaluation benchmarks
See [eval/README.md](eval/README.md).