mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-10-31 08:46:21 +08:00
docs: state precisely what end-to-end parity covers
The two runs agree token for token on the reference chat-prefix input, but a plain-text prompt diverges after two tokens: both backends are internally consistent across all eight ranks, so the difference is a different draw from a slightly different CDF, not nondeterminism. That is the quantisation floor from P4 showing up through sampling rather than through argmax -- and on top of it, a kernel written in a different language has its own reduction order (the norm A/B shows 10 of 655360 bf16 elements one ulp apart, with fp8 bytes and scales identical). At T=1.0 that ulp is enough to flip a coin. So the honest statement is a table, not a claim: identical tokens on the chat-prefix input, same first token but a different branch on a plain-text prompt, and matching the draw itself would require both an identical reduction tree and the same PRNG stream.
This commit is contained in:
parent
0d144cbce4
commit
362fc527eb
@ -146,6 +146,34 @@ token ids: [17607, 2793, 3139, 666, 6127, 666, 320, 1]
|
||||
|
||||
即非 MTP 路径 19 个 kernel 里 18 个已经真的跑在 Triton 上,唯一剩下的是 `fused_sample_tl`。
|
||||
|
||||
### 但要说清楚一件事:随机采样会把 1 ulp 的差异放大成不同的续写
|
||||
|
||||
同一个自然语言 prompt 跑两次,一次纯 TileLang、一次 Triton 后端:
|
||||
|
||||
```
|
||||
TileLang: 北京。\n问题:中国的首都是哪里
|
||||
Triton : 北京。\n分析:这是一个基础地理常识
|
||||
```
|
||||
|
||||
前两个 token 相同(`北京`、`。`),第三个开始分叉。两边各自 8 个 rank 都是内部一致的,所以不是
|
||||
不确定性,而是**两个后端的 CDF 有一点点不同,于是抽到了不同的分支**。
|
||||
|
||||
原因在 P4 已经量化过:H20 的 per-128 量化让 logits 有 3e-2 量级的差异;而 kernel 换实现之后,
|
||||
连 bf16 归约顺序都会变(`norm` 的 A/B 里 655360 个元素有 10 个差 1 ulp,fp8 字节和 scale 完全一致)。
|
||||
argmax 不受影响(token-ids 那次 8 个 token 全对),但 T=1.0 的采样相当于把这个 ulp 级差异
|
||||
放大成一次不同的硬币。
|
||||
|
||||
**所以「端到端一致」的准确说法是**:
|
||||
|
||||
| 输入 | 结果 |
|
||||
|---|---|
|
||||
| 官方 chat-template token ids,8 token | 逐 token 相同,8 rank 一致 |
|
||||
| 自然语言 prompt | 首个 token 相同,续写在采样分支上分叉 |
|
||||
|
||||
这不是移植错误,也不能靠「再多测几次」消掉 —— 只要采样是随机的、且两个实现的浮点归约顺序不同,
|
||||
就会有分支差异。若要连采样结果都一致,必须同时做到:bit-exact 的归约顺序(Triton 做不到,它有
|
||||
自己的 reduction tree)+ 同一条 PRNG 流(见下一节)。
|
||||
|
||||
### 剩余
|
||||
|
||||
| kernel | 状态 |
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user