mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-10-31 00:36:20 +08:00
The dump hooks live in five platform-neutral files -- `sampler.py`, `mlp.py`, `deepseek.py`, `decoder_layer.py`, `model_runner.py` -- and all of them imported `deepinfer.h20_ops.debug_dump`. That is wrong on every platform except SM90: `h20_ops/__init__.py` calls `extend_import_patches(override=True, ...)` at module scope, so importing it from shared code installs the SM90 import shims globally. On an SM100 machine that replaces `flash_mla.decode_mla_mqa_sparse`, `flash_mla.prefill_attn_varlen_sparse` and `deep_select.interface.topk` with the Hopper implementations, and on NPU it would do the same before the Ascend shims get a chance to matter. `debug_dump` itself has nothing platform-specific in it -- `os`, `pathlib`, `numpy`, `torch` -- so it moves to `deepinfer/debug_dump.py` and the five imports follow. Verified that importing it no longer pulls `h20_ops` into `sys.modules`, and re-ran the end-to-end check on H20: same output token for token, `[17607, 2793, 3139, 666, 6127, 666, 320, 1]`. This bug is already in the open PR, so it needs to land before that PR is worth reviewing further. |
||
|---|---|---|
| .. | ||
| mtp | ||
| v4_modules | ||
| vision_encoder | ||
| __init__.py | ||
| decoder_layer.py | ||
| deepseek_vl.py | ||
| deepseek.py | ||