mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-10-29 07:17:08 +08:00
f4144fc3fc
- vendor/deep_ep: EPBuffer dispatch/combine with torch.distributed uneven all_to_all (host-synced split sizes), EngramBuffer sharded lookup, BucketBuffer all_gather (CP-only reduce paths stubbed); verified against the DeepEP-Ascend references at world size 1/2/4. - tilelang_ops/neuron sampling: random_sampling (asc two-level fixed-point sampler, Philox), fused_sample (logprobs / do_sample), sampler metadata, sum_smaller_probs, structured-output bitmask; DSpark/MTP/varlen kernels raise NotImplementedError. All 42 asc *_tl factories now have a Neuron counterpart. - vendor/flash_attn: SDPA varlen stand-in for the ViT (mirrors npu_ops). - transfer_to_neuron: keep `with torch.device(...)` working on patched factories; fix manual_seed recursion in host mode. - engram: gather the packed ue8m0 sf as bytes on Neuron. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
19 lines
664 B
Python
19 lines
664 B
Python
"""Mirrors ``tilelang_ops/asc/dflash.py`` (confidence_head_fused_tl, dflash_input_store_tl).
|
|
|
|
DFlash (MTP drafting) only: the draft module and the decoder-layer DFlash input store exist only with
|
|
DEEPINFER_USE_NEXTN_DECODE>0. Out of scope on Neuron basic mode (NEURON_PORT.md "Scope"): the factories
|
|
keep the asc signatures and raise NotImplementedError.
|
|
"""
|
|
|
|
from typing import Any
|
|
|
|
from ._unsupported import unsupported
|
|
|
|
|
|
def confidence_head_fused_tl(n_embed: int, markov_rank: int, has_markov_emb: bool) -> Any:
|
|
unsupported("confidence_head_fused_tl")
|
|
|
|
|
|
def dflash_input_store_tl(n_embed: int, hc_mult: int) -> Any:
|
|
unsupported("dflash_input_store_tl")
|