mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-10-28 23:07:09 +08:00
ed49792a0c
Run DeepInfer on the native torch-neuronx device (PrivateUse1 `neuron`, torch-neuronx 2.14, no torch-xla), following the Ascend port's structure. - deep_cluster: Neuron discovery (one DeepInfer device = one logical NeuronCore, NEURON_RT_VISIBLE_CORES, NUMA from neuron-ls). - PlatformNeuron: takes the Ascend (is_npu) kernel layouts, adds is_neuron; DEEPINFER_NEURON_DEVICE=cpu runs the same code on the host for correctness work. - neuron_ops/transfer_to_neuron: map "cuda" devices, torch.device and torch.cuda.* onto neuron:0 (counterpart of torch_npu transfer_to_npu). - C++ extension: -DWITH_NEURON host-only build; torch-allocated buffers and tensor-view slot copies instead of raw device-pointer copies. - Pure-torch drop-in vendor packages (Ascend API semantics): deep_gemm, flash_mla, deep_select, tile_kernels, tilelang (constants only). - tilelang_ops/neuron backend: KV cache / metadata / norm / compress kernels; CP, ZOS and VL entry points raise NotImplementedError. - Basic mode defaults on Neuron: eager, no DSpark/MTP drafting, no zero-overhead scheduling; MoE avoids the GPU-only mega kernel; mHC prenorm GEMM uses the Ascend single-split reduction. Not yet included: deep_ep, the sampling kernels, end-to-end harness. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
28 lines
620 B
Bash
Executable File
28 lines
620 B
Bash
Executable File
#!/bin/bash
|
|
|
|
set -euo pipefail
|
|
|
|
case "${DEEPINFER_ACCELERATOR:-}" in
|
|
nvidia|ascend|neuron) ;;
|
|
*)
|
|
echo "Usage: DEEPINFER_ACCELERATOR=nvidia|ascend|neuron $0" >&2
|
|
exit 1
|
|
;;
|
|
esac
|
|
|
|
# Install other Python dependencies
|
|
pip install -r "requirements-${DEEPINFER_ACCELERATOR}.txt"
|
|
|
|
if [ "${DEEPINFER_ACCELERATOR}" = nvidia ]; then
|
|
# Patch NCCL to 2.32.3
|
|
pip install --no-deps --force-reinstall nvidia-nccl-cu13==2.32.3
|
|
fi
|
|
|
|
# Build and install vendored dependencies
|
|
bash ./scripts/build_vendor.sh
|
|
|
|
# Build DeepInfer
|
|
bash ./scripts/build.sh build
|
|
|
|
echo "✅ DeepInfer is successfully built"
|