mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-10-28 14:57:09 +08:00
d0f7e4f69c
- requirements-neuron.txt: requirements.txt + dev deps without tilelang / tile-kernels (pure-torch drop-ins are vendored), torch 2.14 + torch-neuronx 2.14 from the Neuron repository. - build_vendor.sh: nothing to build for neuron; setup.py drops tilelang / tile-kernels from install_requires on neuron. - setup.py and get_platform(): an exported DEEPINFER_ACCELERATOR wins over device detection, so images build where no accelerator is visible. - docker/neuron/Dockerfile (+ per-Dockerfile ignore): on the native PyTorch Neuron DLC public.ecr.aws/neuron/torch-neuronx:2.14.0-neuronx-py314- sdk2.33.0-ubuntu26.04; build.sh with TORCH_DEVICE_BACKEND_AUTOLOAD=0 and an import self-check in Neuron CPU mode. - README: Trainium in supported hardware and a "DeepInfer on AWS Trainium" section (English + folded Chinese); the pip torch-neuronx wheel needs glibc >= 2.43, so on Ubuntu 24.04 layer the venv on the DLAMI env. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
47 lines
1.3 KiB
Plaintext
47 lines
1.3 KiB
Plaintext
# AWS Trainium (Neuron SDK 2.33+, native torch-neuronx -- no torch-xla).
|
|
# Same dependencies as requirements.txt + requirements-dev.txt, except:
|
|
# * tilelang / tile-kernels: not needed -- Neuron uses the pure-torch drop-ins vendored in
|
|
# deepinfer/neuron_ops/vendor (deep_gemm, flash_mla, deep_ep, deep_select, tile_kernels, tilelang);
|
|
# * torch comes with torch-neuronx (torch 2.14), not the CUDA/NPU builds.
|
|
# On a Neuron DLAMI the torch-neuronx stack is already in /opt/aws_neuronx_venv_torch_neuronx_2_14_0;
|
|
# tests/neuron/setup_trn2.sh layers a venv on it instead of reinstalling.
|
|
--extra-index-url https://pip.repos.neuron.amazonaws.com
|
|
--find-links https://download.pytorch.org/whl/cpu/torch/
|
|
torch==2.14.0
|
|
torch-neuronx==2.14.*
|
|
neuronx-cc==2.*
|
|
nki>=0.6
|
|
|
|
einops==0.8.2
|
|
huggingface_hub==0.36.2
|
|
ninja==1.13.0
|
|
numpy==2.3.5
|
|
orjson==3.11.7
|
|
pillow==11.3.0
|
|
posix_ipc==1.3.2
|
|
psutil==7.2.2
|
|
pybind11==3.0.1
|
|
pyzmq==27.1.0
|
|
requests==2.32.5
|
|
safetensors==0.8.0
|
|
scipy==1.17.0
|
|
sympy==1.14.0
|
|
tabulate==0.9.0
|
|
tokenizers==0.21.4
|
|
tqdm==4.67.3
|
|
transformers==4.50.0
|
|
xgrammar==0.2.2
|
|
|
|
# development / tests
|
|
clang-format==22.1.3
|
|
execnet==2.1.2
|
|
pybind11-stubgen==2.5.5
|
|
pytest==8.4.2
|
|
pytest-xdist==3.8.0
|
|
ruff==0.15.0
|
|
tree-sitter==0.26.0
|
|
tree-sitter-cpp==0.23.4
|
|
matplotlib==3.11.2
|
|
fastapi==0.128.0
|
|
uvicorn==0.40.0
|