mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-10-28 23:07:09 +08:00
torch.cat of float8 tensors (e4m3 and e8m0) returns wrong data on the neuron device (torch-neuronx 2.14). The HF loader concatenates w1/w3 (and their UE8M0 scales) into the fused gate_up weights, so on the NeuronCore every routed and shared expert gate_up scale was zero and the MoE output vanished (issue #1). Concatenate 1-byte float dtypes through uint8 views, which is bit-identical on every platform. tests/neuron/device: create the neuron process group before the runtime initializes (torch-neuronx requirement) and add the float8 cat checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| config.py | ||
| loading.py | ||
| names.py | ||
| parameters.py | ||
| processor.py | ||
| tokenizer.py | ||
| weights.py | ||