whn09 1ecd3d9859 neuron: norms feeding only NKI kernels skip their dead fp8 copy (TTFT 1387 -> 1229 ms, step 457 -> 450 ms, tokens identical); capacity MoE wired into the layer region graph with its overflow flag riding the MoE all-reduce (still off: per-layer host sync makes prefill slower, 1823 ms) ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>		2026-10-28 05:29:40 +08:00
..
_C	initial commit	2026-10-20 12:09:42 +08:00
distributed	initial commit	2026-10-20 12:09:42 +08:00
engine	neuron: diagnostics for MoE collective skew (expert flag force, per-follower runtime profiling)	2026-10-26 15:14:19 +08:00
entrypoints	initial commit	2026-10-20 12:09:42 +08:00
hugging_face	Neuron: concatenate FP8 checkpoint tensors as bytes (fixes MoE on device)	2026-10-21 11:46:58 +08:00
model_executor	neuron: norms feeding only NKI kernels skip their dead fp8 copy (TTFT 1387 -> 1229 ms, step 457 -> 450 ms, tokens identical); capacity MoE wired into the layer region graph with its overflow flag riding the MoE all-reduce (still off: per-layer host sync makes prefill slower, 1823 ms)	2026-10-28 05:29:40 +08:00
neuron_ops	neuron: norms feeding only NKI kernels skip their dead fp8 copy (TTFT 1387 -> 1229 ms, step 457 -> 450 ms, tokens identical); capacity MoE wired into the layer region graph with its overflow flag riding the MoE all-reduce (still off: per-layer host sync makes prefill slower, 1823 ms)	2026-10-28 05:29:40 +08:00
npu_ops	initial commit	2026-10-20 12:09:42 +08:00
structure_output	initial commit	2026-10-20 12:09:42 +08:00
ve	Map media-free requests inline instead of through the multimodal worker (DEEPINFER_MULTI_MODAL_INLINE_TEXT=1)	2026-10-27 04:56:47 +08:00
worker	neuron: Engram gather via numpy (torch index_select waited 12-26 ms on its contended intra-op pool); DEEPINFER_NEURON_TORCH_THREADS; CPU bind mode 'all'	2026-10-28 02:10:43 +08:00
__init__.py	Neuron basic mode: try prefill on every step	2026-10-21 21:00:54 +08:00
attrdict.py	initial commit	2026-10-20 12:09:42 +08:00
config.py	initial commit	2026-10-20 12:09:42 +08:00
environ.py	neuron: dense prefill attention over small KV tables and prefill table bounds (both opt-in / dormant by default)	2026-10-27 14:11:06 +08:00
load_class.py	initial commit	2026-10-20 12:09:42 +08:00
logger.py	initial commit	2026-10-20 12:09:42 +08:00
model_config_loader.py	initial commit	2026-10-20 12:09:42 +08:00
multimodal_tokens.py	initial commit	2026-10-20 12:09:42 +08:00
platform.py	neuron: low-latency completion off by default; rank timeline (DEEPINFER_NEURON_RANK_TIMELINE) and Engram prefault	2026-10-27 22:24:34 +08:00
py.typed	initial commit	2026-10-20 12:09:42 +08:00
sampling_params.py	initial commit	2026-10-20 12:09:42 +08:00
sequence.py	initial commit	2026-10-20 12:09:42 +08:00
shared_memory.py	initial commit	2026-10-20 12:09:42 +08:00
testing.py	initial commit	2026-10-20 12:09:42 +08:00
timer.py	initial commit	2026-10-20 12:09:42 +08:00
utils.py	neuron: Engram gather via numpy (torch index_select waited 12-26 ms on its contended intra-op pool); DEEPINFER_NEURON_TORCH_THREADS; CPU bind mode 'all'	2026-10-28 02:10:43 +08:00