Commit Graph

Select branches

Hide Pull Requests

main

nv-dev

#100

#100

#101

#101

#102

#102

#103

#103

#104

#104

#105

#105

#106

#106

#107

#107

#108

#108

#109

#109

#110

#110

#111

#111

#112

#112

#113

#113

#114

#114

#115

#115

#116

#116

#117

#117

#118

#118

#119

#119

#120

#120

#121

#121

#122

#122

#123

#124

#124

#125

#125

#126

#126

#127

#127

#128

#128

#129

#129

#130

#130

#131

#131

#132

#132

#133

#133

#134

#134

#135

#135

#136

#136

#137

#137

#138

#138

#139

#139

#140

#140

#141

#141

#142

#142

#143

#143

#144

#144

#145

#145

#146

#146

#147

#147

#148

#148

#149

#149

#150

#150

#151

#151

#152

#152

#153

#153

#154

#154

#155

#155

#156

#156

#157

#158

#158

#159

#159

#160

#160

#161

#162

#163

#164

#164

#165

#165

#166

#166

#167

#168

#168

#169

#169

#170

#170

#171

#171

#172

#172

#173

#173

#174

#175

#175

#176

#176

#177

#177

#178

#178

#179

#180

#180

#181

#181

#182

#182

#183

#183

#184

#184

#185

#185

#186

#186

#187

#187

#188

#188

#189

#189

#190

#190

#191

#191

#192

#192

#193

#193

#194

#194

#195

#195

#196

#196

#197

#197

#198

#198

#199

#199

#200

#200

#201

#201

#202

#203

#204

#205

#206

#207

#207

#208

#208

#209

#209

#210

#210

#211

#211

#212

#212

#213

#213

#214

#214

#215

#215

#216

#216

#217

#217

#218

#219

#219

#220

#220

#222

#222

#226

#226

#227

#227

#228

#228

#229

#229

#23

#230

#230

#232

#232

#233

#233

#234

#234

#235

#235

#236

#236

#237

#237

#239

#239

#240

#240

#241

#241

#242

#242

#243

#244

#244

#245

#245

#246

#246

#247

#247

#27

#27

#29

#30

#30

#31

#31

#32

#32

#33

#33

#34

#34

#35

#35

#36

#36

#37

#38

#39

#40

#41

#41

#42

#42

#43

#43

#44

#44

#45

#45

#46

#46

#47

#47

#48

#48

#49

#49

#50

#50

#51

#51

#52

#52

#53

#53

#54

#54

#55

#55

#56

#56

#57

#57

#58

#58

#59

#59

#60

#60

#61

#61

#62

#62

#63

#63

#64

#64

#65

#65

#66

#66

#67

#67

#68

#68

#69

#69

#70

#70

#71

#71

#72

#72

#73

#73

#74

#74

#75

#75

#76

#76

#77

#77

#78

#78

#79

#79

#80

#80

#81

#81

#82

#82

#83

#83

#84

#84

#85

#85

#86

#86

#87

#87

#88

#88

#89

#89

#90

#90

#91

#91

#92

#92

#93

#93

#94

#94

#95

#95

#96

#96

#97

#97

#98

#98

#99

#99

Mono Color

• d232d96fb6
  Merge 9a52063a8a0dc86cbbce04fae034e7e037fa3881 into 553741557a57df44a46fb3ab72b6aafeaf945337 fffzlfk 2026-10-29 14:49:59 +08:00
• 572306fc1a
  Merge c4232fb9f6de213362b080289979ac962ba0e69d into 553741557a57df44a46fb3ab72b6aafeaf945337 王鹤男 2026-10-29 06:45:21 +00:00
• c4232fb9f6
  Merge branch 'deepseek-ai:main' into neuron 王鹤男 2026-10-29 14:43:36 +08:00
• fe10625dfc
  Merge 6a934611d538bcce54434c73a886459bca36b021 into 553741557a57df44a46fb3ab72b6aafeaf945337 Md Jahidul Hosssain Mekat 2026-10-29 14:11:02 +08:00
• 14629f12eb
  Merge c2bbb61623c5120364faa879829feabb6d709434 into 553741557a57df44a46fb3ab72b6aafeaf945337 Shuyun Yang 2026-10-28 21:54:34 -07:00
• 2a6e44a999
  Merge d42acdde324375a4c460a30423251c54b4b2336d into 553741557a57df44a46fb3ab72b6aafeaf945337 world-executed 2026-10-28 21:54:34 -07:00
• 47221f7f9f
  Merge d73c0f06cab34867f9d8b146346885ee513aacf3 into 553741557a57df44a46fb3ab72b6aafeaf945337 world-executed 2026-10-28 21:54:34 -07:00
• 066cabf74b
  Merge 62e386fe9ee1ab7bfff409d164cc577e72612fbe into 553741557a57df44a46fb3ab72b6aafeaf945337 qingzhai 2026-10-28 21:54:34 -07:00
• f6d6f78400
  Merge 3a8d5865dd329295d7a62cb4aa58ea9fcd5c7195 into 553741557a57df44a46fb3ab72b6aafeaf945337 Tony Lin 2026-10-28 21:54:34 -07:00
• 8f31ac39ad
  Merge 990ce077054d397b28fbf95c019f0f6e82ee90e8 into 553741557a57df44a46fb3ab72b6aafeaf945337 Aling33 2026-10-28 21:54:34 -07:00
• 7f558a3775 neuron: workers bind memory to their own core's NUMA node; prompt chunks run the attention half as one graph; glibc trims freed heap early (TTFT 796 -> 720 ms on the bench prompt, jitter +-25 -> +-3 ms) whn09 2026-10-29 12:10:31 +08:00
• 700230447c
  Merge f97e82bdb61377eaa10b992296ccd45f9e58286f into 553741557a57df44a46fb3ab72b6aafeaf945337 Yuxiao Huang 2026-10-28 20:27:36 -07:00
• f7e69ccaeb
  Merge 99dff9a825e020c72b355b80fbf0ac9ef58456cd into 553741557a57df44a46fb3ab72b6aafeaf945337 Yuxiao Huang 2026-10-29 10:31:38 +08:00
• e78b983b97 bench_neuron: BENCH_SAME_PROMPT=1 repeats one prompt (run-to-run TTFT jitter is ~50 ms with a fixed prompt) whn09 2026-10-29 01:35:46 +08:00
• dbd57bff65
  Merge d11fda69877939c590c42b26c5522bc00dc80b6b into 553741557a57df44a46fb3ab72b6aafeaf945337 Ziyang Guo 2026-10-29 01:07:07 +08:00
• d11fda6987 fix(ve): align the ultra-wide stretch predicate with the layout clamp Ziyang Guo 2026-10-29 01:04:56 +08:00
• d7650b21c3 neuron: e4m3 KV-cache codes through the hardware conversion in an NKI kernel (neuron_ops/nki/fp8_encode.py; 512x512 0.63 -> 0.11 ms, tokens identical) whn09 2026-10-29 00:21:02 +08:00
• 11f512f0d6 neuron: fp4 expert GEMMs decode their weight in SBUF on both cores (blockquant.fp4_linear_fused_2core, DEEPINFER_NEURON_FP4_FUSED=1): 512x5120x4608 0.527 -> 0.447 ms, TTFT 825 -> 807 ms; tokens differ by rounding whn09 2026-10-28 21:24:41 +08:00
• 07f9209891
  Merge 0a9059c516f9b458f2b271d207a49ac80388763e into 553741557a57df44a46fb3ab72b6aafeaf945337 wukehu 2026-10-28 21:12:59 +08:00
• 0a9059c516 feat: add opt-in local Engram fetch over NVLink wukehu 2026-10-28 20:38:41 +08:00
• 7327516da8
  Merge d1f308e298862d980e2cb1fa63d9b564008489bd into 553741557a57df44a46fb3ab72b6aafeaf945337 u7-u7 2026-10-28 19:55:05 +08:00
• d1f308e298 fix(packaging): retain source distribution build inputs u7-u7 2026-10-28 19:53:55 +08:00
• 54299b06b3
  Merge bf92779eaacea46e113038f71409ce083df05ea3 into 553741557a57df44a46fb3ab72b6aafeaf945337 wch-08 2026-10-28 17:58:19 +08:00
• bf92779eaa fix: fail loudly on per-prompt list length mismatch in generate() w_wangchi 2026-10-28 17:24:50 +08:00
• 8be4f7ff2c neuron: prompt chunks up to 1024 rows keep only the coarse region splits (TTFT 942 -> 813 ms, step 414 -> 406 ms, tokens identical); prefill grouped MoE without host sync (DEEPINFER_NEURON_MOE_PREFILL_CAPACITY, off: 942 -> 1000 ms) whn09 2026-10-28 17:00:00 +08:00
• d27e8951db neuron: the grouped wv projection takes the bf16 context through the NKI qdq kernel instead of an in-graph fp8 encode (TTFT 1009 -> 942 ms, step 437 -> 414 ms, tokens identical) whn09 2026-10-28 14:32:47 +08:00
• 4d0e108c5a
  Merge 17cdd0d852455dedc8546ee7b5ab444877b3f135 into 553741557a57df44a46fb3ab72b6aafeaf945337 Zhang 2026-10-28 09:58:02 +08:00
• feaed768ff
  Merge a8a36f2c74e68fbd9da20e2db85031b39df76064 into 553741557a57df44a46fb3ab72b6aafeaf945337 Zhang 2026-10-28 09:58:02 +08:00
• a19a7f1f45
  Merge 7cd04027633afda9653d8daf3f5ced17b8401446 into 553741557a57df44a46fb3ab72b6aafeaf945337 zz 2026-10-28 09:57:09 +08:00
• deb8c5a958 neuron: single-request prompt chunks attend densely over their own SWA ring and compressed blocks, masks from positions (DEEPINFER_NEURON_PREFILL_DENSE, check mode DEEPINFER_NEURON_PREFILL_DENSE_CHECK) whn09 2026-10-28 09:09:34 +08:00
• 1ecd3d9859 neuron: norms feeding only NKI kernels skip their dead fp8 copy (TTFT 1387 -> 1229 ms, step 457 -> 450 ms, tokens identical); capacity MoE wired into the layer region graph with its overflow flag riding the MoE all-reduce (still off: per-layer host sync makes prefill slower, 1823 ms) whn09 2026-10-28 05:29:40 +08:00
• 8374df4b9c neuron: Engram gather via numpy (torch index_select waited 12-26 ms on its contended intra-op pool); DEEPINFER_NEURON_TORCH_THREADS; CPU bind mode 'all' whn09 2026-10-28 02:10:43 +08:00
• 81aee64ac2 neuron: low-latency completion off by default; rank timeline (DEEPINFER_NEURON_RANK_TIMELINE) and Engram prefault whn09 2026-10-27 22:24:34 +08:00
• 0c428af751
  Merge 94d1f69332a2472e6c76766369340568467d5d8b into 553741557a57df44a46fb3ab72b6aafeaf945337 Anupam Mediratta 2026-10-27 20:59:19 +08:00
• 0e36d808b5 perf: decouple the Engram buffer size from max_num_batched_tokens zhaozhuoliang 2026-10-27 20:37:03 +08:00
• c871f558a3
  Merge 4e75db9c23bc87131c6f9b7a931f27df7b50dd7e into 553741557a57df44a46fb3ab72b6aafeaf945337 lhy027 2026-10-27 15:50:35 +08:00
• 4e75db9c23 fix: harden the profiling HTTP server against forged and out-of-tree requests w_lihongyan 2026-10-27 15:50:05 +08:00
• 8ed38b4602
  Merge ddd3fa468f9773b49660025c6812cff06070547d into 553741557a57df44a46fb3ab72b6aafeaf945337 Nora 2026-10-27 15:31:17 +08:00
• ab74c46389 neuron: dense prefill attention over small KV tables and prefill table bounds (both opt-in / dormant by default) whn09 2026-10-27 14:11:06 +08:00
• 5072084f1b
  Merge f03f3b0899206fbe34458494d86b880a417c7e05 into 553741557a57df44a46fb3ab72b6aafeaf945337 lhy027 2026-10-27 14:03:11 +08:00
• 48705fcf98
  Merge f0b869cf0af7f709b5556cd9cdd6e6de51a94378 into 553741557a57df44a46fb3ab72b6aafeaf945337 王召德 2026-10-27 12:27:58 +08:00
• c43ae36671
  Merge feca2310d136230715c0005232ed296ae19b0000 into 553741557a57df44a46fb3ab72b6aafeaf945337 nolanchic 2026-10-27 12:19:26 +08:00
• feca2310d1 fix(so): reject an unsupported schema_type at the request boundary Nolan 2026-10-27 12:18:35 +08:00
• cedd237772 neuron: fp8-activation linears take the NKI kernel (Engram projection); drop unused fp8 hidden copies before region splits; log weights without NKI scales whn09 2026-10-27 11:50:34 +08:00
• f03f3b0899 fix: raise when the engine rejects a request in LLM.generate w_lihongyan 2026-10-27 11:30:43 +08:00
• 729a1ec4e0
  Merge aad408280e0ec5f9b77e86c9502189dacb8eadcd into 553741557a57df44a46fb3ab72b6aafeaf945337 determine 2026-10-27 11:11:36 +08:00
• aad408280e test(eval): verify scoring recovers after benchmark import failure determine123 2026-10-27 11:11:02 +08:00
• c826e81b71 neuron: mHC collapse through torch by default (DEEPINFER_NEURON_HC_TORCH=collapse); fix Neuron bind_cpu core index whn09 2026-10-27 09:18:56 +08:00
• b9b109b036 neuron: decode only the SWA ring sub-blocks a step's windows reference (DEEPINFER_NEURON_SWA_SUBBLOCK=1) whn09 2026-10-27 07:45:17 +08:00
• 5a026d3aad Map media-free requests inline instead of through the multimodal worker (DEEPINFER_MULTI_MODAL_INLINE_TEXT=1) whn09 2026-10-27 04:56:47 +08:00
• 416bc1da1a
  Merge 5d389ab4599996b1f2a8651b06f54b2f15537b53 into 553741557a57df44a46fb3ab72b6aafeaf945337 Sharma Vikash Kaulesh 2026-10-26 23:58:03 +05:30
• 5d389ab459 Fix(llm): Normalize multi_modal_data dictionary and validate lengths in LLM.generate Sharma Vikash Kaulesh 2026-10-26 23:46:29 +05:30
• c4c50f1ea5 neuron: shared expert sliced over ranks, summed in the MoE all-reduce (DEEPINFER_NEURON_SHARED_SLICE=1) whn09 2026-10-27 01:31:41 +08:00
• b5fe1de89a
  Merge 06f2790c10c369dffd6f59e07ca88a5de1b6f504 into 553741557a57df44a46fb3ab72b6aafeaf945337 franz101 2026-10-26 10:07:20 -07:00
• d854ba4c92
  Merge 0e80d7c835ab1f022b1e3949414aebfa5ee1dc14 into 553741557a57df44a46fb3ab72b6aafeaf945337 akauncle 2026-10-26 10:06:07 -07:00
• 6c653df08c neuron: drafter forward as one region, compiled dflash metadata fill; verification + drafts regions on by default whn09 2026-10-27 00:14:32 +08:00
• 0a7d490cdb neuron: compiled decode verification behind DEEPINFER_NEURON_COMPILE_VERIFY (default off) whn09 2026-10-26 21:33:34 +08:00
• 949218eb29
  Merge cf1b959d985a160d639dc54311654a5dfe0257c1 into 553741557a57df44a46fb3ab72b6aafeaf945337 Tony Lin 2026-10-26 13:08:00 +00:00
• 66d97372a5
  Merge 625606db4eb8fafd8dc5243dd02c328bf38dafc5 into 553741557a57df44a46fb3ab72b6aafeaf945337 王鹤男 2026-10-26 12:50:53 +00:00
• edb69333e3
  Merge 4e679e904f5ffee7b410eecb5340f50c0e8a97b6 into 553741557a57df44a46fb3ab72b6aafeaf945337 杨润钊 2026-10-26 20:48:32 +08:00
• 4eb955b9e7
  Merge d54fe25993a44a7f09f5bbe6420fec6438c1ea79 into 553741557a57df44a46fb3ab72b6aafeaf945337 王鹤男 2026-10-26 12:42:56 +00:00
• e17e22c5de
  Merge a9baf6803c96284322e2ab95a11a91d269138b99 into 553741557a57df44a46fb3ab72b6aafeaf945337 tastelikefeet 2026-10-26 19:24:23 +08:00
• 625606db4e hicache L3: retry failed Store objects once before tripping the breaker whn09 2026-10-26 19:23:33 +08:00
• b18af0027f neuron: sampling flags per request batch, NUMA-local CPU binding, sync/op-site tracing whn09 2026-10-26 19:04:53 +08:00
• b53e8d3216
  Merge 4bda76dda5b9158380f2b89eecad68fa4d1614e4 into 553741557a57df44a46fb3ab72b6aafeaf945337 ccl125 2026-10-26 17:32:27 +08:00
• c8c943dd57
  Merge 777bd89ff8e1b05f7152f6bcd18e3da00e7767a6 into 553741557a57df44a46fb3ab72b6aafeaf945337 Yinfang Chen 2026-10-26 09:26:50 +00:00
• 8a4cd5c4b4
  Merge 4673ba93e0c7797794fb1afbe68efa9797edd231 into 553741557a57df44a46fb3ab72b6aafeaf945337 LiRenHao 2026-10-26 16:45:52 +08:00
• be67f5e135
  Merge c8f438c10a64ad660b198ae857345ba65866fbaf into 553741557a57df44a46fb3ab72b6aafeaf945337 u7-u7 2026-10-26 16:42:12 +08:00
• c8f438c10a fix(environ): reject unsupported DEEPINFER_ACCELERATOR values u7-u7 2026-10-26 16:40:18 +08:00
• f709d0dae3
  Merge 0d927ca8a045baf525c13671e7c35fd7ed2cc148 into 553741557a57df44a46fb3ab72b6aafeaf945337 frankliee 2026-10-26 16:40:39 +08:00
• 5354924073
  Merge 1692ef3f589c9cb83b7c469b3a67c0105fa41a3a into 553741557a57df44a46fb3ab72b6aafeaf945337 Yuvraj Singh Bhadoria 2026-10-26 16:34:42 +08:00
• ace20cec55
  Merge 5dbea6ef5474578b5b7825d1475f13bdc5d3a81b into 553741557a57df44a46fb3ab72b6aafeaf945337 Aling33 2026-10-26 16:33:18 +08:00
• 96ed49739d
  Merge 4908e1de941475d6d14a67d63972fd339998789d into 553741557a57df44a46fb3ab72b6aafeaf945337 Shuyun Yang 2026-10-26 16:31:34 +08:00
• 8136c32b97
  Merge 0f48b6130e4c8d4dd6857b2af0d18ff78c608c49 into 553741557a57df44a46fb3ab72b6aafeaf945337 ledd 2026-10-26 16:27:46 +08:00
• fe4746090e
  Merge c928039a2161f23fad220b95d6d721c17a06af16 into 553741557a57df44a46fb3ab72b6aafeaf945337 Jackson 2026-10-26 16:21:17 +08:00
• c928039a21 test(mtp_varlen): pin zero-rate slack in confidence-based seqlens_q allocation Zhiren Xiao 2026-10-26 16:18:59 +08:00
• 06cfc90597 docs(mtp_varlen): state zero-rate participation at the ranking site Zhiren Xiao 2026-10-26 16:17:33 +08:00
• 42614c144b
  Merge d98d239608b659eb6353b8d23e827dec380124e2 into 553741557a57df44a46fb3ab72b6aafeaf945337 Zhang 2026-10-26 15:53:11 +08:00
• a9636e81d1
  Merge ca0057bc7ee1d3d0553107d17d9700d110e6b59a into 553741557a57df44a46fb3ab72b6aafeaf945337 tonyh168 2026-10-26 15:42:05 +08:00
• ca0057bc7e chore: stop tracking the *.import-orig files, and make the ignore patterns work loopyt 2026-10-26 15:40:51 +08:00
• 5b080b1780 docs(h20): a runbook for the FlagGems configuration, plus the Dockerfile that reproduces it loopyt 2026-10-26 15:39:13 +08:00
• b6d8df7484 neuron: diagnostics for MoE collective skew (expert flag force, per-follower runtime profiling) whn09 2026-10-26 15:14:19 +08:00
• a10a54c80c
  Merge c78146bfb1cef7ec94326a51b84b1d968aef7d3f into 553741557a57df44a46fb3ab72b6aafeaf945337 Akash Murali 2026-10-26 07:12:18 +00:00
• d54fe25993 serve: Mooncake Transfer Engine KV transport for the prefill/decode split (--kv-transport mooncake) whn09 2026-10-23 10:31:23 +08:00
• 703f4727ff
  Merge 38200b9e1f69d1c27430a1ad38a14cfd7768386b into 553741557a57df44a46fb3ab72b6aafeaf945337 determine 2026-10-26 06:59:37 +00:00
• dc55e7454d
  Merge 58bab43ab3c8dfcdc05d3148aebcdb435327d4c3 into 553741557a57df44a46fb3ab72b6aafeaf945337 determine 2026-10-26 06:59:36 +00:00
• 575d857139
  Merge 647ffeaeb48942b888a056c72247a1a3ee6b633d into 553741557a57df44a46fb3ab72b6aafeaf945337 Zhang 2026-10-26 14:47:45 +08:00
• 1b8993b3ce
  Merge a6183402d36ca4b7899b588031659133750e90d8 into 553741557a57df44a46fb3ab72b6aafeaf945337 Akash Murali 2026-10-26 14:38:14 +08:00
• 4631f328bc
  Merge e6afeb54aac21c1865653700b88023d7a9ad22b3 into 553741557a57df44a46fb3ab72b6aafeaf945337 王召德 2026-10-26 14:37:49 +08:00
• f61f7c5346
  Merge 61c229917c2a68d3fe09c646ce0b19fffa730fe9 into 553741557a57df44a46fb3ab72b6aafeaf945337 JoyFul 2026-10-26 14:33:18 +08:00
• c9dba916a2
  Merge 2afdb3d2d8ed1fb3a83a0169485aa25d281196e8 into 553741557a57df44a46fb3ab72b6aafeaf945337 Akash Murali 2026-10-26 15:22:59 +09:00
• a3a6d7e186
  Merge 479d7f787358baf1cb0c0704733e099210a382a2 into 553741557a57df44a46fb3ab72b6aafeaf945337 Yijun Zhao 2026-10-26 13:48:15 +08:00
• 2a03457269
  Merge 0609effbccd167cf2def91ceb853b00b72ccf964 into 553741557a57df44a46fb3ab72b6aafeaf945337 hx 2026-10-26 05:39:46 +00:00
• 0820f5cbc6
  Merge 24722800a2fae60bee944eedd1c35bd6fc3e11e7 into 553741557a57df44a46fb3ab72b6aafeaf945337 Jiejing 2026-10-26 05:37:58 +00:00
• cb721b5c2a
  Merge 538caeef2ef83ab2532788fd9aca76c541afef6c into 553741557a57df44a46fb3ab72b6aafeaf945337 Eddy Liu 2026-10-26 05:35:06 +00:00
• 5be8ab8b3e
  Merge 8fdc28005071d2319e9455eeba2b875536a64c42 into 553741557a57df44a46fb3ab72b6aafeaf945337 morluto 2026-10-26 13:20:04 +08:00
• dcf00aca7c feat(h20): enable FlagGems end to end, from an allow-list of ops proven equal to PyTorch loopyt 2026-10-26 12:53:47 +08:00
• 24722800a2 Merge remote-tracking branch 'fork/openai-server' into pd-split-fix Jiejing 2026-10-26 04:44:25 +00:00
• 7416bb2e4d
  Merge f6388940003e9c962337bcd1f8d0ba693d977f86 into 553741557a57df44a46fb3ab72b6aafeaf945337 Jiejing 2026-10-25 21:43:55 -07:00
• f638894000 serve: default --prefill-any-interval to 30 (throughput over TTFT) Jiejing 2026-10-26 04:43:27 +00:00