AArch64: Avoid lane-indexed LD4 dependency in 32 bit transpose

The original main loop builds v0-v3 with four lane-indexed LD4 load
instructions. Since the first LD4 only updates lane 0, it retains a
false dependency on the registers produced by the previous loop
iteration, leading to a large performance penalty on larger out-of-order
micro-architectures.

Replace the first LD4 with two scalar-register LDPs to zero the top part
of the four registers, breaking the loop-carried dependency.

Performance versus the existing Neon code, higher is better:

 Cortex-A55: 0.993x
Cortex-A510: 1.072x
Cortex-A720: 0.999x
  Cortex-X4: 1.052x
    C1-Nano: 2.093x
     C1-Pro: 2.011x
   C1-Ultra: 2.128x

Change-Id: Ib5d71e8150d7b11048ddf0fb91e107f9c9323a3f
This commit is contained in:
George Steed 2026-09-27 21:31:52 +01:00
parent f249eaa72f
commit dc630852d7

View File

@ -253,7 +253,9 @@ void TransposeWx4_32_NEON(const uint8_t* src,
// Main loop transpose 4x4. Read a column, write a row.
"1: \n"
"ld4 {v0.s, v1.s, v2.s, v3.s}[0], [%0], %9 \n"
"ldp s0, s1, [%0] \n"
"ldp s2, s3, [%0, #8] \n"
"add %0, %0, %9 \n"
"ld4 {v0.s, v1.s, v2.s, v3.s}[1], [%1], %9 \n"
"ld4 {v0.s, v1.s, v2.s, v3.s}[2], [%2], %9 \n"
"ld4 {v0.s, v1.s, v2.s, v3.s}[3], [%3], %9 \n"