5 Commits

Author SHA1 Message Date
Frank Barchard
eeb06a6451 [libyuv] Optimize ARGBRotate for Intel SSE2, AVX2, and AVX-512
Optimize 32-bit ARGBRotate (0, 90, 180, 270) across Intel x86/x64
architectures (SSE2, AVX2, AVX512BW), with matching kernels for ARM
(NEON, SVE2, SME), WASM, LoongArch (LSX, LASX), and RISC-V (RVV):

- Replace column-by-column ScaleARGBRowDownEven gather loops in
  ARGBTranspose (90/270) with full-plane horizontal tile transposes:
  - AVX512BW (TransposeWxH_32_AVX512BW / TransposeNx4_32_AVX512BW):
    Transpose 16x16 blocks across zmm0..zmm31 using 4-stage
    vpunpckldq/hdq, vpunpcklqdq/hqdq, and vshufi32x4 butterflies with
    k-masked right and bottom tails, eliminating Any wrappers and scalar
    C edge fallbacks.
  - AVX2 (TransposeWxH_32_AVX2 / TransposeNx4_32_AVX2):
    Transpose 8x8 blocks across ymm0..ymm15 using vpunpck* and
    vperm2i128 with vpmaskmovd masked tails.
  - SSE2 (Transpose4x4_32_SSE2 / Transpose4x4_32_Any_SSE2):
    Transpose 4x4 blocks in xmm registers.
- Implement ARGBRotate180 directly via ARGBMirror with negative height,
  avoiding intermediate row buffer allocations and extra copies.
- Implement ARGBRotate0 via ARGBCopy.

Speedup vs Before (1275x1650, US Letter 150 DPI, 1000 iterations):
CPU                  | 0 deg   | 90 deg  | 180 deg | 270 deg
---------------------|---------|---------|---------|--------
Intel Ice Lake       | 1.02x   | 1.34x   | 1.18x   | 1.33x
Intel Skylake-X      | 1.02x   | 1.20x   | 1.07x   | 1.13x
Intel Emerald Rapids | 1.01x   | 1.07x   | 1.00x   | 1.07x
Intel AVX-512 vs C   | 1.00x   | 2.67x   | 2.41x   | 2.63x

R=rrwinterton@gmail.com

Bug: b/42280915
Test: LIBYUV_WIDTH=1275 LIBYUV_HEIGHT=1650 LIBYUV_REPEAT=1000 ./out/Release/libyuv_unittest --gtest_filter=*ARGBRotate*
Change-Id: Ib308a225c17e46eab083a9da4033598fcf0c303c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8538986
Reviewed-by: richard winterton <rrwinterton@gmail.com>
2026-10-08 16:20:46 -07:00
Frank Barchard
20b0a91bc1 Transpose 16x16 for AVX2
- TransposeWx16_AVX2 and TransposeUVWx16_AVX2 using 2 lanes of 8 ymm
  registers, with Any for tail.
- Move TransposeUVWx16_C to rotate_common.cc and pass the C function to
  TUVANY, so the LSX 16 row Any version transposes all 16 rows.

I420Rotate90_Opt (LIBYUV_WIDTH=1279 LIBYUV_HEIGHT=719 LIBYUV_REPEAT=1000):
CPU                      SSSE3   AVX2  AVX512  AVX2 x   512 x
Intel Haswell             1812    916     945   1.98x   1.92x
AMD Milan (Zen 3)          177    118     118   1.50x   1.50x
Intel Granite Rapids       245    186     186   1.32x   1.32x
Intel Emerald Rapids       246    193     176   1.27x   1.40x
Intel Sapphire Rapids      278    233     225   1.19x   1.24x
Intel Ice Lake             308    213     221   1.45x   1.39x
Intel Skylake-X            365    267     246   1.37x   1.48x
AMD Turin (Zen 5)          106     75      75   1.41x   1.41x
AMD Genoa (Zen 4)          149     98      93   1.52x   1.60x
AMD Venice (Zen 6)         106     74      73   1.43x   1.45x

NV12Rotate90_Opt (LIBYUV_WIDTH=1279 LIBYUV_HEIGHT=719 LIBYUV_REPEAT=1000):
CPU                      SSSE3   AVX2  AVX512  AVX2 x   512 x
Intel Haswell             1771    963     935   1.84x   1.89x
AMD Milan (Zen 3)          173    118     118   1.47x   1.47x
Intel Granite Rapids       275    189     186   1.46x   1.48x
Intel Emerald Rapids       256    188     179   1.36x   1.43x
Intel Sapphire Rapids      286    232     218   1.23x   1.31x
Intel Ice Lake             309    208     218   1.49x   1.42x
Intel Skylake-X            380    269     251   1.41x   1.51x
AMD Turin (Zen 5)          112     74      73   1.51x   1.53x
AMD Genoa (Zen 4)          146     97      93   1.51x   1.57x
AMD Venice (Zen 6)         114     73      72   1.56x   1.58x

Bug: None
Change-Id: Ia89219e3336f39a817aa662e1452824d781664a3
TAG=agy
CONV=bea53b37-b2bf-46fa-9e44-a708b438fabc
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8489456
Reviewed-by: richard winterton <rrwinterton@gmail.com>
2026-10-01 17:31:33 -07:00
George Steed
4f7fd808b7 [AArch64] Use full vectors in TransposeWx{8 => 16}_NEON
The existing Neon code only makes use of 64-bit vectors throughout which
limits the performance on larger cores. To avoid this, swap the Neon
code from a Wx8 implementation to a Wx16 implementation and process
blocks of 16 full vectors at a time.

The original code also handled widths that were not exact multiples of
16, however this should already be handled by the "any" kernel so it is
removed.

Finally, avoid duplicating the TransposeWx16_C fallback kernel
definition in all architectures that need it, and just put it once in
rotate_common.cc instead.

Observed speedups for TransposePlane across a range of
micro-architectures:

 Cortex-A53: -40.0%
 Cortex-A55: -20.7%
 Cortex-A57: -43.9%
Cortex-A510: -43.5%
Cortex-A520: -43.9%
Cortex-A720: -31.1%
  Cortex-X2: -38.3%
  Cortex-X4: -43.6%

Change-Id: Ic7c4d5f24eb27091d743ddc00cd95ef178b6984e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/5545459
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
2024-05-21 07:46:42 +00:00
Frank Barchard
804980bbab DetilePlane and unittest for NEON
Bug: libyuv:915, b/215425056
Change-Id: Iccab1ed3f6d385f02895d44faa94d198ad79d693
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/3424820
Reviewed-by: Justin Green <greenjustin@google.com>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
2022-01-31 20:05:55 +00:00
Hao Chen
f8e2da48ae Add optimization functions in rotate_lsx.cc file.
Optimize two functions in source/rotate_lsx.cc file.
All test cases passed on loongarch platform.

Bug: libyuv:913
Change-Id: Idf670a1bc078f6284a499a292e0cb795f5b603b4
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/3351468
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
2022-01-21 01:34:38 +00:00