Frank Barchard
|
2404f9730e
|
[libyuv] Rewrite LoongArch LSX/LASX kernels in compact inline asm
Reduce loop unrolling across LoongArch LSX (128-bit) and LASX (256-bit)
compare, row, rotate, and scale kernels to match SSE2/SSSE3 and AVX2
(1-2 vectors per iteration), rewrite intrinsics/macro implementations
into GCC/Clang inline assembly, update all _Any wrapper masks and
IS_ALIGNED dispatch checks, and port all remaining LSX Convert (+19) and
Rotate (+2) kernels to LASX for full parity:
ISA | Compare | Convert | Rotate | Scale
-----------------|---------|---------|---------|--------
MIPS/Loong (Any) | 1 | 68 | 4 | 20
LSX | 1 | 68 | 4 | 20
LASX | 1 | 68 | 4 | 0
Removes 1,935 macro/helper lines from loongson_intrinsics.h (-99.3%) and
reduces total LoongArch compiled .text size by 9,799 bytes (-25.1%
overall, -37.4% row_lsx.cc, -52.3% scale_lsx.cc, -31.2% rotate_lsx.cc,
-4.9% row_lasx.cc while adding +25 LASX functions):
File | Before | After | Change
-------------------|----------|----------|--------
scale_lsx.cc.o | 6,055 B | 2,888 B | -52.3%
row_lsx.cc.o | 17,252 B | 10,792 B | -37.4%
rotate_lsx.cc.o | 1,488 B | 1,024 B | -31.2%
row_lasx.cc.o | 13,740 B | 13,072 B | -4.9%
rotate_lasx.cc.o | 544 B | 1,504 B | +176.5%
TOTAL .text | 39,079 B | 29,280 B | -25.1%
R=wtc@google.com, yuanhecai@loongson.cn
Test: qemu-loongarch64 -cpu la464 libyuv_unittest (LASX and LIBYUV_DISABLE_LASX=1)
Change-Id: I11c1143ad2aad0399f10b2c3fc9bc4c02c740c59
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8564659
Reviewed-by: richard winterton <rrwinterton@gmail.com>
|
2026-10-16 17:25:23 -07:00 |
|
Frank Barchard
|
16af162cf9
|
[libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes
Implement SIMD 16-bit plane rotation (TransposePlane_16,
RotatePlane90_16, RotatePlane180_16, RotatePlane270_16) used by
I010Rotate, I210Rotate, and I410Rotate, keeping tile sizes compact
with full-register writes and zero cross-lane shuffles:
- AVX2: 8x16 tile in ymm0..ymm15 with _Any tail (low-end x86 SIMD)
- AVX-512BW: 8x32 tile in zmm0..zmm15 with k1/k2 right/bottom masks
- NEON (ARM64/ARM32): 8x16 tile in v0..v23 / q0..q11 with _Any tail
- SME: 8xVL tile using ZA16 horizontal/vertical array slices
- RVV: 8xVL tile using widening unpacks with hardware vl tail
- WASM SIMD: 8x8 tile in v128 with _Any tail
Also update 32-bit ARGBRotate x86 transposes to compact vertical
in-lane tiles with full-register writes and zero cross-lane shuffles:
- TransposeWxH_32_AVX2: 4x8 tile in ymm0..ymm7 with vpmaskmovd tails
- TransposeWxH_32_AVX512BW: 4x16 tile in zmm0..zmm7 with k1/k2 tails
Add RotatePlane16_SplitMergeUV_Opt and RotatePlane16_SplitMergeUV_Odd
unit tests comparing 16-bit rotation against SplitUVPlane +
RotatePlane + MergeUVPlane across 0/90/180/270 degrees.
Performance on 1275x1650 (1000 iterations, before -> after):
CPU 90 degree 180 degree
Coral RVV (Verilator) 7306 -> 1415c (5.16x) 2274 -> 719c (3.16x)
AMD Rome (Zen 2) 821 -> 215ms (3.82x) 259 -> 181ms (1.44x)
AMD Genoa (Zen 4) 483 -> 129ms (3.73x) 149 -> 111ms (1.34x)
AMD Turin (Zen 5) 321 -> 101ms (3.18x) 109 -> 78ms (1.40x)
AMD Milan (Zen 3) 505 -> 189ms (2.67x) 199 -> 118ms (1.69x)
Intel Skylake-X 1076 -> 447ms (2.41x) 353 -> 317ms (1.11x)
WASM SIMD (d8) 415 -> 203ms (2.04x) 370 -> 106ms (3.49x)
Intel Emerald Rapids 735 -> 422ms (1.74x) 238 -> 233ms (1.02x)
Intel Haswell 1120 -> 648ms (1.73x) 438 -> 393ms (1.11x)
Intel Ice Lake 774 -> 474ms (1.63x) 279 -> 290ms (0.96x)
ARM Neoverse-N1 864 -> 544ms (1.59x) 404 -> 266ms (1.52x)
Ampere Siryn 862 -> 576ms (1.50x) 568 -> 329ms (1.72x)
Google Axion (Maple) 554 -> 379ms (1.46x) 199 -> 142ms (1.41x)
Intel Sapphire Rapids 1126 -> 772ms (1.46x) 301 -> 288ms (1.04x)
Intel Granite Rapids 730 -> 565ms (1.29x) 280 -> 280ms (1.00x)
NVIDIA Grace 462 -> 365ms (1.26x) 192 -> 126ms (1.52x)
ARM Cortex-A53 28290 -> 24386ms (1.16x) 7178 -> 5075ms (1.41x)
Google Axion 2 (Tamar) 523 -> 453ms (1.15x) 217 -> 112ms (1.93x)
Test: libyuv_unittest --gtest_filter=*Rotate*
Bug: b/42280942
Change-Id: I91ba6cac22185b5d294fcb98344fdb8a23cc94ff
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8545294
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
|
2026-10-12 10:57:34 -07:00 |
|
Frank Barchard
|
eeb06a6451
|
[libyuv] Optimize ARGBRotate for Intel SSE2, AVX2, and AVX-512
Optimize 32-bit ARGBRotate (0, 90, 180, 270) across Intel x86/x64
architectures (SSE2, AVX2, AVX512BW), with matching kernels for ARM
(NEON, SVE2, SME), WASM, LoongArch (LSX, LASX), and RISC-V (RVV):
- Replace column-by-column ScaleARGBRowDownEven gather loops in
ARGBTranspose (90/270) with full-plane horizontal tile transposes:
- AVX512BW (TransposeWxH_32_AVX512BW / TransposeNx4_32_AVX512BW):
Transpose 16x16 blocks across zmm0..zmm31 using 4-stage
vpunpckldq/hdq, vpunpcklqdq/hqdq, and vshufi32x4 butterflies with
k-masked right and bottom tails, eliminating Any wrappers and scalar
C edge fallbacks.
- AVX2 (TransposeWxH_32_AVX2 / TransposeNx4_32_AVX2):
Transpose 8x8 blocks across ymm0..ymm15 using vpunpck* and
vperm2i128 with vpmaskmovd masked tails.
- SSE2 (Transpose4x4_32_SSE2 / Transpose4x4_32_Any_SSE2):
Transpose 4x4 blocks in xmm registers.
- Implement ARGBRotate180 directly via ARGBMirror with negative height,
avoiding intermediate row buffer allocations and extra copies.
- Implement ARGBRotate0 via ARGBCopy.
Speedup vs Before (1275x1650, US Letter 150 DPI, 1000 iterations):
CPU | 0 deg | 90 deg | 180 deg | 270 deg
---------------------|---------|---------|---------|--------
Intel Ice Lake | 1.02x | 1.34x | 1.18x | 1.33x
Intel Skylake-X | 1.02x | 1.20x | 1.07x | 1.13x
Intel Emerald Rapids | 1.01x | 1.07x | 1.00x | 1.07x
Intel AVX-512 vs C | 1.00x | 2.67x | 2.41x | 2.63x
R=rrwinterton@gmail.com
Bug: b/42280915
Test: LIBYUV_WIDTH=1275 LIBYUV_HEIGHT=1650 LIBYUV_REPEAT=1000 ./out/Release/libyuv_unittest --gtest_filter=*ARGBRotate*
Change-Id: Ib308a225c17e46eab083a9da4033598fcf0c303c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8538986
Reviewed-by: richard winterton <rrwinterton@gmail.com>
|
2026-10-08 16:20:46 -07:00 |
|