Add AVX-512 VNNI (VPDPBUSD / _mm512_dpbusd_epi32) row kernels for 64-bit
x86 targets alongside existing AVX-512BW implementations in both
row_gcc.cc (inline assembly) and row_win.cc (C++ intrinsics):
- ARGBToYMatrixRow_AVX512VNNI and ARGBToYMatrixRow_Any_AVX512VNNI:
Pre-computes the 32-bit signed-pixel bias once per row and replaces
vpmaddubsw + vpmaddwd + vpaddw with a single vpdpbusd per 16 pixels.
- ARGBToUV444MatrixRow_AVX512VNNI and
ARGBToUV444MatrixRow_Any_AVX512VNNI: Loads 64 ARGB pixels once instead
of twice and computes 32-bit U and V dot products directly via
vpdpbusd with signed UV matrix coefficients.
- ARGBToUVMatrixRow_AVX512VNNI: Computes 2x2 box-filtered ARGB to U and
V conversion using vpdpbusd with k-masked tail handling for arbitrary
widths.
- ARGBGrayRow_AVX512VNNI: Computes BT.601 grayscale luminance using a
single vpdpbusd per 16 pixels, broadcasts Y to RGB via vpshufb, and
preserves alpha via vpternlogd with k-masked tail handling.
- Wire runtime dispatch via TestCpuFlag(kCpuHasAVX512VNNI) across
convert.cc, convert_from_argb.cc, and planar_functions.cc.
Add AVX512VNNIVBMI variants of the Y, UV444 and UV kernels. These use
both AVX-512 VNNI (vpdpbusd) and AVX-512 VBMI (vpermt2b) and are only
selected when the CPU has both features; there are no VBMI versions
without VNNI. The AVX512VNNI kernels remain for CPUs with VNNI but no
VBMI (Cascade Lake, Cooper Lake).
- The 32-bit sums are always in [0, 65535], so Y, U and V are byte 1 of
each int32. Two vpermt2b gather 64 results from 4 accumulators,
replacing vpackusdw/vpackssdw + vpsrlw + vpackuswb + vpermd.
- U and V accumulators start at ~kAddUV so they hold ~(kAddUV - dot).
The inverse is folded into the vpternlogd that merges the two
256-bit halves. kAddUV is loaded from ArgbConstants rather than
hard coded as 0x8000.
- Y folds the source loads into vpxorq.
- row_win.cc hides the vpermt2b indices from clang, which otherwise
constant folds them into a longer vpshufb + vpor + vpermd sequence.
Performance (1280x720, 5000 iterations, AVX512BW -> AVX512VNNI):
uArch / Benchmark AVX512BW AVX512VNNI Speedup
--------------------------- -------- ---------- -------
Intel Ice Lake ARGBToI420 941 ms 862 ms +9.2%
Intel Ice Lake ARGBToI422 1059 ms 947 ms +11.8%
Intel Ice Lake ARGBToI444 1140 ms 1038 ms +9.8%
Intel Ice Lake ARGBGray 357 ms 273 ms +30.8%
Intel Sapphire Rapids GrayTo 317 ms 218 ms +45.4%
Intel Emerald Rapids GrayTo 310 ms 213 ms +45.5%
Intel Granite Rapids GrayTo 349 ms 241 ms +44.8%
AMD Zen 4 ARGBToI420 648 ms 517 ms +25.3%
AMD Zen 4 ARGBToI444 695 ms 526 ms +32.1%
AMD Zen 4 ARGBGray 403 ms 227 ms +77.5%
AMD Zen 5 ARGBToI422 460 ms 403 ms +14.1%
AMD Zen 5 ARGBToI444 382 ms 321 ms +19.0%
Performance (row_gcc, AVX512VNNI -> AVX512VNNIVBMI):
uArch / Benchmark AVX512VNNI VNNIVBMI Speedup
---------------------------- ---------- -------- -------
AMD Zen 6 ARGBToI420 1280x720 327 ms 284 ms +15.1%
AMD Zen 6 ARGBToI422 1280x720 372 ms 330 ms +12.7%
AMD Zen 6 ARGBToI444 1280x720 357 ms 334 ms +6.9%
AMD Zen 6 ARGBToI400 256x24 94 ms 76 ms +23.7%
AMD Zen 5 ARGBToI422 1280x720 406 ms 350 ms +16.0%
AMD Zen 5 ARGBToI444 1280x720 330 ms 301 ms +9.6%
AMD Zen 5 ARGBToI400 256x24 100 ms 90 ms +11.1%
AMD Zen 4 ARGBToI420 1280x720 631 ms 540 ms +16.9%
AMD Zen 4 ARGBToI422 1280x720 993 ms 825 ms +20.4%
AMD Zen 4 ARGBToI400 256x24 245 ms 218 ms +12.4%
Intel GNR ARGBToI422 256x24 654 ms 616 ms +6.2%
Intel SPR ARGBToI422 256x24 653 ms 613 ms +6.5%
Intel GNR, EMR, SPR and Ice Lake are within +/-3% at 1280x720.
R=rrwinterton@gmail.com
Bug: none
Test: out/Release/libyuv_unittest --gtest_filter=*ARGBToI4*:*ARGBGray*
Test: LIBYUV_DISABLE_AVX512VBMI=1 out/Release/libyuv_unittest
Test: sde64 -gnr / -clx with is_msan=true (row_win.cc)
Change-Id: Ia7a37b39dd2948fe4055a8a7694dd1ff8ac3e55c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8647811
Reviewed-by: richard winterton <rrwinterton@gmail.com>
1. Fix multi-bit TestCpuFlag(A | B) checks in AYUVToNV12, AYUVToNV21,
RAWToRGBA, NV12ToRGB24Matrix, NV21ToRGB24Matrix, MergeRGBPlane, and
MergeXR30Plane to compare == (A | B) so AVX-512 VBMI/BW kernels are
not dispatched when AVX512VBMI or AVX512VL is absent.
2. Fix 2-byte SVE2 heap buffer overflow on odd widths in
I422ToARGB1555Row_SVE_SC and I422ToARGB4444Row_SVE_SC using
trn1/trn2 + st2b under .b predicates.
3. Fix ARGBPolynomialRow_SSE2 to use packssdw before packuswb, and add
missing vzeroupper to SplitRGBRow_AVX2.
4. Fix CalcFrameSsim to return 0.0 instead of 0.0 / 0 (NaN) when
width <= 8 || height <= 8 so I420Ssim stays finite on small frames.
5. Fix negative-height vertical inversion in ConvertFromI420,
SplitARGBPlane (when dst_a is NULL), I422Rotate, I010Rotate, and
I210Rotate.
6. Fix horizontal flip (negative x/dx) in ScaleFilterCols_NEON (signed
sbfx #16, #16 offset extraction) and ScaleAddCols1_16_C (shift
x >>= 16 before indexing src_ptr).
7. Fix ConvertToARGB odd crop_x chroma alignment in NV12/NV21, src_u
offset in I420/YV12/J420/H420/U420, and FOURCC_U422 calling
U422ToARGB instead of H422ToARGB.
8. Remove dead 14-instruction block in ScaleRowUp2_Bilinear_16_SSE2.
9. Fix UVScale_16 for src_width == 1 (P010ToP410/P210ToP410), enable
and fix column order in TransposeWx4_32_SVE2, fix ByteToFloatRow
NEON Any BPP (3 -> 4), fix AB64ToARGBRow_Any_*, ScaleColsUp2_SSE2,
ScaleUVRowDown2Box_SME, and UYVYToARGBRow_SME macro guards, fix SVE2
loop tail branches (b.lt/b.ge), and remove duplicate HAS_* macros.
10. Fix ConvertFP16ToFP32Column_NEON loop condition and unsigned b.lo
underflow on widths < 8 or not multiples of 8.
11. Fix UYVYToNV12 IS_ALIGNED(awidth, ...) checks and WASM SIMD
IS_ALIGNED(width, 16) checks in I420ToRGB24Matrix /
I422ToRGB24Matrix.
12. Wire SplitUVRow_SVE2, SplitUVRow_LSX, and SplitUVRow_LASX in
NV24ToARGB.
13. Fix MJpegDecoder heap buffer overflows in LoadFrame (reallocate
databuf_[i] when scanlines_size changes), DecodeToBuffers, and
DecodeToCallback (clamp scanlines_to_copy to lines_left and align
skip to max_v_samp_factor).
14. Replace width & (vl - 1) tail calculations with post-loop remainder
across 18 SVE2 functions in row_sve.h for non-power-of-2 SVE VL,
and remove duplicate label 2: in UYVYToARGBRow_SVE_SC.
15. Handle width & 15 tail pixels in DetileToYUY2_C and validate
IS_POWEROFTWO(tile_height) in DetileSplitUVPlane and DetileToYUY2.
16. Avoid signed 32-bit overflow in DivideRow_16_C and C16TO8, fix
ClampFloatToZero_SSE2 pointer increment (lea 0x4) and loop counter
(sub $0x1), and initialize xmm3 in ARGBCopyYToAlphaRow_SSE2.
17. Check max leaf 7 subleaf and max extended CPUID leaf in GetCpuFlags
before querying subleaf 7:1 and extended leaf 0x80000021.
18. Free plane_u on error in I422ToNV21, and fix 4x/2x scratch row
over-allocation in ScaleARGBBilinearDown and ScaleUVBilinearDown.
19. Add ARGBSetRow_RVV and ARGBShuffleRow_RVV in row_rvv.cc.
20. Fix NEON mask (15 -> 7) in ScaleUVRowUp2_Linear_Any_NEON /
_16_Any_NEON, use SIMD InterpolateRow + SplitUVRow in NV16ToI420,
wire SSSE3/SSE2 in ARGBToI400Matrix / ARGBToYUY2Matrix /
ARGBToUYVYMatrix and RGB24Mirror, and split HalfFloatRow multiply
into (src * scale) * 2^-112 to avoid 50x Intel denormal stalls.
Bug: b/574714081
Test: libyuv_unittest
Change-Id: Iec3ae5c58d52bee3ef13cf9b765ac84b733aa576
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8601931
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
1275x1650 (US Letter 150 DPI, 1000 iterations) speedup:
CPU ARGBExtractAlpha ARGBCopyYToAlpha ARGBMirror RGB24Mirror
-----------------------------------------------------------------------
Intel Granite Rapids 1.00x 1.00x 1.00x 1.04x
Intel Emerald Rapids 1.01x 0.97x 1.03x 1.07x
Intel Sapphire Rapids 0.97x 1.01x 1.08x 1.26x
Intel Ice Lake 0.99x 1.02x 0.97x 1.08x
Intel Skylake-X 1.04x 1.03x 1.01x NA
Intel Haswell NA NA NA NA
AMD Turin (Zen 5) 1.01x 1.10x 1.25x 1.31x
AMD Genoa (Zen 4) 0.97x 1.04x 1.09x 1.42x
AMD Milan (Zen 3) NA NA NA NA
AMD Rome (Zen 2) NA NA NA NA
NVIDIA Grace (V2) 1.57x 2.51x NA NA
Google Axion 2 1.12x 3.17x NA NA
Ampere Siryn 1.06x 1.70x NA NA
ARM Neoverse-N1 1.49x 2.01x NA NA
Google Axion (Maple) 1.47x 2.03x NA NA
ARM Cortex-A53 1.06x 1.69x NA NA
Add AVX512BW ARGBExtractAlphaRow, ARGBCopyYToAlphaRow and
ARGBMirrorRow, and AVX512VBMI RGB24MirrorRow, in row_gcc.cc and as
intrinsics in row_win.cc. They are x64 only. The remainder is done
with a k-mask (bzhi) zero-masked load and a masked store, so they
handle any width without an Any wrapper and never access memory
outside the row.
RGB24MirrorRow_AVX512VBMI loads and stores 3 full zmm vectors per
64 pixels using 3 vpermi2b. The one output byte that comes from the
third input vector is merged with a masked vpbroadcastb. The 1 to 63
pixel tail rebases the same tables by 192 - 3 * r; indices >= 128 are
taken from the third vector with vpmovb2m and a masked vpermb.
Add ARGBCopyYToAlphaRow_NEON for Arm 32 bit (vld4 + vst4) and
AArch64 (zip + sli), and rewrite AArch64 ARGBExtractAlphaRow_NEON
with ldp + uzp2. Both prefetch source and destination, which is
2.5x faster on Neoverse-N1 and 1.2x to 1.5x faster on Cortex-A53
for 1280x720. Add WASM SIMD ARGBExtractAlphaRow and
ARGBCopyYToAlphaRow, using a swizzle table for CopyYToAlpha. Rework
the RVV versions to use unit stride loads with vnsrl and vzext.vf4.
R=rrwinterton@gmail.com
Bug: None
Change-Id: I436f27d70a3db0a2b176c91eb789a9491133cfcb
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8523531
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Implement RAWToRGB24Row_WASMSIMD and RAWToRGB24Row_Any_WASMSIMD in
source/row_wasm.cc for RGB24ToRAW and RAWToRGB24 conversions:
- Process 16 pixels (48 bytes) per loop iteration using three 128-bit
loads, three byte-wise shuffles, and four 2-way blends.
- Use wasm_i8x16_relaxed_swizzle when __wasm_relaxed_simd__ is enabled
(compiling to a single vpshufb on x86 without bounds-checking
overhead) and fall back to wasm_i8x16_swizzle for baseline SIMD128.
- Add kCpuHasWASM (0x100) and kCpuHasWASMSIMD (0x101) CPU feature
flags and enable -msimd128 in BUILD.bazel, BUILD.gn, and
CMakeLists.txt for WebAssembly targets.
Performance (RAWToRGB24_Opt, 1280x720, V8 d8):
CPU WASM C SIMD128 Relaxed SIMD Spd Rlx Spd
-------------------------------------------------------------------
AMD Zen 4 595.0 us 137.0 us 84.0 us 4.34x 7.08x
AMD Zen 3 708.0 us 148.0 us 107.0 us 4.78x 6.62x
AMD Zen 5 375.0 us 107.0 us 67.0 us 3.50x 5.60x
ARM Neoverse-V2 638.0 us 118.0 us 116.0 us 5.41x 5.50x
ARM Neoverse-N1 1046.0 us 294.0 us 294.0 us 3.56x 3.56x
Intel Ice Lake 707.0 us 209.0 us 220.0 us 3.38x 3.21x
Intel Haswell 1110.0 us 344.0 us 360.0 us 3.23x 3.08x
Intel Emerald Rapids 519.0 us 207.0 us 208.0 us 2.51x 2.50x
Intel Sapphire Rapids 551.0 us 220.0 us 255.0 us 2.50x 2.16x
ARM Cortex-A53 14440.0 us 8660.0us 6920.0us 1.67x 2.09x
Intel Granite Rapids 453.0 us 225.0 us 221.0 us 2.01x 2.05x
Bug: 42271072
Test: libyuv_test --gunit_filter=*RAWToRGB24*:*RGB24ToRAW*:*Cpu*
Change-Id: Id902668263a2ab70aa300eeeea18851490c917b5
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8515581
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Enable LIBYUV_ENABLE_ROWWIN instead of LIBYUV_DISABLE_X86 under
MemorySanitizer so that C++ SIMD intrinsics in row_win.cc and their
corresponding _Any wrappers in row_any.cc are compiled and tested
under MSan.
Rename MEMSET to MSAN_MEMSET in row_any.cc and scale_any.cc, and
initialize vout buffers across all ANY* macros in row_any.cc. Use
calloc for align_buffer_64 and align_buffer_page_end under
LIBYUV_MSAN, and zero-initialize stack pixel buffers in color_test.cc.
Test: out/msan/libyuv_unittest
Bug: None
Change-Id: I0b9fc69af159531699bb2b2db8b48f39831289ee
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8456982
Reviewed-by: richard winterton <rrwinterton@gmail.com>
I422ToRGB24Matrix (299x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Skylake-X 275 ms 213 ms -22.5% (1.29x)
Intel Ice Lake 225 ms 177 ms -21.3% (1.27x)
Intel Sapphire Rapids 181 ms 144 ms -20.4% (1.26x)
Intel Emerald Rapids 167 ms 136 ms -18.6% (1.23x)
Intel Granite Rapids 170 ms 140 ms -17.6% (1.21x)
AMD Zen 4 (Genoa) 192 ms 136 ms -29.2% (1.41x)
AMD Zen 5 (Turin) 102 ms 68 ms -33.3% (1.50x)
AMD Zen 6 (Venice) 102 ms 60 ms -41.2% (1.70x)
====================================================================
I422ToRGB24Matrix (298x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Skylake-X 271 ms 184 ms -32.1% (1.47x)
Intel Ice Lake 235 ms 152 ms -35.3% (1.55x)
Intel Sapphire Rapids 199 ms 128 ms -35.7% (1.55x)
Intel Emerald Rapids 170 ms 121 ms -28.8% (1.40x)
Intel Granite Rapids 177 ms 125 ms -29.4% (1.42x)
AMD Zen 4 (Genoa) 185 ms 119 ms -35.7% (1.55x)
AMD Zen 5 (Turin) 102 ms 59 ms -42.2% (1.73x)
AMD Zen 6 (Venice) 100 ms 51 ms -49.0% (1.96x)
====================================================================
Bug: None
Change-Id: Id75e3d1df6d26bc28a252672816a7c438cb664a4
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8452502
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
I422ToRGB24Matrix (299x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Ice Lake 220 ms 181 ms -17.7% (1.22x)
Intel Sapphire Rapids 181 ms 145 ms -19.9% (1.25x)
Intel Emerald Rapids 169 ms 148 ms -12.4% (1.14x)
Intel Granite Rapids 191 ms 141 ms -26.2% (1.35x)
AMD Zen 4 (Genoa) 171 ms 136 ms -20.5% (1.26x)
AMD Zen 5 (Turin) 94 ms 72 ms -23.4% (1.31x)
AMD Zen 6 (Venice) 95 ms 66 ms -30.5% (1.44x)
====================================================================
Bug: None
Change-Id: I4cd657410cc5253b5e8fd2caa78d17e2b3812393
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8445804
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- Also remove redundant neon check for I8MM kernels
======================================================================
CPU Before After Speedup
----------------------------------------------------------------------
Intel Granite Rapids (AVX10.1) 251 ms 141 ms 1.78x (+43.8%)
Intel Emerald Rapids (AVX512) 199 ms 108 ms 1.84x (+45.7%)
Intel Sapphire Rapids (AVX512) 230 ms 112 ms 2.05x (+51.3%)
AMD Turin Zen 5 (AVX512) 156 ms 99 ms 1.58x (+36.5%)
AMD Venice Zen 6 (AVX512) 204 ms 106 ms 1.92x (+48.0%)
Intel Skylake-X (AVX512) 320 ms 156 ms 2.05x (+51.2%)
AMD Genoa Zen 4 (AVX512) 236 ms 346 ms 0.68x (-46.6%)
Intel Ice Lake (AVX512) 242 ms 316 ms 0.77x (-30.6%)
AMD Milan Zen 3 (AVX2) 252 ms 230 ms 1.10x (+8.7%)
Intel Haswell (AVX2) 369 ms 456 ms 0.81x (-23.6%)
NVIDIA Grace (SVE2) 336 ms 337 ms 1.00x (-0.3%)
ARM Neoverse-N1 (NEON) 1081 ms 1105 ms 0.98x (-2.2%)
======================================================================
Bug: 537818862
Change-Id: I3ddeddb040fc52a30dabe7ae2c71d16dda021568
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8343467
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- NV16/24 FOURCC and document
- Rotate take uint8* for ubsan
Bug: 537818862
Change-Id: I2bc7109abaffe2464e031e33c22296c007077d82
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8339427
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- Add avx512bw, neon 32 bit and RVV versions
- Adapt SSE2 to psrlw instead of pmulhuw
CPU Before After Improvement
---------------------------------------------------------------
Granite Rapids (AVX10) 77 ms 75 ms +2.6% (1.03x)
Emerald Rapids (AVX512) 75 ms 67 ms +10.7% (1.12x)
Sapphire Rapids (AVX512) 91 ms 85 ms +6.6% (1.07x)
AMD Venice Zen 6 (AVX512) 30 ms 30 ms 0.0% (1.00x)
AMD Turin Zen 5 (AVX512) 32 ms 32 ms 0.0% (1.00x)
AMD Genoa Zen 4 (AVX512) 35 ms 36 ms -2.9% (0.97x)
Skylake-X (AVX512) 119 ms 119 ms 0.0% (1.00x)
Ice Lake (AVX512) 93 ms 104 ms -11.8% (0.89x)
AMD Milan Zen 3 (AVX2) 43 ms 44 ms -2.3% (0.98x)
Haswell (AVX2) 129 ms 138 ms -7.0% (0.93x)
NVIDIA Grace (SVE2) 42 ms 43 ms -2.4% (0.98x)
Neoverse-N1 (NEON) 87 ms 87 ms 0.0% (1.00x)
---------------------------------------------------------------
Bug: None
Change-Id: Ia8f310a70c8b1229927fcdf117fe22e131d70ea9
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8330912
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Add missing RGBToYMatrixRow_Any_{LSX,LASX} and kAddY offset fix so
RAW/RGB24 to YUV works on LoongArch.
Change-Id: I4c990a3705d7f521fa6d0c9e7e01f1e36f81c379
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8292090
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reduces include/libyuv/basic_types.h to only define LIBYUV_API.
Directly includes <stdint.h> and <stddef.h> where needed in public
headers.
Replaces LIBYUV_BOOL, LIBYUV_FALSE, and LIBYUV_TRUE with C++ bool,
true, and false in C++ code (MJpegDecoder, unit tests) and int (0, 1)
for C API functions.
Updates documentation and unit tests.
Test: ninja -C out/Release libyuv_unittest && ./out/Release/libyuv_unittest --gtest_filter=*
Change-Id: I2d58e872f0f976263bcda546dcc3b1b5f7e53d58
TAG=agy
CONV=2730bc56-93ff-48ed-b615-dcde1b834394
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280662
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
CPU / Architecture Before (C) After (NEON) Improvement
----------------------------------------------------------------------
Cortex-A53 (Pixel 2 XL Little) 3248 ms 792 ms 4.10x
Cortex-A73 (Pixel 2 XL Big) 432 ms 392 ms 1.10x
Cortex-A55 (Pixel 7 Little) 780 ms 563 ms 1.39x
Cortex-A78 (Pixel 7 Mid) 254 ms 170 ms 1.49x
Cortex-X1 (Pixel 7 Big) 158 ms 129 ms 1.22x
Cortex-A510 (Galaxy S22 Little) 816 ms 441 ms 1.85x
Cortex-A710 (Galaxy S22 Mid) 270 ms 129 ms 2.09x
Cortex-X2 (Galaxy S22 Big) 116 ms 80 ms 1.45x
Cortex-A510 (Galaxy S23 Little) 723 ms 373 ms 1.94x
Cortex-A715 (Galaxy S23 Mid) 183 ms 119 ms 1.54x
Cortex-X3 (Galaxy S23 Big) 122 ms 72 ms 1.69x
Lumix C1 Pro (PD2509 Little) 246 ms 156 ms 1.58x
Lumix C1 Premium (PD2509 Mid) 211 ms 112 ms 1.88x
Lumix C1 Ultra (PD2509 Big) 82 ms 50 ms 1.64x
Oryon-v3M (OnePlus Mid) 233 ms 150 ms 1.55x
Oryon-v3L (OnePlus Big) 103 ms 85 ms 1.21x
On Icelake
Function (1280x720) C SSSE3 AVX2 AVX-512BW Gain
----------------------------------------------------------------------
BlendPlane 901 ms 165 ms 135 ms 115 ms 1.17x
I420Blend 1398 ms 287 ms 198 ms 187 ms 1.06x
Bug: 546077674
Change-Id: I189c44cc757c0e26838a7b6fdb2db1cbe5f96f20
TAG=agy
CONV=0f82c25c-616e-4bb3-beeb-981decb0f40c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8297264
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
CPU / Arch Before (Kernel) After (Kernel) Delta
----------------------------------------------------------------------
Haswell (x86_64) 263 ms (SSSE3) 263 ms (AVX2) 0.0%
Milan (Zen 3) 115 ms (SSSE3) 138 ms (AVX2) -20.0%
SapphRap (x86_64) 173 ms (SSSE3) 178 ms (AVX512VBMI) -2.9%
GNR (Granite R) 191 ms (SSSE3) 189 ms (AVX512VBMI) +1.0%
Turin (Zen 5) 69 ms (SSSE3) 55 ms (AVX512VBMI) +20.3%
Icelake (x86_64) 204 ms (SSSE3) 186 ms (AVX512VBMI) +8.8%
Cascade Lake 237 ms (SSSE3) 229 ms (AVX512BW) +3.4%
Neoverse (aarch64)* 103 ms (SVE2) 103 ms (SVE2) 0.0%
Bug: None
Change-Id: Ib218632e16b3308a214e9ffcbc745d6c41116c6e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280536
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- improves I420ToRAW performance
I420ToRAW on Skylake
Was AVX2
I420ToRAW_Opt (262 ms)
Now AVX512
I420ToRAW_Opt (212 ms)
I420ToRAW on Icelake with avx512vbmi
Was I420ToRAW_Opt (200 ms)
Now I420ToRAW_Opt (187 ms)
Bug: None
Change-Id: If5b171801ba931557d379696fc128d9b155f7c44
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8267746
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
This reverts commit 87c797b3c29cc6fb233fc329a93aace2942ba2d9.
Reason for revert: Only commit 8aeb3a9 needs the workaround
Original change's description:
> Add the ARGBToRGB565Row_SSE2() function back
>
> Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
> back because they hide the unaligned access in ARGBToRGB565Row_C().
>
> To reproduce the unalgined access error, run the following command in
> Google's internal repository:
> blaze test --config=asan third_party/libyuv:libyuv_test \
> --test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
>
> Bug: b:547803993
> Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
> Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
> Commit-Queue: Wan-Teh Chang <wtc@google.com>
> Reviewed-by: Frank Barchard <fbarchard@google.com>
Bug: b:547803993
Change-Id: I904126f032f4ed37c470d8972152a137ca315168
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8258919
Reviewed-by: Frank Barchard <fbarchard@google.com>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
back because they hide the unaligned access in ARGBToRGB565Row_C().
To reproduce the unalgined access error, run the following command in
Google's internal repository:
blaze test --config=asan third_party/libyuv:libyuv_test \
--test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
Bug: b:547803993
Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
- Implemented `I422ToRGB24Row_AVX2` and `I422ToRGB24Row_AVX512VBMI` in:
- `row_gcc.cc`: Inline assembly for GCC/Clang.
- `row_win.cc`: C++ intrinsics for MSVC (also verified with Clang).
Reduced width alignment requirement: changed from 32-pixel to 16-pixel
alignment in `convert_argb.cc` and `row_any.cc`. This allows the AVX2
path to be used for more common video resolutions.
```
I420ToRAW vs Rust on Icelake Xeon
Size I420ToRAW yuv420_to_rgb iterations
------- --------- --------------- ----------
640x480 57 us 77.377 µs/iter 3000
1280x720 170 us 221.671 µs/iter 1000
1920x1080 396 us 494.324 µs/iter 500
3840x2160 2040 us 2357 µs/iter 200
```
Bug: 42280902
Change-Id: I07c0505c95410ea16a6218c858844791a11ef073
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7908323
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
"libyuv/basic_types.h" includes <stddef.h>. So it is not necessary to
include both <stddef.h> and "libyuv/basic_types.h".
Change-Id: I5a461258a3c6820d1007ac635838f910237f367f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7884381
Reviewed-by: Frank Barchard <fbarchard@google.com>
Here are the functions flagged for mixing both SSE and AVX (or AVX-512)
instructions, which can trigger an AVX transition/assist performance
penalty:
Libyuv Functions addressed in this CL
* I422ToARGBRow_AVX512BW
* HalfFloatRow_SSE2
Not addressed:
* ScaleFilterCols_SSSE3
Bug: libyuv:509681367
Change-Id: I8ced6065dfe0c516d05857086393782c8590062a
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7814945
Reviewed-by: richard winterton <rrwinterton@gmail.com>
