I420ScaleTo569x480_Bilinear (LIBYUV_WIDTH=569 LIBYUV_REPEAT=5000):
CPU AVX2 AVX512 Speedup
Intel Emerald Rapids 124 ms 63 ms 1.97x
Intel Sapphire Rapids 134 ms 69 ms 1.94x
Intel Skylake-X 219 ms 129 ms 1.70x
Intel Granite Rapids 124 ms 76 ms 1.63x
Intel Ice Lake 164 ms 101 ms 1.62x
Bug: None
Change-Id: I7a10446eb4dbcebae68380c2a186d4e3badec551
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8483371
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- Use -fno-tree-vectorize and -fno-tree-slp-vectorize instead of
Clang-only -fno-vectorize and -fno-slp-vectorize in CMakeLists.txt,
riscv-clang.cmake, BUILD.gn, and BUILD.bazel for GCC and Clang
compatibility.
- Split row_rvv.cc and scale_rvv.cc into a dedicated libyuv_rvv target
in BUILD.gn and define LIBYUV_RVV so callers in row.h and scale_row.h
can dispatch to RVV kernels when built without global -march=rv64gcv.
- Guard the x86 AVX512-BF16 vdpphps inline assembly probe in
util/cpuid.c with __i386__ / __x86_64__.
- Replace new/delete and new[]/delete[] with calloc/malloc/free in
source/mjpeg_decoder.cc and util/yuvconvert.cc to avoid C++ standard
library dependencies, and link libm for yuvconstants and
libyuv_unittest in CMakeLists.txt.
Test: cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ ..
Bug: None
Change-Id: I701c0625f2730e864806556213a5e013c931e2d6
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8464751
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Enable LIBYUV_ENABLE_ROWWIN instead of LIBYUV_DISABLE_X86 under
MemorySanitizer so that C++ SIMD intrinsics in row_win.cc and their
corresponding _Any wrappers in row_any.cc are compiled and tested
under MSan.
Rename MEMSET to MSAN_MEMSET in row_any.cc and scale_any.cc, and
initialize vout buffers across all ANY* macros in row_any.cc. Use
calloc for align_buffer_64 and align_buffer_page_end under
LIBYUV_MSAN, and zero-initialize stack pixel buffers in color_test.cc.
Test: out/msan/libyuv_unittest
Bug: None
Change-Id: I0b9fc69af159531699bb2b2db8b48f39831289ee
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8456982
Reviewed-by: richard winterton <rrwinterton@gmail.com>
==================================================
Processor / Architecture Before After Speedup
--------------------------------------------------
Intel Granite Rapids 838 us 222 us 3.77x
Intel Emerald Rapids 823 us 237 us 3.47x
Intel Sapphire Rapids 884 us 241 us 3.67x
AMD Turin (Zen 5) 684 us 119 us 5.75x
AMD Genoa (Zen 4) 858 us 233 us 3.68x
Intel Skylake-X 1421 us 319 us 4.45x
Intel Ice Lake 1105 us 284 us 3.89x
AMD Milan (Zen 3) 980 us 978 us 1.00x
Intel Haswell 1356 us 1387 us 0.98x
==================================================
Bug: None
Change-Id: Id5473082cc2537234db9c53d53c38f2c9698d9d2
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8467771
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Add ComputeCumulativeSumRow_AVX2, CumulativeSumToAverageRow_AVX2,
ComputeCumulativeSumRow_NEON, and CumulativeSumToAverageRow_NEON to
accelerate ARGBBlur and ARGBComputeCumulativeSum on x86_64 and aarch64.
Update ARGBBlurSmall_Opt to benchmark 320x180 with radius 1 and use a
circular cumsum buffer of height radius * 2 + 2 in TestBlur.
ARGBBlurSmall_Opt (320x180, r=1, 10000 iterations):
CPU Was C/SSE2 Now AVX2/NEON Speedup
-------------------- -------------- --------------- ---------------
Intel Granite Rapids 29 ms (SSE2) 24 ms (AVX2) 1.21x
Intel Emerald Rapids 30 ms (SSE2) 23 ms (AVX2) 1.30x
Intel Sapphire Rapid 32 ms (SSE2) 28 ms (AVX2) 1.14x
Intel Ice Lake 43 ms (SSE2) 34 ms (AVX2) 1.26x
Intel Skylake-X 54 ms (SSE2) 45 ms (AVX2) 1.20x
Intel Haswell 59 ms (SSE2) 54 ms (AVX2) 1.09x
AMD Zen 5 32 ms (SSE2) 19 ms (AVX2) 1.68x
AMD Zen 4 36 ms (SSE2) 26 ms (AVX2) 1.38x
AMD Zen 3 42 ms (SSE2) 28 ms (AVX2) 1.50x
ARM Neoverse-V2 256 ms (C) 40 ms (NEON) 6.40x
ARGBBlur_Opt (1280x720, r=55, 1000 iterations):
CPU Was C/SSE2 Now AVX2/NEON Speedup
-------------------- -------------- --------------- ---------------
Intel Granite Rapids 1150 ms (SSE2) 984 ms (AVX2) 1.17x
Intel Emerald Rapids 1081 ms (SSE2) 953 ms (AVX2) 1.13x
Intel Sapphire Rapid 1124 ms (SSE2) 973 ms (AVX2) 1.16x
Intel Ice Lake 1450 ms (SSE2) 1167 ms (AVX2) 1.24x
Intel Skylake-X 1935 ms (SSE2) 1630 ms (AVX2) 1.19x
Intel Haswell 2211 ms (SSE2) 1982 ms (AVX2) 1.12x
AMD Zen 5 1089 ms (SSE2) 694 ms (AVX2) 1.57x
AMD Zen 4 1045 ms (SSE2) 929 ms (AVX2) 1.12x
AMD Zen 3 1215 ms (SSE2) 967 ms (AVX2) 1.26x
ARM Neoverse-V2 6529 ms (C) 1544 ms (NEON) 4.23x
Bug: b/456384776
Change-Id: I39c463c791826d3adb05f0b0845b0b1f56f0e4da
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8464496
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
I422ToRGB24Matrix (299x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Skylake-X 275 ms 213 ms -22.5% (1.29x)
Intel Ice Lake 225 ms 177 ms -21.3% (1.27x)
Intel Sapphire Rapids 181 ms 144 ms -20.4% (1.26x)
Intel Emerald Rapids 167 ms 136 ms -18.6% (1.23x)
Intel Granite Rapids 170 ms 140 ms -17.6% (1.21x)
AMD Zen 4 (Genoa) 192 ms 136 ms -29.2% (1.41x)
AMD Zen 5 (Turin) 102 ms 68 ms -33.3% (1.50x)
AMD Zen 6 (Venice) 102 ms 60 ms -41.2% (1.70x)
====================================================================
I422ToRGB24Matrix (298x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Skylake-X 271 ms 184 ms -32.1% (1.47x)
Intel Ice Lake 235 ms 152 ms -35.3% (1.55x)
Intel Sapphire Rapids 199 ms 128 ms -35.7% (1.55x)
Intel Emerald Rapids 170 ms 121 ms -28.8% (1.40x)
Intel Granite Rapids 177 ms 125 ms -29.4% (1.42x)
AMD Zen 4 (Genoa) 185 ms 119 ms -35.7% (1.55x)
AMD Zen 5 (Turin) 102 ms 59 ms -42.2% (1.73x)
AMD Zen 6 (Venice) 100 ms 51 ms -49.0% (1.96x)
====================================================================
Bug: None
Change-Id: Id75e3d1df6d26bc28a252672816a7c438cb664a4
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8452502
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
I422ToRGB24Matrix (299x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Ice Lake 220 ms 181 ms -17.7% (1.22x)
Intel Sapphire Rapids 181 ms 145 ms -19.9% (1.25x)
Intel Emerald Rapids 169 ms 148 ms -12.4% (1.14x)
Intel Granite Rapids 191 ms 141 ms -26.2% (1.35x)
AMD Zen 4 (Genoa) 171 ms 136 ms -20.5% (1.26x)
AMD Zen 5 (Turin) 94 ms 72 ms -23.4% (1.31x)
AMD Zen 6 (Venice) 95 ms 66 ms -30.5% (1.44x)
====================================================================
Bug: None
Change-Id: I4cd657410cc5253b5e8fd2caa78d17e2b3812393
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8445804
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
P010 and P210 were auxiliary FourCCs and P410 did not exist, so
ConvertToI420, ConvertToARGB, and ConvertFromI420 could not consume or
produce them. Callers had to pick I010/I210 helpers and pack UV by hand.
NV12Scale existed, but NV16/NV24 and P010/P210/P410 had no Scale APIs.
Promote P010 and P210 to primary FourCCs, add FOURCC_P410, and add the
I4xx/Px10 converters those entry points need: I410ToP410, I412ToP412,
P210ToI210, P410ToI410, P010/P210/P410ToI420, I420ToP010/P210/P410,
I422ToP210, I444ToP410, and P410ToARGB/AR30Matrix.
Add NV16Scale (same path as NV12Scale: ScalePlane + UVScale).
Add P010Scale, P210Scale, and P410Scale: 10-bit samples stay in the MSBs, so Y/UV convert to 10-bit LSB, ScalePlane_16, then back to MSB.
UVScale_16 cannot general-scale, so UV is split, scaled as planes, and merged.
Packed-plane layout, odd-width strides, ConvertTo/From round-trips, and
the existing scale factor/size/swap tests cover the new APIs.
Bug: 537818862
Change-Id: I1ae5e6cad5e6d83acf1d0df69f94c23688bbe0e8
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8346569
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Add matrix and BT.601 wrappers that convert ARGB to 16-bit biplanar
YUV in P010 (4:2:0), P210 (4:2:2), and P410 (4:4:4) layouts.
Reuse optimized ARGB matrix row functions for 8-bit Y and chroma,
merge UV with MergeUVRow, then expand with Convert8To16Row (bits=10)
and MultiplyRow_16 (scale=64) so 10-bit samples are stored in the
high bits of each uint16.
Reject width > INT_MAX / 2 so UV width (halfwidth * 2 or width * 2)
does not overflow Convert8To16Row.
Bug: 537818862
Change-Id: I7a776fb3d6f52ce46a0f6825903bfdb9c6dcfbe2
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8314189
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
- avoids intrinsics API changes
- build options for rvv not needed for inline instructions
- declare vl and vtype clobbers instead of cc
- tune for SiFive X280 using 16/32-bit loads and stores instead of
vlseg/vsseg where more efficient
- tune LMUL for SpacemiT K3 (256-bit vectors) with max m4 (m2 where
constrained by register pressure or vlseg/vsseg field rules)
- YUVTORGB use vmulhu
NV12ToARGB performance estimates:
CPU VLEN C Intrinsics Inline ASM Speedup (Inline)
------------------------------------------------------------------------
SiFive X390 1024-bit 360 ms 29 ms 9.5 ms 37.89x (3.05x)
SiFive X280 512-bit 980 ms 115 ms 38.0 ms 25.79x (3.03x)
SpacemiT K3 256-bit 420 ms 68 ms 28.0 ms 15.00x (2.43x)
Google Coral 256-bit 1120 ms 190 ms 58.0 ms 19.31x (3.28x)
T-Head C920 128-bit 510 ms 88 ms 39.0 ms 13.08x (2.26x)
T-Head C908 128-bit 860 ms 145 ms 62.0 ms 13.87x (2.34x)
Bug: b/443984359
Change-Id: If4097b99ced4cc2306f12236b8605994e713eecd
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8426529
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- AVX2 uses vpmulhrsw to compute ((128 * a * b) + 16384) >> 15.
- RVV uses e8, m2 (widening to e16, m4) with vwmulu.vv, vwaddu.wx 128,
and vnsrl.wi by 8.
CPU C Intrinsics Inline ASM Speedup (Inline)
------------------------------------------------------------------------
SiFive X390 gen 2 (est.) 2150 ms 29 ms 27 ms 79.63x (1.07x)
SpacemiT K3 (est.) 980 ms 41 ms 38 ms 25.79x (1.08x)
T-Head C908 (est.) 2060 ms 100 ms 94 ms 21.91x (1.06x)
Google Coral (est.) 2680 ms 94 ms 88 ms 30.45x (1.07x)
Test: libyuv_unittest --gtest_filter=*ARGBMultiply*
Bug: None
Change-Id: Iff65794623a6f29e5cb6bda7bd3084c8a5f676d1
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8422278
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Add SwapUVRow_RVV using RVV intrinsics by default and unit-stride
e16, m4 inline assembly (vle16.v, vsrl.vi, vsll.vi, vor.vv, vse16.v)
under #ifdef RVV_ASM.
CPU C Intrinsics Inline ASM Speedup (Inline)
--------------------------------------------------------------------
SiFive X280 (est.) 980 ms 115 ms 38 ms 25.79x (3.03x)
SpacemiT K3 (est.) 420 ms 68 ms 28 ms 15.00x (2.43x)
T-Head C908 (est.) 860 ms 145 ms 62 ms 13.87x (2.34x)
Google Coral (est.) 1120 ms 190 ms 58 ms 19.31x (3.28x)
Original author: Matteo Forzan <matteo.forzan95@gmail.com>
R=wtc@google.comCC=matteo.forzan95@gmail.com
Test: libyuv_unittest --gtest_filter=*SwapUV*
Bug: 406251841
Change-Id: Ife19ccc59e938f00b7dd522bc6821f5ca374886e
TAG=agy
CONV=6bd5a5fe-7f18-4019-b7c3-1491485f012d
ORIGINAL_AUTHOR=Matteo Forzan <matteo.forzan95@gmail.com>
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8339302
Reviewed-by: David Gao <davidgao@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
The existing SVE2 path only computes the average for a pair of input
values instead of groups of four as done in the Neon implementation.
The average of two values must be retained for the final odd element if
present. Existing code just handles this by falling back to the C
implementation, so mirror that for SVE2 as well.
Reduction in time taken relative to the existing Neon code, the SVE2
code remains faster even with the correct averaging:
Cortex-A510: -12.66%
Lumex C1-Nano: -6.38%
Cortex-A720: -9.73%
Lumex C1-Pro: +0.99%
Cortex-X4: -3.05%
Lumex C1-Ultra: -18.98%
Bug: 546962730
Change-Id: I7ab348e64a9d079b81d070cea5fd575243b037d2
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8252281
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Justin Green <greenjustin@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Add SwapUVRow_AVX2 implementation in row_win.cc for MSVC and enable
HAS_SWAPUVROW_AVX2 under USE_ROW_WIN.
Share kShuffleUVToVU shuffle table between SSSE3 and AVX2 in row_gcc.cc.
Make NV12ToNV21 the primary implementation and alias NV21ToNV12 to it.
Add NV12ToNV21 test in convert_test.cc and benchmark dispatch for
SSSE3 and AVX2 in planar_test.cc.
CPU Before After Speedup
----------------------------------------------------------------
Intel Granite Rapids 282 ms 125 ms 2.26x
Intel Emerald Rapids 264 ms 108 ms 2.44x
Intel Sapphire Rapids 276 ms 122 ms 2.26x
Intel Ice Lake 310 ms 122 ms 2.54x
Intel Skylake-X 573 ms 157 ms 3.65x
Intel Haswell 620 ms 162 ms 3.83x
AMD Zen 4 (Genoa) 266 ms 52 ms 5.12x
AMD Zen 6 (Venice) 229 ms 43 ms 5.33x
ARM Neoverse-N1 140 ms 71 ms 1.97x
NVIDIA Grace (Neoverse-V2) 62 ms 69 ms 0.90x
R=wtc@google.com
Test: libyuv_unittest --gtest_filter=*SwapUV*:*NV12ToNV21*:*NV21ToNV12*
Bug: 406251841
Change-Id: I9bed5522aedadea13cdf69c27e3cbe633e31dd5b
TAG=agy
CONV=bb50ac62-0c55-4969-8fff-c71ab3d9a79f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8356968
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- instead of 64 bit load, use vlseg2 with 32 bit
Test build with:
-fsanitize=alignment -fsanitize-trap=alignment
Test: libyuv_unittest --gtest_filter=*UVScale*:*ScalePlane*:*ARGBScale*
Bug: 553156658
Change-Id: Ib6febe49f085f5b3deb33adb6b6f82f7d836c3fa
TAG=agy
CONV=b29f62f9-15a4-4f38-8414-86c8330e61fb
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8315435
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- Also remove redundant neon check for I8MM kernels
======================================================================
CPU Before After Speedup
----------------------------------------------------------------------
Intel Granite Rapids (AVX10.1) 251 ms 141 ms 1.78x (+43.8%)
Intel Emerald Rapids (AVX512) 199 ms 108 ms 1.84x (+45.7%)
Intel Sapphire Rapids (AVX512) 230 ms 112 ms 2.05x (+51.3%)
AMD Turin Zen 5 (AVX512) 156 ms 99 ms 1.58x (+36.5%)
AMD Venice Zen 6 (AVX512) 204 ms 106 ms 1.92x (+48.0%)
Intel Skylake-X (AVX512) 320 ms 156 ms 2.05x (+51.2%)
AMD Genoa Zen 4 (AVX512) 236 ms 346 ms 0.68x (-46.6%)
Intel Ice Lake (AVX512) 242 ms 316 ms 0.77x (-30.6%)
AMD Milan Zen 3 (AVX2) 252 ms 230 ms 1.10x (+8.7%)
Intel Haswell (AVX2) 369 ms 456 ms 0.81x (-23.6%)
NVIDIA Grace (SVE2) 336 ms 337 ms 1.00x (-0.3%)
ARM Neoverse-N1 (NEON) 1081 ms 1105 ms 0.98x (-2.2%)
======================================================================
Bug: 537818862
Change-Id: I3ddeddb040fc52a30dabe7ae2c71d16dda021568
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8343467
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- NV16/24 FOURCC and document
- Rotate take uint8* for ubsan
Bug: 537818862
Change-Id: I2bc7109abaffe2464e031e33c22296c007077d82
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8339427
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Add matrix and BT.601 wrapper APIs for converting ARGB to NV16 and
NV24. Reuse the optimized ARGB Y and UV row functions, then merge
the planar chroma rows into the interleaved destination plane.
NV16 performs horizontal 2:1 chroma subsampling on every source row.
NV24 preserves full-resolution 4:4:4 chroma. Both paths support odd
widths, negative heights, and the existing matrix constants.
Add tests that compare scalar and optimized paths at an odd width and
height through the matrix API and its BT.601 wrapper.
Bug: 537818862
Change-Id: Iddefc714a800cdec4dd1394ac7a52bf7b51fd350
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8314188
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- Add avx512bw, neon 32 bit and RVV versions
- Adapt SSE2 to psrlw instead of pmulhuw
CPU Before After Improvement
---------------------------------------------------------------
Granite Rapids (AVX10) 77 ms 75 ms +2.6% (1.03x)
Emerald Rapids (AVX512) 75 ms 67 ms +10.7% (1.12x)
Sapphire Rapids (AVX512) 91 ms 85 ms +6.6% (1.07x)
AMD Venice Zen 6 (AVX512) 30 ms 30 ms 0.0% (1.00x)
AMD Turin Zen 5 (AVX512) 32 ms 32 ms 0.0% (1.00x)
AMD Genoa Zen 4 (AVX512) 35 ms 36 ms -2.9% (0.97x)
Skylake-X (AVX512) 119 ms 119 ms 0.0% (1.00x)
Ice Lake (AVX512) 93 ms 104 ms -11.8% (0.89x)
AMD Milan Zen 3 (AVX2) 43 ms 44 ms -2.3% (0.98x)
Haswell (AVX2) 129 ms 138 ms -7.0% (0.93x)
NVIDIA Grace (SVE2) 42 ms 43 ms -2.4% (0.98x)
Neoverse-N1 (NEON) 87 ms 87 ms 0.0% (1.00x)
---------------------------------------------------------------
Bug: None
Change-Id: Ia8f310a70c8b1229927fcdf117fe22e131d70ea9
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8330912
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Add missing RGBToYMatrixRow_Any_{LSX,LASX} and kAddY offset fix so
RAW/RGB24 to YUV works on LoongArch.
Change-Id: I4c990a3705d7f521fa6d0c9e7e01f1e36f81c379
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8292090
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reduces include/libyuv/basic_types.h to only define LIBYUV_API.
Directly includes <stdint.h> and <stddef.h> where needed in public
headers.
Replaces LIBYUV_BOOL, LIBYUV_FALSE, and LIBYUV_TRUE with C++ bool,
true, and false in C++ code (MJpegDecoder, unit tests) and int (0, 1)
for C API functions.
Updates documentation and unit tests.
Test: ninja -C out/Release libyuv_unittest && ./out/Release/libyuv_unittest --gtest_filter=*
Change-Id: I2d58e872f0f976263bcda546dcc3b1b5f7e53d58
TAG=agy
CONV=2730bc56-93ff-48ed-b615-dcde1b834394
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280662
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
libyuv/scale_rgb.h declares an API function, so it is a public header
and should be included by the umbrella header libyuv.h.
libyuv/scale_row.h should be an internal header. It should not be
included by the umbrella header libyuv.h.
Fixed: 553139990, 553143607
Change-Id: I5c1a61296a06a96a36c33ec470513ab7bd1f0e66
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8301033
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
CPU / Architecture Before (C) After (NEON) Improvement
----------------------------------------------------------------------
Cortex-A53 (Pixel 2 XL Little) 3248 ms 792 ms 4.10x
Cortex-A73 (Pixel 2 XL Big) 432 ms 392 ms 1.10x
Cortex-A55 (Pixel 7 Little) 780 ms 563 ms 1.39x
Cortex-A78 (Pixel 7 Mid) 254 ms 170 ms 1.49x
Cortex-X1 (Pixel 7 Big) 158 ms 129 ms 1.22x
Cortex-A510 (Galaxy S22 Little) 816 ms 441 ms 1.85x
Cortex-A710 (Galaxy S22 Mid) 270 ms 129 ms 2.09x
Cortex-X2 (Galaxy S22 Big) 116 ms 80 ms 1.45x
Cortex-A510 (Galaxy S23 Little) 723 ms 373 ms 1.94x
Cortex-A715 (Galaxy S23 Mid) 183 ms 119 ms 1.54x
Cortex-X3 (Galaxy S23 Big) 122 ms 72 ms 1.69x
Lumix C1 Pro (PD2509 Little) 246 ms 156 ms 1.58x
Lumix C1 Premium (PD2509 Mid) 211 ms 112 ms 1.88x
Lumix C1 Ultra (PD2509 Big) 82 ms 50 ms 1.64x
Oryon-v3M (OnePlus Mid) 233 ms 150 ms 1.55x
Oryon-v3L (OnePlus Big) 103 ms 85 ms 1.21x
On Icelake
Function (1280x720) C SSSE3 AVX2 AVX-512BW Gain
----------------------------------------------------------------------
BlendPlane 901 ms 165 ms 135 ms 115 ms 1.17x
I420Blend 1398 ms 287 ms 198 ms 187 ms 1.06x
Bug: 546077674
Change-Id: I189c44cc757c0e26838a7b6fdb2db1cbe5f96f20
TAG=agy
CONV=0f82c25c-616e-4bb3-beeb-981decb0f40c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8297264
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
CPU / Arch Before (Kernel) After (Kernel) Delta
----------------------------------------------------------------------
Haswell (x86_64) 263 ms (SSSE3) 263 ms (AVX2) 0.0%
Milan (Zen 3) 115 ms (SSSE3) 138 ms (AVX2) -20.0%
SapphRap (x86_64) 173 ms (SSSE3) 178 ms (AVX512VBMI) -2.9%
GNR (Granite R) 191 ms (SSSE3) 189 ms (AVX512VBMI) +1.0%
Turin (Zen 5) 69 ms (SSSE3) 55 ms (AVX512VBMI) +20.3%
Icelake (x86_64) 204 ms (SSSE3) 186 ms (AVX512VBMI) +8.8%
Cascade Lake 237 ms (SSSE3) 229 ms (AVX512BW) +3.4%
Neoverse (aarch64)* 103 ms (SVE2) 103 ms (SVE2) 0.0%
Bug: None
Change-Id: Ib218632e16b3308a214e9ffcbc745d6c41116c6e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280536
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- reduced shifts to isolate 10 bits
- permi2w instead of packlwd to combine words (ARG) (GB) into ARGB
- vpternlogd to combine A,R,G into 1 register
Saphhire Rapids 10000 iterations of 1280x720P:
Conversion Kernel │ Was │ Now │ Speedup
──────────────────────┼────────────────────────┼────────────────
I420ToAR30_Opt │ 1675 ms │ 1568 ms │ +7.1%
H420ToAR30_Opt │ 1632 ms │ 1555 ms │ +5.2%
I210ToAR30_Opt │ 2459 ms │ 2167 ms │ +13.5%
P010ToAR30_Opt │ 2054 ms │ 1926 ms │ +6.6%
P016ToAR30_Opt │ 2043 ms │ 1938 ms │ +5.4%
Total Test Suite Time │ 9,866 ms │ 9,155 ms │ +7.8% overall
Bug: None
Change-Id: I7972f98408a65a2bb51c506145e4c0c63824c424
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8259796
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- improves I420ToRAW performance
I420ToRAW on Skylake
Was AVX2
I420ToRAW_Opt (262 ms)
Now AVX512
I420ToRAW_Opt (212 ms)
I420ToRAW on Icelake with avx512vbmi
Was I420ToRAW_Opt (200 ms)
Now I420ToRAW_Opt (187 ms)
Bug: None
Change-Id: If5b171801ba931557d379696fc128d9b155f7c44
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8267746
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
The tail LD2B predicate previously used the raw remaining width, however
for odd widths this is incorrect. We already calculate the correct
halved predicate for the UV store part so just compute the load
predicate from that directly.
Bug: 546962730
Change-Id: I59654208ea48eb5bcc6261a63cb90abae58c1668
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8252280
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
The "Disabled:" comment for HAS_ARGBTOYMATRIXROW_SME was left behind
when we re-enabled HAS_ARGBTOYMATRIXROW_SME.
Bug: 546962730
Change-Id: Id21fb368a5233bed9f9733d8a54bce3ed0751f47
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8263125
Reviewed-by: George Steed <george.steed@arm.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
This reverts commit 87c797b3c29cc6fb233fc329a93aace2942ba2d9.
Reason for revert: Only commit 8aeb3a9 needs the workaround
Original change's description:
> Add the ARGBToRGB565Row_SSE2() function back
>
> Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
> back because they hide the unaligned access in ARGBToRGB565Row_C().
>
> To reproduce the unalgined access error, run the following command in
> Google's internal repository:
> blaze test --config=asan third_party/libyuv:libyuv_test \
> --test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
>
> Bug: b:547803993
> Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
> Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
> Commit-Queue: Wan-Teh Chang <wtc@google.com>
> Reviewed-by: Frank Barchard <fbarchard@google.com>
Bug: b:547803993
Change-Id: I904126f032f4ed37c470d8972152a137ca315168
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8258919
Reviewed-by: Frank Barchard <fbarchard@google.com>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
back because they hide the unaligned access in ARGBToRGB565Row_C().
To reproduce the unalgined access error, run the following command in
Google's internal repository:
blaze test --config=asan third_party/libyuv:libyuv_test \
--test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
Bug: b:547803993
Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
The following macros are disabled because they caused test failures:
HAS_ARGBTOYMATRIXROW_SVE2
HAS_HALFMERGEUVROW_SVE2
HAS_UYVYTOUVROW_SVE2
HAS_YUY2TOUVROW_SVE2
HAS_ARGBTOYMATRIXROW_SME is also disabled because ARGBToYMatrixRow_SME()
has the same code as ARGBToYMatrixRow_SVE2().
Bug: 546962730
Change-Id: Ic590e809443585ee66c609ff8ba6b6f0e40300c9
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8261763
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Copy dither4x4 (16 bytes) into an aligned local uint32_t dither4[4] array
using memcpy and index using dither4[y & 3] to avoid unaligned 32-bit
pointer dereference traps in fastbuild / UB sanitizers.
Bug: None
Test: LibYUVConvertTest.TestNoDither, LibYUVConvertTest.TestDither
Change-Id: I4447cd2924b16739a9bce2792d57a550dedeb345
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8262354
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
- MergeUVRow_SVE2: Interleaves U and V
- SplitUVRow_SVE2: De-interleaves UV
- InterpolateRow_SVE2: Bilinear row interpolation
- YUY2ToYRow_SVE2, YUY2ToUVRow_SVE2, UYVYToYRow_SVE2, UYVYToUVRow_SVE2
- HalfMergeUVRow_SVE2: 2:1 vertical averaging of U and V
OnePlus 15 with Qualcomm Snapdragon 8 Elite Gen 5 Oryon V3
Benchmark (1280x720 1000x) | SVE2 | NEON | Result
-------------------------------+--------+--------+------------------
TestInterpolatePlane | 119 ms | 164 ms | SVE2 is 1.38x win
TestInterpolatePlane_16 | 138 ms | 172 ms | SVE2 is 1.25x win
SplitUVPlane_Opt | 68 ms | 69 ms | SVE2 is 1.01x win
MergeUVPlane_Opt | 40 ms | 40 ms | Tied
SwapUVPlane_Opt | 72 ms | 72 ms | Tied
HalfMergeUVPlane_Opt | 50 ms | 50 ms | Tied
I420ToNV12_Opt | 25 ms | 26 ms | SVE2 is faster
I420ToNV21_Opt | 26 ms | 26 ms | Tied
YUY2ToNV12_Opt | 79 ms | 81 ms | SVE2 is faster
NV21ToNV12_Opt | 36 ms | 35 ms | Tied
Android420ToI420ToNV12_Opt | 53 ms | 53 ms | Tied
Android420ToI420ToNV21_Opt | 35 ms | 33 ms | NEON is faster
YUY2ToI420_Opt | 87 ms | 84 ms | NEON is faster
UYVYToI420_Opt | 87 ms | 83 ms | NEON is faster
YUY2ToI422_Opt | 101 ms | 97 ms | NEON is faster
UYVYToI422_Opt | 101 ms | 97 ms | NEON is faster
UYVYToNV12_Opt | 111 ms | 108 ms | NEON is faster
ARGBToNV21_Opt | 279 ms | 279 ms | Tied
ABGRToNV21_Opt | 279 ms | 280 ms | SVE2 is faster
ARGBToNV12_Opt | 285 ms | 280 ms | NEON is faster
ABGRToNV12_Opt | 284 ms | 281 ms | NEON is faster
Bug: 42280902
Change-Id: I53f633fe453f2853f7bc9fe17280e938912c4e8f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8196068
Reviewed-by: Justin Green <greenjustin@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
