Reduces include/libyuv/basic_types.h to only define LIBYUV_API.
Directly includes <stdint.h> and <stddef.h> where needed in public
headers.
Replaces LIBYUV_BOOL, LIBYUV_FALSE, and LIBYUV_TRUE with C++ bool,
true, and false in C++ code (MJpegDecoder, unit tests) and int (0, 1)
for C API functions.
Updates documentation and unit tests.
Test: ninja -C out/Release libyuv_unittest && ./out/Release/libyuv_unittest --gtest_filter=*
Change-Id: I2d58e872f0f976263bcda546dcc3b1b5f7e53d58
TAG=agy
CONV=2730bc56-93ff-48ed-b615-dcde1b834394
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280662
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
CPU / Architecture Before (C) After (NEON) Improvement
----------------------------------------------------------------------
Cortex-A53 (Pixel 2 XL Little) 3248 ms 792 ms 4.10x
Cortex-A73 (Pixel 2 XL Big) 432 ms 392 ms 1.10x
Cortex-A55 (Pixel 7 Little) 780 ms 563 ms 1.39x
Cortex-A78 (Pixel 7 Mid) 254 ms 170 ms 1.49x
Cortex-X1 (Pixel 7 Big) 158 ms 129 ms 1.22x
Cortex-A510 (Galaxy S22 Little) 816 ms 441 ms 1.85x
Cortex-A710 (Galaxy S22 Mid) 270 ms 129 ms 2.09x
Cortex-X2 (Galaxy S22 Big) 116 ms 80 ms 1.45x
Cortex-A510 (Galaxy S23 Little) 723 ms 373 ms 1.94x
Cortex-A715 (Galaxy S23 Mid) 183 ms 119 ms 1.54x
Cortex-X3 (Galaxy S23 Big) 122 ms 72 ms 1.69x
Lumix C1 Pro (PD2509 Little) 246 ms 156 ms 1.58x
Lumix C1 Premium (PD2509 Mid) 211 ms 112 ms 1.88x
Lumix C1 Ultra (PD2509 Big) 82 ms 50 ms 1.64x
Oryon-v3M (OnePlus Mid) 233 ms 150 ms 1.55x
Oryon-v3L (OnePlus Big) 103 ms 85 ms 1.21x
On Icelake
Function (1280x720) C SSSE3 AVX2 AVX-512BW Gain
----------------------------------------------------------------------
BlendPlane 901 ms 165 ms 135 ms 115 ms 1.17x
I420Blend 1398 ms 287 ms 198 ms 187 ms 1.06x
Bug: 546077674
Change-Id: I189c44cc757c0e26838a7b6fdb2db1cbe5f96f20
TAG=agy
CONV=0f82c25c-616e-4bb3-beeb-981decb0f40c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8297264
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
CPU / Arch Before (Kernel) After (Kernel) Delta
----------------------------------------------------------------------
Haswell (x86_64) 263 ms (SSSE3) 263 ms (AVX2) 0.0%
Milan (Zen 3) 115 ms (SSSE3) 138 ms (AVX2) -20.0%
SapphRap (x86_64) 173 ms (SSSE3) 178 ms (AVX512VBMI) -2.9%
GNR (Granite R) 191 ms (SSSE3) 189 ms (AVX512VBMI) +1.0%
Turin (Zen 5) 69 ms (SSSE3) 55 ms (AVX512VBMI) +20.3%
Icelake (x86_64) 204 ms (SSSE3) 186 ms (AVX512VBMI) +8.8%
Cascade Lake 237 ms (SSSE3) 229 ms (AVX512BW) +3.4%
Neoverse (aarch64)* 103 ms (SVE2) 103 ms (SVE2) 0.0%
Bug: None
Change-Id: Ib218632e16b3308a214e9ffcbc745d6c41116c6e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280536
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- improves I420ToRAW performance
I420ToRAW on Skylake
Was AVX2
I420ToRAW_Opt (262 ms)
Now AVX512
I420ToRAW_Opt (212 ms)
I420ToRAW on Icelake with avx512vbmi
Was I420ToRAW_Opt (200 ms)
Now I420ToRAW_Opt (187 ms)
Bug: None
Change-Id: If5b171801ba931557d379696fc128d9b155f7c44
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8267746
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
This reverts commit 87c797b3c29cc6fb233fc329a93aace2942ba2d9.
Reason for revert: Only commit 8aeb3a9 needs the workaround
Original change's description:
> Add the ARGBToRGB565Row_SSE2() function back
>
> Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
> back because they hide the unaligned access in ARGBToRGB565Row_C().
>
> To reproduce the unalgined access error, run the following command in
> Google's internal repository:
> blaze test --config=asan third_party/libyuv:libyuv_test \
> --test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
>
> Bug: b:547803993
> Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
> Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
> Commit-Queue: Wan-Teh Chang <wtc@google.com>
> Reviewed-by: Frank Barchard <fbarchard@google.com>
Bug: b:547803993
Change-Id: I904126f032f4ed37c470d8972152a137ca315168
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8258919
Reviewed-by: Frank Barchard <fbarchard@google.com>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
back because they hide the unaligned access in ARGBToRGB565Row_C().
To reproduce the unalgined access error, run the following command in
Google's internal repository:
blaze test --config=asan third_party/libyuv:libyuv_test \
--test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
Bug: b:547803993
Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
- Implemented `I422ToRGB24Row_AVX2` and `I422ToRGB24Row_AVX512VBMI` in:
- `row_gcc.cc`: Inline assembly for GCC/Clang.
- `row_win.cc`: C++ intrinsics for MSVC (also verified with Clang).
Reduced width alignment requirement: changed from 32-pixel to 16-pixel
alignment in `convert_argb.cc` and `row_any.cc`. This allows the AVX2
path to be used for more common video resolutions.
```
I420ToRAW vs Rust on Icelake Xeon
Size I420ToRAW yuv420_to_rgb iterations
------- --------- --------------- ----------
640x480 57 us 77.377 µs/iter 3000
1280x720 170 us 221.671 µs/iter 1000
1920x1080 396 us 494.324 µs/iter 500
3840x2160 2040 us 2357 µs/iter 200
```
Bug: 42280902
Change-Id: I07c0505c95410ea16a6218c858844791a11ef073
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7908323
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
"libyuv/basic_types.h" includes <stddef.h>. So it is not necessary to
include both <stddef.h> and "libyuv/basic_types.h".
Change-Id: I5a461258a3c6820d1007ac635838f910237f367f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7884381
Reviewed-by: Frank Barchard <fbarchard@google.com>
Here are the functions flagged for mixing both SSE and AVX (or AVX-512)
instructions, which can trigger an AVX transition/assist performance
penalty:
Libyuv Functions addressed in this CL
* I422ToARGBRow_AVX512BW
* HalfFloatRow_SSE2
Not addressed:
* ScaleFilterCols_SSSE3
Bug: libyuv:509681367
Change-Id: I8ced6065dfe0c516d05857086393782c8590062a
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7814945
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- implement wrappers with RAW, RGB24, NV21 and JNV21 to call it.
Zen5
Was [ OK ] LibYUVConvertTest.RAWToJNV21_Opt (1146 ms)
Now [ OK ] LibYUVConvertTest.RAWToJNV21_Opt (1446 ms)
reason - the new code uses 1 pass for RAWToY but 2 pass for RAWToARGB,ARGBToUV. needs 1 RGBToUV
Bug: libyuv:42280902
Change-Id: Ife6fbed0829484045409e6d42b85cec1d1fd6052
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7780026
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Adds RGBToYMatrixRow_AVX2 which reads 24 bit RGB values by reading 3 vectors instead of 4 and permutes them into 4 ARGB vectors before conversion.
Also adds RGBToYMatrixRow_Opt and RGBToYMatrixRow_2Step_Opt to convert_argb_test.cc to benchmark and compare the direct AVX2 conversion vs a 2-step approach.
./libyuv_test '--gunit_filter=*RAWToJ400_Opt' --libyuv_width=1280 --libyuv_height=720 --libyuv_repeat=10000 --libyuv_flags=-1 --libyuv_cpu_info=-1
AMD Zen 5
Was LibYUVConvertTest.RAWToJ400_Opt (757 ms)
Now LibYUVConvertTest.RAWToJ400_Opt (699 ms)
Intel Skylake
Was LibYUVConvertTest.RAWToJ400_Opt (1705 ms)
Now LibYUVConvertTest.RAWToJ400_Opt (1426 ms)
Bug: 477295731
Change-Id: I29866baf4ad5fe7a3725e4a01f2fe24649510a7d
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7777325
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Justin Green <greenjustin@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Benchmark on Icelake Xeon
Now AVX512BW:
[ OK ] LibYUVConvertTest.ARGBToNV12_Opt (1723 ms)
Was AVX2:
[ OK ] LibYUVConvertTest.ARGBToNV12_Opt (2144 ms)
- Added `ARGBToUVMatrixRow_AVX512BW` implementation in `source/row_gcc.cc`.
- Added corresponding `ARGBToUVRow_AVX512BW` and `ABGRToUVRow_AVX512BW` functions.
- Added unaligned wrappers `ARGBToUVRow_Any_AVX512BW` and `ABGRToUVRow_Any_AVX512BW` in `source/row_any.cc`.
- Updated `source/row_any.cc` to correctly size `vin` and `vout` buffers for AVX512BW width and adjusted the `ANY12MS` and `ANY12S` macros to handle `MASK=63`.
- Updated `include/libyuv/row.h` with the required AVX512BW headers and definitions, scoped appropriately.
- Wired all callers of `ARGBToUVRow_AVX2` and related functions in `source/convert.cc` and `source/convert_from_argb.cc` to dynamically use the `AVX512BW` implementations if the CPU flag indicates AVX-512BW support.
- Optimized AVX-512 code to generate the `-1` multiplier in a single instruction (`vpternlogd`) and reused it across word (`vpmaddwd`) dot products. Handled the resulting negation by replacing a subtraction with `vpaddw` offset adjustment.
Bug: 477295731
R=dalecurtis@chromium.org, rrwinterton@gmail.com
Change-Id: Ida5fb27e59ae4c1c3824737f009b80549cd20a06
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7763257
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
- add ARGBToYMatrixRow_AVX512BW
- refactor SSE and AVX to use Matrix functions, making old functions
call the new ones.
Zen5 1280x720
Was AVX2 LibYUVConvertTest.ARGBToI444_Opt (1125 ms)
Now AVX512 LibYUVConvertTest.ARGBToI444_Opt (641 ms)
Details by Gemini:
1. Created 3 new Matrix functions:
Added ARGBToYMatrixRow_SSSE3, ARGBToYMatrixRow_AVX2, and
ARGBToYMatrixRow_AVX512BW to source/row_gcc.cc. These take the
const struct ArgbConstants* c parameter similarly to
ARGBToUV444MatrixRow_*. The x86 vector instructions dynamically
calculate the needed values using the properties of the constants
struct, including using vpmaddwd inside the AVX512 code to offset
the lack of a native vphaddw.
2. Replaced Old Functions with Wrappers:
Modified the existing implementations of ARGBToYRow_SSSE3,
ARGBToYJRow_SSSE3, ABGRToYRow_SSSE3, ABGRToYJRow_SSSE3,
RGBAToYRow_SSSE3, RGBAToYJRow_SSSE3, BGRAToYRow_SSSE3 (and their
_AVX2 equivalents) in source/row_gcc.cc to act as inline wrappers
calling the new ARGBToYMatrixRow_* functions, passing the right
matrix parameters (e.g. &kArgbI601Constants, &kArgbJPEGConstants,
&kAbgrI601Constants).
3. Added row_any.cc Handlers:
Added ANY11MC definitions to source/row_any.cc to autogenerate
ARGBToYMatrixRow_Any_SSSE3, ARGBToYMatrixRow_Any_AVX2, and
ARGBToYMatrixRow_Any_AVX512BW which safely handles non-aligned
tails.
4. Updated include/libyuv/row.h:
Updated the headers with the proper void declarations for all newly
generated Matrix and Any_ variants. Also defined
HAS_ARGBTOYROW_AVX512BW in the CPU macros.
5. Tested the Implementations:
Compiled and tested on Linux x86, which resulted in all tests passing
cleanly. Also successfully completed all Windows 32-bit build checks
ensuring 32-bit regression prevention without issues.
Bug: 477295731
Change-Id: I4f5eec9a961e24a9d760d0a1c0810fb5e29a0bd1
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7759494
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Increases buffer sizes from 128 to 256 in ANY11, ANY11C, ANY11MC, ANY12,
and ANY12M macros to safely accommodate AVX512BW processing which can
write up to 256 bytes per operation.
Bug: libyuv:42280902, libyuv:502250231, 501882928
Change-Id: Icfba1982dc5fb6545255464f7decb2baec7be90f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7758060
Reviewed-by: James Zern <jzern@google.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
These are about 25% faster than the C versions.
Bug: libyuv:42280902
Change-Id: I8b298670ee5f3ed5db35527fc41d6d9a51b020a1
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7573682
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Mostly just a straightforward copy of the existing SVE2 code ported to
Streaming-SVE. Introduce new "any" kernels for non-multiple of two
cases, matching what we already do for SVE2.
The existing SVE2 code makes use of the Neon MOVI instruction that is
not supported in Streaming-SVE, so adjust the code to use FMOV instead
which has the same performance characteristics.
Change-Id: I74b7ea1fe8e6af75dfaf92826a4de775a1559f77
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/6663806
Reviewed-by: Justin Green <greenjustin@google.com>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
- Add +i8mm build option for sve ARGBToUV which uses usdot
- util/cpuid Get cpu count (windows, macos, linux)
- For each x86 cpu, detect hybrid (e-core)
- Includes a comment fix for ubsan unittest
- Bump version
- Apply clang format to util/*.c as well as all *.cc/*.h
Bug: 424637372
Change-Id: I08310e18051fff62c9e4e4a10d1e4361871119ac
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/6635640
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Add a Neon implementation of the Convert8To16Row kernel. Compared to the
C implementation we can take advantage of knowing that the "scale"
parameter is always an unsigned power of two and fits in 16-bits,
allowing us to combine this with the shift and avoid needing to widen
the input data.
Reduction in run times observed compared to the existing C
implementation:
Cortex-A55: -44.5%
Cortex-A510: -26.1%
Cortex-A520: -30.6%
Cortex-A76: -61.6%
Cortex-A710: -57.6%
Cortex-X1: -46.5%
Cortex-X2: -54.4%
Cortex-X3: -57.1%
Cortex-X4: -55.0%
Cortex-X925: -49.3%
Change-Id: I34b858605ece47e46588c0680a1d2afa7a90d7a0
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/6516186
Reviewed-by: Justin Green <greenjustin@google.com>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
The maximum coefficient is 128, so store constants negated to take
advantage of -128 being representable in 8-bit integers. This allows us
to use the I8MM USDOT instructions.
Reduction in time taken observed compared to the existing Neon
implementation, as a geomean of all ARGBToUV variants:
Cortex-A510: -7.1%
Cortex-A520: -2.1%
Cortex-A710: -8.4%
Cortex-A715: -0.3%
Cortex-A720: -0.3%
Cortex-X2: -40.0%
Cortex-X3: -43.3%
Cortex-X4: -11.3%
Cortex-X925: -2.5%
Change-Id: Id06dc17d101b66975b84b93e5abe91c0032921dd
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/6535686
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Justin Green <greenjustin@google.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
