Refactor CalcFrameSsim to decompose overlapping 8x8 SSIM windows
(stepped by 4 pixels horizontally and vertically) into non-overlapping
4x4 pixel tiles across a 2-row rotating buffer (row0, row1), reducing
memory reads and pixel accumulation work by 4x:
- Ssim4x4Row_* computes 4 uint32_t statistics {sum_a, sum_b, sum_sq_ab,
sum_axb} per 4x4 pixel tile across a 4-row strip (also sized to fit
future 10-bit/12-bit formats like I010 without uint32_t overflow).
- SsimWindowRow_C combines top and bottom 4x4 tile strips across
adjacent horizontal pairs and returns the exact double SSIM sum,
producing bit-identical results to Ssim8x8_C.
- CalcFrameSsim rotates row0 and row1 (with thread-local buffers and
continuity tracking under _OPENMP) and falls back to Ssim8x8_C only if
buffer allocation fails.
- Implement Ssim4x4Row across all SIMD architectures:
- x86/x64: Ssim4x4Row_SSE2, Ssim4x4Row_AVX2, and Ssim4x4Row_AVX512BW
(with k-masked tail handling for 4..60 pixels) in both
compare_gcc.cc and compare_win.cc.
- ARM: Ssim4x4Row_NEON (compare_neon.cc, compare_neon64.cc),
Ssim4x4Row_NEON_DotProd (compare_neon64.cc), Ssim4x4Row_SVE2
(row_sve.h, row_sve.cc), and Ssim4x4Row_SME (row_sme.cc).
- RISC-V: Ssim4x4Row_RVV (row_rvv.cc).
- LoongArch: Ssim4x4Row_LSX (row_lsx.cc) and Ssim4x4Row_LASX
(row_lasx.cc).
- WebAssembly: Ssim4x4Row_WASMSIMD (compare_wasm.cc).
- Enable BenchmarkSsim_Opt and add BenchmarkSsim_Unaligned and
TestSsimAnyWidth in unit_test/compare_test.cc.
R=rrwinterton@gmail.com
Bug: None
Test: out/Release/libyuv_unittest --gtest_filter=*Ssim*
Change-Id: Ibb95ae74424ecdda073ab7ab10ab1cf610b121b6
- Add HammingDistance_AVX512BW, SumSquareError_AVX512BW and
HashDjb2_AVX512BW for 32 and 64 bit, as inline assembly in
compare_gcc.cc and intrinsics in compare_win.cc.
- AVX512BW processes 64 bytes per loop with full vectors, and the
remainder of 1 to 63 bytes with one masked load, so compare.cc passes
any count and does not use C for the remainder. HashDjb2 loads the
remainder into the high bytes, ending at src + count, so the same
33 ^ (63 - i) multipliers apply, and multiplies the hash by 33 ^ n.
- Add SumSquareError_AVX2 and HashDjb2_AVX2 to compare_gcc.cc and
HammingDistance_AVX2 to compare_win.cc, so both files have the same
functions. Enable AVX2 in compare_gcc.cc for gcc as well as clang.
- HashDjb2 AVX2 and AVX512BW split the 32 bit multipliers 33 ^ i into
16 bit halves and use vpmaddwd, instead of vpmulld which is 2 uops
with 10 cycle latency. Bytes are zero extended with vpmovzxbw from
memory. The hash = hash * 33 ^ 64 + sum dependency uses scalar imul
instead of vpmulld. AVX2 does 64 bytes per horizontal add.
- Port compare_win.cc from 32 bit Visual C inline assembly to
intrinsics and enable it for Visual C 32/64 bit and
LIBYUV_ENABLE_ROWWIN, which MemorySanitizer also enables. Visual C 64
bit and MemorySanitizer previously used C.
- Fix HammingDistance_SSSE3 and HammingDistance_AVX2 to allow unaligned
pointers. They used movdqa and crashed on unaligned src_a.
- Add TestCompareAnyCount to test every count from 0 to 321, aligned
and unaligned, ending at the end of a page.
Functions available:
Function compare_gcc compare_win
32/64 bit 32/64 bit
HammingDistance_SSE42 asm intrinsics
HammingDistance_SSSE3 asm -
HammingDistance_AVX2 asm intrinsics
HammingDistance_AVX512BW asm intrinsics
SumSquareError_SSE2 asm intrinsics
SumSquareError_AVX2 asm intrinsics
SumSquareError_AVX512BW asm intrinsics
HashDjb2_SSE41 asm intrinsics
HashDjb2_AVX2 asm intrinsics
HashDjb2_AVX512BW asm intrinsics
Performance, 1280x720, LIBYUV_REPEAT=1000, best of 3, ms.
Before is main (SSE, and AVX2 Hamming only, in compare_gcc).
BenchmarkDjb2_Opt main gcc gcc win win
AVX2 AVX512 AVX2 AVX512
Intel Haswell 179 77 - 75 -
AMD Milan (Zen 3) 101 35 - 35 -
Intel Skylake-X 214 63 55 67 59
Intel Ice Lake 191 51 47 52 49
Intel Sapphire Rapids 175 39 39 38 39
Intel Granite Rapids 170 40 38 40 38
AMD Genoa (Zen 4) 91 34 37 34 38
AMD Turin (Zen 5) 80 29 22 28 23
C 502 to 669
BenchmarkSumSquareError main gcc gcc win win
_Opt AVX2 AVX512 AVX2 AVX512
Intel Haswell 79 37 - 38 -
AMD Milan (Zen 3) 47 24 - 25 -
Intel Skylake-X 78 32 26 34 24
Intel Ice Lake 86 43 23 43 22
Intel Sapphire Rapids 48 24 21 24 22
Intel Granite Rapids 47 23 21 23 21
AMD Genoa (Zen 4) 50 25 21 25 21
AMD Turin (Zen 5) 57 28 13 28 14
C 154 to 425
Testing:
LIBYUV_WIDTH=1279 LIBYUV_HEIGHT=719 out/Release/libyuv_unittest \
--gtest_filter='*Compare*:*Djb2*:*Hamming*:*SumSquare*:*Psnr*:*Ssim*'
With out/Release, out/Release_x86, out/rowwin, out/rowwin_x86 and
out/msan, with LIBYUV_DISABLE_AVX512BW and LIBYUV_DISABLE_AVX2, and
LIBYUV_WIDTH of 1, 31 to 33, 63 to 65, 95, 127, 129 and 200. Under
sde64 -skx for AVX512BW with out/Release and out/rowwin.
Built out/android_x86.
Bug: None
Change-Id: Ief197d546daca3db02ee1503d4649f4c755eb189
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8504717
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reduces include/libyuv/basic_types.h to only define LIBYUV_API.
Directly includes <stdint.h> and <stddef.h> where needed in public
headers.
Replaces LIBYUV_BOOL, LIBYUV_FALSE, and LIBYUV_TRUE with C++ bool,
true, and false in C++ code (MJpegDecoder, unit tests) and int (0, 1)
for C API functions.
Updates documentation and unit tests.
Test: ninja -C out/Release libyuv_unittest && ./out/Release/libyuv_unittest --gtest_filter=*
Change-Id: I2d58e872f0f976263bcda546dcc3b1b5f7e53d58
TAG=agy
CONV=2730bc56-93ff-48ed-b615-dcde1b834394
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280662
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
This is to resolve issues when the library is compiled with different
compiler and/or flags than the tests.
BUG=libyuv:836
Change-Id: I80727bfbd2fe1e02c842a7dba68a3deac941e23e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/1757114
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
H010ToAR30 uses Convert16To8Row_SSSE3 to convert 10 bit YUV to 8 bit.
Then standard YUV conversion can be used. This improves performance
on low end CPUs.
Future CL will by pass this conversion allowing for 10 bit YUV source,
but the function will be useful as a utility for YUV conversions.
Bug: libyuv:559, libyuv:751
Test: out/Release/libyuv_unittest --gtest_filter=*H010ToAR30* --libyuv_width=1280 --libyuv_height=720 --libyuv_repeat=999 --libyuv_flags=-1 --libyuv_cpu_info=-1
Change-Id: I9b3ef22d88a5fd861de4cf1900b4c6e8fd24d0af
Reviewed-on: https://chromium-review.googlesource.com/792334
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
popcnt has a fake dependency on the destination.
This assembly avoids the dependency by using a different
register for each popcnt.
Bug: libyuv:701
Test: LIBYUV_DISABLE_SSSE3=1 out/Release/libyuv_unittest --gtest_filter=*Ham*Opt --libyuv_width=1280 --libyuv_height=720 --libyuv_repeat=9999 --libyuv_flags=-1 --libyuv_cpu_info=-1
Change-Id: Ie1d202e2613b7fa8a3c02acd433940e92c80eafa
Reviewed-on: https://chromium-review.googlesource.com/731826
Reviewed-by: Cheng Wang <wangcheng@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
The low level hamming distance functions have size limitations
based on counter sizes. The higher level calls the low level
in blocks that avoid overflow and then accumulators in int64.
This test compares the results of the low levels to the high
level and against a known value (all ones) to ensure the
count is correct for any specified size.
The the size is very large, the result is expected to be
different.
Bug: libyuv:701
Test: TestHammingDistance_Opt
Change-Id: I6716af7cd09ac4d88a8afa25bc845a1b62af7c93
Reviewed-on: https://chromium-review.googlesource.com/710800
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
This reverts commit ec75df5894845b8d6b1341885a78db1de83decd8.
Reason for revert: <INSERT REASONING HERE>
Original change's description:
> ComputeHammingDistance reduce SIMD loop to 1 call when possible.
>
> 32 bit x86 has high overhead due to -fpic. So this reduces the
> number of calls by 1.
>
> TBR=kjellander@chromium.org
> Bug: libyuv:701
> Test: BenchmarkHammingDistance
> Change-Id: I7f557ef047920db65eab362a5f93abbd274ca051
> Reviewed-on: https://chromium-review.googlesource.com/701755
> Reviewed-by: Frank Barchard <fbarchard@google.com>
> Reviewed-by: Cheng Wang <wangcheng@google.com>
TBR=rrwinterton@gmail.com,fbarchard@google.com,wangcheng@google.com
Change-Id: Ia61e8558a8f083c14be5f51e0e141550b6f2b5c1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: libyuv:701
Reviewed-on: https://chromium-review.googlesource.com/707823
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
32 bit x86 has high overhead due to -fpic. So this reduces the
number of calls by 1.
TBR=kjellander@chromium.org
Bug: libyuv:701
Test: BenchmarkHammingDistance
Change-Id: I7f557ef047920db65eab362a5f93abbd274ca051
Reviewed-on: https://chromium-review.googlesource.com/701755
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: Cheng Wang <wangcheng@google.com>
If length of HammingDistance was not a multiple of 4,
the result was incorrect. The old tests did not catch this
so a new test is done to count 1s.
Bug: libyuv:740
Test: LibYUVCompareTest.TestHammingDistance
Change-Id: I93db5437821c597f1f162ac263d4a594bb83231f
Reviewed-on: https://chromium-review.googlesource.com/699614
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Reviewed-by: Cheng Wang <wangcheng@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
BUG=libyuv:701
TEST=built and disassembled for aarch64
R=kjellander@chromium.org
Change-Id: I7712b1c7934e5dfb55fda1fa7c8405c32d6964ce
Reviewed-on: https://chromium-review.googlesource.com/495327
Reviewed-by: Henrik Kjellander <kjellander@chromium.org>
Reviewed-by: Cheng Wang <wangcheng@google.com>
in order to compare C and Neon code, a new command line flag is added.
historically environment variables controlled cpu features, but on
android apk it is easier to pass a command line option to disable cpu
optimizations.
R=harryjin@google.com
BUG=libyuv:516
Review URL: https://codereview.chromium.org/1407193009 .
random / rand is slow and impacts performance testing.
Although its only called to clear a frame once, a typical profile shows
it high in the overall profile, when doing 1000 frames for a benchmark.
95.10% libyuv_unittest libyuv_unittest [.] YUY2ToARGBRow_SSSE3
2.01% libyuv_unittest libc-2.19.so [.] __random_r
1.13% libyuv_unittest libc-2.19.so [.] __random
Replace random is a faster version for unittests.
set LIBYUV_WIDTH=1280
set LIBYUV_HEIGHT=720
set LIBYUV_REPEAT=999
set LIBYUV_FLAGS=-1
out\release\libyuv_unittest --gtest_filter=*YUY2ToARGB* | findms
Was
libyuvTest.YUY2ToARGB_Opt (497 ms)
Now
libyuvTest.YUY2ToARGB_Opt (454 ms)
R=harryjin@google.com
BUG=none
Review URL: https://codereview.chromium.org/1361813002 .
