Clang MemorySanitizer builds define LIBYUV_ENABLE_ROWWIN and compile
row_win.cc and scale_win.cc instead of inline assembly. Two AVX-512
intrinsic patterns in those files are not modeled by LLVM MSan:
1. _mm512_mask_cvtepi32_storeu_epi8 and _mm512_mask_cvtepi64_storeu_epi8
lower to @llvm.x86.avx512.mask.pmov.db.mem.512 and
@llvm.x86.avx512.mask.pmov.qb.mem.512, which LLVM MSan handles as
unknown intrinsics without unpoisoning destination shadow memory.
Replace them in ARGBExtractAlphaRow_AVX512BW, ScaleAddCols1_AVX512BW,
and ScaleAddCols2_AVX512BW with _mm512_cvtepi32_epi8 /
_mm512_cvtepi64_epi8 followed by _mm_mask_storeu_epi8, which lowers
to @llvm.masked.store and still compiles to a single vpmovdb/vpmovqb
instruction.
2. ARGBCopyYToAlphaRow_AVX512BW loaded dst and blended the alpha channel
using _mm512_ternarylogic_epi32, which LLVM MSan does not model
bitwise and flags when any destination byte (such as the old alpha
channel) is uninitialized. Replace the load + vpternlogd + store in
both row_win.cc and row_gcc.cc with byte-masked stores
(_mm512_mask_storeu_epi8 / vmovdqu8 with mask 0x8888888888888888)
that write only the alpha bytes without reading dst.
Expand ARGBCopyAlpha, ARGBExtractAlpha, and ARGBCopyYToAlpha unit tests
into _Any, _Unaligned, _Invert, and _Opt variants and poison destination
buffers under MSan to catch unmodeled masked stores. Document MSan
AVX-512 intrinsic guidelines in GEMINI.md.
Test: out/msan/libyuv_unittest --gtest_filter=*Alpha*:*ScalePlaneBox_Any*
Bug: 578615529
Change-Id: I54ac4a5844aca6cfe3aa3445fa04b8221c610b35
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8672001
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Add AVX-512 VNNI (VPDPBUSD / _mm512_dpbusd_epi32) row kernels for 64-bit
x86 targets alongside existing AVX-512BW implementations in both
row_gcc.cc (inline assembly) and row_win.cc (C++ intrinsics):
- ARGBToYMatrixRow_AVX512VNNI and ARGBToYMatrixRow_Any_AVX512VNNI:
Pre-computes the 32-bit signed-pixel bias once per row and replaces
vpmaddubsw + vpmaddwd + vpaddw with a single vpdpbusd per 16 pixels.
- ARGBToUV444MatrixRow_AVX512VNNI and
ARGBToUV444MatrixRow_Any_AVX512VNNI: Loads 64 ARGB pixels once instead
of twice and computes 32-bit U and V dot products directly via
vpdpbusd with signed UV matrix coefficients.
- ARGBToUVMatrixRow_AVX512VNNI: Computes 2x2 box-filtered ARGB to U and
V conversion using vpdpbusd with k-masked tail handling for arbitrary
widths.
- ARGBGrayRow_AVX512VNNI: Computes BT.601 grayscale luminance using a
single vpdpbusd per 16 pixels, broadcasts Y to RGB via vpshufb, and
preserves alpha via vpternlogd with k-masked tail handling.
- Wire runtime dispatch via TestCpuFlag(kCpuHasAVX512VNNI) across
convert.cc, convert_from_argb.cc, and planar_functions.cc.
Add AVX512VNNIVBMI variants of the Y, UV444 and UV kernels. These use
both AVX-512 VNNI (vpdpbusd) and AVX-512 VBMI (vpermt2b) and are only
selected when the CPU has both features; there are no VBMI versions
without VNNI. The AVX512VNNI kernels remain for CPUs with VNNI but no
VBMI (Cascade Lake, Cooper Lake).
- The 32-bit sums are always in [0, 65535], so Y, U and V are byte 1 of
each int32. Two vpermt2b gather 64 results from 4 accumulators,
replacing vpackusdw/vpackssdw + vpsrlw + vpackuswb + vpermd.
- U and V accumulators start at ~kAddUV so they hold ~(kAddUV - dot).
The inverse is folded into the vpternlogd that merges the two
256-bit halves. kAddUV is loaded from ArgbConstants rather than
hard coded as 0x8000.
- Y folds the source loads into vpxorq.
- row_win.cc hides the vpermt2b indices from clang, which otherwise
constant folds them into a longer vpshufb + vpor + vpermd sequence.
Performance (1280x720, 5000 iterations, AVX512BW -> AVX512VNNI):
uArch / Benchmark AVX512BW AVX512VNNI Speedup
--------------------------- -------- ---------- -------
Intel Ice Lake ARGBToI420 941 ms 862 ms +9.2%
Intel Ice Lake ARGBToI422 1059 ms 947 ms +11.8%
Intel Ice Lake ARGBToI444 1140 ms 1038 ms +9.8%
Intel Ice Lake ARGBGray 357 ms 273 ms +30.8%
Intel Sapphire Rapids GrayTo 317 ms 218 ms +45.4%
Intel Emerald Rapids GrayTo 310 ms 213 ms +45.5%
Intel Granite Rapids GrayTo 349 ms 241 ms +44.8%
AMD Zen 4 ARGBToI420 648 ms 517 ms +25.3%
AMD Zen 4 ARGBToI444 695 ms 526 ms +32.1%
AMD Zen 4 ARGBGray 403 ms 227 ms +77.5%
AMD Zen 5 ARGBToI422 460 ms 403 ms +14.1%
AMD Zen 5 ARGBToI444 382 ms 321 ms +19.0%
Performance (row_gcc, AVX512VNNI -> AVX512VNNIVBMI):
uArch / Benchmark AVX512VNNI VNNIVBMI Speedup
---------------------------- ---------- -------- -------
AMD Zen 6 ARGBToI420 1280x720 327 ms 284 ms +15.1%
AMD Zen 6 ARGBToI422 1280x720 372 ms 330 ms +12.7%
AMD Zen 6 ARGBToI444 1280x720 357 ms 334 ms +6.9%
AMD Zen 6 ARGBToI400 256x24 94 ms 76 ms +23.7%
AMD Zen 5 ARGBToI422 1280x720 406 ms 350 ms +16.0%
AMD Zen 5 ARGBToI444 1280x720 330 ms 301 ms +9.6%
AMD Zen 5 ARGBToI400 256x24 100 ms 90 ms +11.1%
AMD Zen 4 ARGBToI420 1280x720 631 ms 540 ms +16.9%
AMD Zen 4 ARGBToI422 1280x720 993 ms 825 ms +20.4%
AMD Zen 4 ARGBToI400 256x24 245 ms 218 ms +12.4%
Intel GNR ARGBToI422 256x24 654 ms 616 ms +6.2%
Intel SPR ARGBToI422 256x24 653 ms 613 ms +6.5%
Intel GNR, EMR, SPR and Ice Lake are within +/-3% at 1280x720.
R=rrwinterton@gmail.com
Bug: none
Test: out/Release/libyuv_unittest --gtest_filter=*ARGBToI4*:*ARGBGray*
Test: LIBYUV_DISABLE_AVX512VBMI=1 out/Release/libyuv_unittest
Test: sde64 -gnr / -clx with is_msan=true (row_win.cc)
Change-Id: Ia7a37b39dd2948fe4055a8a7694dd1ff8ac3e55c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8647811
Reviewed-by: richard winterton <rrwinterton@gmail.com>
1275x1650 (US Letter 150 DPI, 1000 iterations) speedup:
CPU ARGBExtractAlpha ARGBCopyYToAlpha ARGBMirror RGB24Mirror
-----------------------------------------------------------------------
Intel Granite Rapids 1.00x 1.00x 1.00x 1.04x
Intel Emerald Rapids 1.01x 0.97x 1.03x 1.07x
Intel Sapphire Rapids 0.97x 1.01x 1.08x 1.26x
Intel Ice Lake 0.99x 1.02x 0.97x 1.08x
Intel Skylake-X 1.04x 1.03x 1.01x NA
Intel Haswell NA NA NA NA
AMD Turin (Zen 5) 1.01x 1.10x 1.25x 1.31x
AMD Genoa (Zen 4) 0.97x 1.04x 1.09x 1.42x
AMD Milan (Zen 3) NA NA NA NA
AMD Rome (Zen 2) NA NA NA NA
NVIDIA Grace (V2) 1.57x 2.51x NA NA
Google Axion 2 1.12x 3.17x NA NA
Ampere Siryn 1.06x 1.70x NA NA
ARM Neoverse-N1 1.49x 2.01x NA NA
Google Axion (Maple) 1.47x 2.03x NA NA
ARM Cortex-A53 1.06x 1.69x NA NA
Add AVX512BW ARGBExtractAlphaRow, ARGBCopyYToAlphaRow and
ARGBMirrorRow, and AVX512VBMI RGB24MirrorRow, in row_gcc.cc and as
intrinsics in row_win.cc. They are x64 only. The remainder is done
with a k-mask (bzhi) zero-masked load and a masked store, so they
handle any width without an Any wrapper and never access memory
outside the row.
RGB24MirrorRow_AVX512VBMI loads and stores 3 full zmm vectors per
64 pixels using 3 vpermi2b. The one output byte that comes from the
third input vector is merged with a masked vpbroadcastb. The 1 to 63
pixel tail rebases the same tables by 192 - 3 * r; indices >= 128 are
taken from the third vector with vpmovb2m and a masked vpermb.
Add ARGBCopyYToAlphaRow_NEON for Arm 32 bit (vld4 + vst4) and
AArch64 (zip + sli), and rewrite AArch64 ARGBExtractAlphaRow_NEON
with ldp + uzp2. Both prefetch source and destination, which is
2.5x faster on Neoverse-N1 and 1.2x to 1.5x faster on Cortex-A53
for 1280x720. Add WASM SIMD ARGBExtractAlphaRow and
ARGBCopyYToAlphaRow, using a swizzle table for CopyYToAlpha. Rework
the RVV versions to use unit stride loads with vnsrl and vzext.vf4.
R=rrwinterton@gmail.com
Bug: None
Change-Id: I436f27d70a3db0a2b176c91eb789a9491133cfcb
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8523531
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Fix _M_X86 to _M_IX86 for Visual C 32 bit
Visual C defines _M_IX86 for 32 bit x86, not _M_X86.
row.h and row_win.cc tested _M_X86, so a real 32 bit Visual C build
got neither USE_ROW_WIN nor USE_ROW_GCC, row_win.cc compiled to nothing,
and the HAS_*_AVX2 defines for all x86 platforms were skipped.
clang-cl and Android x86 are unaffected since clang defines __i386__.
Testing:
ninja -C out/Release libyuv_unittest && out/Release/libyuv_unittest
ninja -C out/Release_x86 libyuv_unittest && out/Release_x86/libyuv_unittest
ninja -C out/rowwin libyuv_unittest && out/rowwin/libyuv_unittest
blaze test --config=windows_x86 //third_party/libyuv:libyuv_test
blaze build --config=android_x86 //third_party/libyuv:libyuv
Bug: None
Change-Id: I0f90c03855f2981040ab4f250cd2facc9ef47ac0
TAG=agy
CONV=bea53b37-b2bf-46fa-9e44-a708b438fabc
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8499834
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: Mirko Bonadei <mbonadei@chromium.org>
I420ScaleTo569x480_Bilinear (LIBYUV_WIDTH=569 LIBYUV_REPEAT=5000):
CPU AVX2 AVX512 Speedup
Intel Emerald Rapids 124 ms 63 ms 1.97x
Intel Sapphire Rapids 134 ms 69 ms 1.94x
Intel Skylake-X 219 ms 129 ms 1.70x
Intel Granite Rapids 124 ms 76 ms 1.63x
Intel Ice Lake 164 ms 101 ms 1.62x
Bug: None
Change-Id: I7a10446eb4dbcebae68380c2a186d4e3badec551
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8483371
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
I422ToRGB24Matrix (299x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Skylake-X 275 ms 213 ms -22.5% (1.29x)
Intel Ice Lake 225 ms 177 ms -21.3% (1.27x)
Intel Sapphire Rapids 181 ms 144 ms -20.4% (1.26x)
Intel Emerald Rapids 167 ms 136 ms -18.6% (1.23x)
Intel Granite Rapids 170 ms 140 ms -17.6% (1.21x)
AMD Zen 4 (Genoa) 192 ms 136 ms -29.2% (1.41x)
AMD Zen 5 (Turin) 102 ms 68 ms -33.3% (1.50x)
AMD Zen 6 (Venice) 102 ms 60 ms -41.2% (1.70x)
====================================================================
I422ToRGB24Matrix (298x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Skylake-X 271 ms 184 ms -32.1% (1.47x)
Intel Ice Lake 235 ms 152 ms -35.3% (1.55x)
Intel Sapphire Rapids 199 ms 128 ms -35.7% (1.55x)
Intel Emerald Rapids 170 ms 121 ms -28.8% (1.40x)
Intel Granite Rapids 177 ms 125 ms -29.4% (1.42x)
AMD Zen 4 (Genoa) 185 ms 119 ms -35.7% (1.55x)
AMD Zen 5 (Turin) 102 ms 59 ms -42.2% (1.73x)
AMD Zen 6 (Venice) 100 ms 51 ms -49.0% (1.96x)
====================================================================
Bug: None
Change-Id: Id75e3d1df6d26bc28a252672816a7c438cb664a4
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8452502
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Add SwapUVRow_AVX2 implementation in row_win.cc for MSVC and enable
HAS_SWAPUVROW_AVX2 under USE_ROW_WIN.
Share kShuffleUVToVU shuffle table between SSSE3 and AVX2 in row_gcc.cc.
Make NV12ToNV21 the primary implementation and alias NV21ToNV12 to it.
Add NV12ToNV21 test in convert_test.cc and benchmark dispatch for
SSSE3 and AVX2 in planar_test.cc.
CPU Before After Speedup
----------------------------------------------------------------
Intel Granite Rapids 282 ms 125 ms 2.26x
Intel Emerald Rapids 264 ms 108 ms 2.44x
Intel Sapphire Rapids 276 ms 122 ms 2.26x
Intel Ice Lake 310 ms 122 ms 2.54x
Intel Skylake-X 573 ms 157 ms 3.65x
Intel Haswell 620 ms 162 ms 3.83x
AMD Zen 4 (Genoa) 266 ms 52 ms 5.12x
AMD Zen 6 (Venice) 229 ms 43 ms 5.33x
ARM Neoverse-N1 140 ms 71 ms 1.97x
NVIDIA Grace (Neoverse-V2) 62 ms 69 ms 0.90x
R=wtc@google.com
Test: libyuv_unittest --gtest_filter=*SwapUV*:*NV12ToNV21*:*NV21ToNV12*
Bug: 406251841
Change-Id: I9bed5522aedadea13cdf69c27e3cbe633e31dd5b
TAG=agy
CONV=bb50ac62-0c55-4969-8fff-c71ab3d9a79f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8356968
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- Also remove redundant neon check for I8MM kernels
======================================================================
CPU Before After Speedup
----------------------------------------------------------------------
Intel Granite Rapids (AVX10.1) 251 ms 141 ms 1.78x (+43.8%)
Intel Emerald Rapids (AVX512) 199 ms 108 ms 1.84x (+45.7%)
Intel Sapphire Rapids (AVX512) 230 ms 112 ms 2.05x (+51.3%)
AMD Turin Zen 5 (AVX512) 156 ms 99 ms 1.58x (+36.5%)
AMD Venice Zen 6 (AVX512) 204 ms 106 ms 1.92x (+48.0%)
Intel Skylake-X (AVX512) 320 ms 156 ms 2.05x (+51.2%)
AMD Genoa Zen 4 (AVX512) 236 ms 346 ms 0.68x (-46.6%)
Intel Ice Lake (AVX512) 242 ms 316 ms 0.77x (-30.6%)
AMD Milan Zen 3 (AVX2) 252 ms 230 ms 1.10x (+8.7%)
Intel Haswell (AVX2) 369 ms 456 ms 0.81x (-23.6%)
NVIDIA Grace (SVE2) 336 ms 337 ms 1.00x (-0.3%)
ARM Neoverse-N1 (NEON) 1081 ms 1105 ms 0.98x (-2.2%)
======================================================================
Bug: 537818862
Change-Id: I3ddeddb040fc52a30dabe7ae2c71d16dda021568
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8343467
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- NV16/24 FOURCC and document
- Rotate take uint8* for ubsan
Bug: 537818862
Change-Id: I2bc7109abaffe2464e031e33c22296c007077d82
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8339427
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- Add avx512bw, neon 32 bit and RVV versions
- Adapt SSE2 to psrlw instead of pmulhuw
CPU Before After Improvement
---------------------------------------------------------------
Granite Rapids (AVX10) 77 ms 75 ms +2.6% (1.03x)
Emerald Rapids (AVX512) 75 ms 67 ms +10.7% (1.12x)
Sapphire Rapids (AVX512) 91 ms 85 ms +6.6% (1.07x)
AMD Venice Zen 6 (AVX512) 30 ms 30 ms 0.0% (1.00x)
AMD Turin Zen 5 (AVX512) 32 ms 32 ms 0.0% (1.00x)
AMD Genoa Zen 4 (AVX512) 35 ms 36 ms -2.9% (0.97x)
Skylake-X (AVX512) 119 ms 119 ms 0.0% (1.00x)
Ice Lake (AVX512) 93 ms 104 ms -11.8% (0.89x)
AMD Milan Zen 3 (AVX2) 43 ms 44 ms -2.3% (0.98x)
Haswell (AVX2) 129 ms 138 ms -7.0% (0.93x)
NVIDIA Grace (SVE2) 42 ms 43 ms -2.4% (0.98x)
Neoverse-N1 (NEON) 87 ms 87 ms 0.0% (1.00x)
---------------------------------------------------------------
Bug: None
Change-Id: Ia8f310a70c8b1229927fcdf117fe22e131d70ea9
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8330912
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
CPU / Architecture Before (C) After (NEON) Improvement
----------------------------------------------------------------------
Cortex-A53 (Pixel 2 XL Little) 3248 ms 792 ms 4.10x
Cortex-A73 (Pixel 2 XL Big) 432 ms 392 ms 1.10x
Cortex-A55 (Pixel 7 Little) 780 ms 563 ms 1.39x
Cortex-A78 (Pixel 7 Mid) 254 ms 170 ms 1.49x
Cortex-X1 (Pixel 7 Big) 158 ms 129 ms 1.22x
Cortex-A510 (Galaxy S22 Little) 816 ms 441 ms 1.85x
Cortex-A710 (Galaxy S22 Mid) 270 ms 129 ms 2.09x
Cortex-X2 (Galaxy S22 Big) 116 ms 80 ms 1.45x
Cortex-A510 (Galaxy S23 Little) 723 ms 373 ms 1.94x
Cortex-A715 (Galaxy S23 Mid) 183 ms 119 ms 1.54x
Cortex-X3 (Galaxy S23 Big) 122 ms 72 ms 1.69x
Lumix C1 Pro (PD2509 Little) 246 ms 156 ms 1.58x
Lumix C1 Premium (PD2509 Mid) 211 ms 112 ms 1.88x
Lumix C1 Ultra (PD2509 Big) 82 ms 50 ms 1.64x
Oryon-v3M (OnePlus Mid) 233 ms 150 ms 1.55x
Oryon-v3L (OnePlus Big) 103 ms 85 ms 1.21x
On Icelake
Function (1280x720) C SSSE3 AVX2 AVX-512BW Gain
----------------------------------------------------------------------
BlendPlane 901 ms 165 ms 135 ms 115 ms 1.17x
I420Blend 1398 ms 287 ms 198 ms 187 ms 1.06x
Bug: 546077674
Change-Id: I189c44cc757c0e26838a7b6fdb2db1cbe5f96f20
TAG=agy
CONV=0f82c25c-616e-4bb3-beeb-981decb0f40c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8297264
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
CPU / Arch Before (Kernel) After (Kernel) Delta
----------------------------------------------------------------------
Haswell (x86_64) 263 ms (SSSE3) 263 ms (AVX2) 0.0%
Milan (Zen 3) 115 ms (SSSE3) 138 ms (AVX2) -20.0%
SapphRap (x86_64) 173 ms (SSSE3) 178 ms (AVX512VBMI) -2.9%
GNR (Granite R) 191 ms (SSSE3) 189 ms (AVX512VBMI) +1.0%
Turin (Zen 5) 69 ms (SSSE3) 55 ms (AVX512VBMI) +20.3%
Icelake (x86_64) 204 ms (SSSE3) 186 ms (AVX512VBMI) +8.8%
Cascade Lake 237 ms (SSSE3) 229 ms (AVX512BW) +3.4%
Neoverse (aarch64)* 103 ms (SVE2) 103 ms (SVE2) 0.0%
Bug: None
Change-Id: Ib218632e16b3308a214e9ffcbc745d6c41116c6e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280536
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
The READYUV444, READYUVA444, READYUVA422, and STOREARGB macros are
defined but not used.
Change-Id: Ic1174b5b9c7dc8554f362871de73080c32a58537
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8279223
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
- reduced shifts to isolate 10 bits
- permi2w instead of packlwd to combine words (ARG) (GB) into ARGB
- vpternlogd to combine A,R,G into 1 register
Saphhire Rapids 10000 iterations of 1280x720P:
Conversion Kernel │ Was │ Now │ Speedup
──────────────────────┼────────────────────────┼────────────────
I420ToAR30_Opt │ 1675 ms │ 1568 ms │ +7.1%
H420ToAR30_Opt │ 1632 ms │ 1555 ms │ +5.2%
I210ToAR30_Opt │ 2459 ms │ 2167 ms │ +13.5%
P010ToAR30_Opt │ 2054 ms │ 1926 ms │ +6.6%
P016ToAR30_Opt │ 2043 ms │ 1938 ms │ +5.4%
Total Test Suite Time │ 9,866 ms │ 9,155 ms │ +7.8% overall
Bug: None
Change-Id: I7972f98408a65a2bb51c506145e4c0c63824c424
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8259796
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- improves I420ToRAW performance
I420ToRAW on Skylake
Was AVX2
I420ToRAW_Opt (262 ms)
Now AVX512
I420ToRAW_Opt (212 ms)
I420ToRAW on Icelake with avx512vbmi
Was I420ToRAW_Opt (200 ms)
Now I420ToRAW_Opt (187 ms)
Bug: None
Change-Id: If5b171801ba931557d379696fc128d9b155f7c44
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8267746
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- Implemented `I422ToRGB24Row_AVX2` and `I422ToRGB24Row_AVX512VBMI` in:
- `row_gcc.cc`: Inline assembly for GCC/Clang.
- `row_win.cc`: C++ intrinsics for MSVC (also verified with Clang).
Reduced width alignment requirement: changed from 32-pixel to 16-pixel
alignment in `convert_argb.cc` and `row_any.cc`. This allows the AVX2
path to be used for more common video resolutions.
```
I420ToRAW vs Rust on Icelake Xeon
Size I420ToRAW yuv420_to_rgb iterations
------- --------- --------------- ----------
640x480 57 us 77.377 µs/iter 3000
1280x720 170 us 221.671 µs/iter 1000
1920x1080 396 us 494.324 µs/iter 500
3840x2160 2040 us 2357 µs/iter 200
```
Bug: 42280902
Change-Id: I07c0505c95410ea16a6218c858844791a11ef073
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7908323
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Remove __attribute__((no_sanitize("cfi-icall"))) from
ARGBToUVMatrixRow_AVX2(). This breaks MSVC compilation, and no other
libyuv function is marked with this attribute.
Change-Id: I2bb6a688e296dd4acff325c5bd750573a577f246
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7904777
Reviewed-by: Frank Barchard <fbarchard@google.com>
Was C
LibYUVConvertTest.ARGBToI444_Opt (1027 ms)
Now AVX2
LibYUVConvertTest.ARGBToI444_Opt (310 ms)
Bug: libyuv:508639302
Change-Id: I0bc7f5c5b72160d24226a98d5fddb184a004ed00
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7841655
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- remove inline asm which was only for 32 bit
- add ARGBToYMatrixRow_AVX2
- add gn flag libyuv_enable_rowwin=true
Example of building with GN and Ninja:
Without the new flag:
gn gen out/Release "--args=is_debug=false"
ninja -C out/Release
With the new flag:
gn gen out/Release "--args=is_debug=false libyuv_enable_rowwin=true"
ninja -C out/Release
Bug: libyuv:42280806, 477295731, libyuv:42280902, libyuv:439628764
R=dalecurtis@chromium.org, rrwinterton@gmail.com
Change-Id: I451bf814622fba690005c02fbf5816819c6a08c2
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7765790
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
- MCA says old version was 4 cycles and new version is 2.5 cycles/loop
- lunarlake is the only known cpu
mca -mcpu=lunarlake 100 iterations
Was vpmulhu
Iterations: 100
Instructions: 1200
Total Cycles: 426
Total uOps: 1200
Dispatch Width: 8
uOps Per Cycle: 2.82
IPC: 2.82
Block RThroughput: 4.0
Now vpsrlw
Iterations: 100
Instructions: 1200
Total Cycles: 279
Total uOps: 1400
Dispatch Width: 8
uOps Per Cycle: 5.02
IPC: 4.30
Block RThroughput: 2.5
Bug: None
Change-Id: I5a49e1cf1ed3dfb59fe9861a871df9862417c6a6
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/6697745
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- Was using avgb twice for non-exact and C for exact.
On Skylake Xeon:
Now SSE3
ARGBToJ420_Opt (326 ms)
Was
Exact C
ARGBToJ420_Opt (871 ms)
Not exact AVX2
ARGBToJ420_Opt (237 ms)
Not exact SSSE3
ARGBToJ420_Opt (312 ms)
Bug: 381138208
Change-Id: I6d1081bb52e36f06736c0c6575fa82bb2268629b
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/6629605
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Ben Weiss <bweiss@google.com>
- use negative coefficients for UV to allow -128
- change shift to truncate instead of round for UV
- adapt all row_gcc RGB to UV into matrix functions
- add -DLIBYUV_ENABLE_ROWWIN to allow clang on Windows to use row_win.cc
Bug: 381138208
Change-Id: I6016062c859faf147a8a2cdea6c09976cbf2963c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/6277710
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: James Zern <jzern@google.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
include intrin.h rather than emmintrin.h; fixes:
C:\...\VC\Tools\MSVC\14.35.32215\include\emmintrin.h(28,1):
fatal error C1189: #error: this header should only be included through
Change-Id: Ief9c81f6f1971e552c8aac301d678b64fe5bd7cc
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/4513825
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
- YUV to RGB use linear for first and last row.
- add assert(yuvconstants)
- rename pointers to match row functions.
- use macros that match row functions.
- use 12 bit upsampler for conversions of 10 and 12 bits
Cortex A53 AArch32
I420ToRGB24_Opt (3627 ms)
I422ToRGB24_Opt (4099 ms)
I444ToRGB24_Opt (4186 ms)
I420ToRGB24Filter_Opt (5451 ms)
I422ToRGB24Filter_Opt (5430 ms)
AVX2
Was I420ToRGB24Filter_Opt (583 ms)
Now I420ToRGB24Filter_Opt (560 ms)
Neon Cortex A7
Was I420ToRGB24Filter_Opt (5447 ms)
Now I420ToRGB24Filter_Opt (5439 ms)
Bug: libyuv:938
Change-Id: I1731f2dd591073ae11a756f06574103ba0f803c7
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/3906082
Reviewed-by: Justin Green <greenjustin@google.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Fix the following MSVC warnings:
src\source\row_win.cc(117): warning C4309: 'argument': truncation of
constant value
src\source\row_win.cc(136): warning C4309: 'argument': truncation of
constant value
src\source\row_win.cc(155): warning C4309: 'argument': truncation of
constant value
src\source\row_win.cc(174): warning C4309: 'argument': truncation of
constant value
src\source\row_common.cc(1712): warning C4244: 'initializing':
conversion from 'uint16_t' to 'int8_t', possible loss of data
src\source\row_common.cc(1731): warning C4244: 'initializing':
conversion from 'int16_t' to 'int8_t', possible loss of data
src\source\row_common.cc(1786): warning C4244: 'initializing':
conversion from 'uint16_t' to 'int8_t', possible loss of data
src\source\row_common.cc(1805): warning C4244: 'initializing':
conversion from 'uint16_t' to 'int8_t', possible loss of data
Bug: libyuv:939
Change-Id: Ie87ba6e716732d1ff1ae5c236dfd9cfdac13439d
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/3807105
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
- swap U and V when crop x is odd
- document YUY2 and UYVY formats
- apply clang-format
Bug: libyuv:902
Change-Id: I045e44c907f4a9eb625d7c024b669bb308055f32
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/3039549
Reviewed-by: Mirko Bonadei <mbonadei@chromium.org>
Add tests of all macros used by libyuv public headers
When a 1 step conversion is added, a 2 step test can compare
the old 2 step method to the 1 step. A 1 step unittest is
also added which compares C to SIMD. Making the 2 step
conversions measure performance of the 2 steps allows the
old 2 step performance to be compared to 1 step.
All macros used in public headers are added to an ifdef test.
Showing them in a unittest allows some diagnostics when
a test is failing.
Bug: libyuv:901
Change-Id: I7ffa6ed0cb3b506fa1b7fd4b7b1b729658c3c266
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/2857916
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Mirko Bonadei <mbonadei@chromium.org>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
