I422ToRGB24Matrix (299x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Skylake-X 275 ms 213 ms -22.5% (1.29x)
Intel Ice Lake 225 ms 177 ms -21.3% (1.27x)
Intel Sapphire Rapids 181 ms 144 ms -20.4% (1.26x)
Intel Emerald Rapids 167 ms 136 ms -18.6% (1.23x)
Intel Granite Rapids 170 ms 140 ms -17.6% (1.21x)
AMD Zen 4 (Genoa) 192 ms 136 ms -29.2% (1.41x)
AMD Zen 5 (Turin) 102 ms 68 ms -33.3% (1.50x)
AMD Zen 6 (Venice) 102 ms 60 ms -41.2% (1.70x)
====================================================================
I422ToRGB24Matrix (298x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Skylake-X 271 ms 184 ms -32.1% (1.47x)
Intel Ice Lake 235 ms 152 ms -35.3% (1.55x)
Intel Sapphire Rapids 199 ms 128 ms -35.7% (1.55x)
Intel Emerald Rapids 170 ms 121 ms -28.8% (1.40x)
Intel Granite Rapids 177 ms 125 ms -29.4% (1.42x)
AMD Zen 4 (Genoa) 185 ms 119 ms -35.7% (1.55x)
AMD Zen 5 (Turin) 102 ms 59 ms -42.2% (1.73x)
AMD Zen 6 (Venice) 100 ms 51 ms -49.0% (1.96x)
====================================================================
Bug: None
Change-Id: Id75e3d1df6d26bc28a252672816a7c438cb664a4
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8452502
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
I422ToRGB24Matrix (299x299, repeat=10000)
====================================================================
CPU Before After Improvement
--------------------------------------------------------------------
Intel Ice Lake 220 ms 181 ms -17.7% (1.22x)
Intel Sapphire Rapids 181 ms 145 ms -19.9% (1.25x)
Intel Emerald Rapids 169 ms 148 ms -12.4% (1.14x)
Intel Granite Rapids 191 ms 141 ms -26.2% (1.35x)
AMD Zen 4 (Genoa) 171 ms 136 ms -20.5% (1.26x)
AMD Zen 5 (Turin) 94 ms 72 ms -23.4% (1.31x)
AMD Zen 6 (Venice) 95 ms 66 ms -30.5% (1.44x)
====================================================================
Bug: None
Change-Id: I4cd657410cc5253b5e8fd2caa78d17e2b3812393
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8445804
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
P010 and P210 were auxiliary FourCCs and P410 did not exist, so
ConvertToI420, ConvertToARGB, and ConvertFromI420 could not consume or
produce them. Callers had to pick I010/I210 helpers and pack UV by hand.
NV12Scale existed, but NV16/NV24 and P010/P210/P410 had no Scale APIs.
Promote P010 and P210 to primary FourCCs, add FOURCC_P410, and add the
I4xx/Px10 converters those entry points need: I410ToP410, I412ToP412,
P210ToI210, P410ToI410, P010/P210/P410ToI420, I420ToP010/P210/P410,
I422ToP210, I444ToP410, and P410ToARGB/AR30Matrix.
Add NV16Scale (same path as NV12Scale: ScalePlane + UVScale).
Add P010Scale, P210Scale, and P410Scale: 10-bit samples stay in the MSBs, so Y/UV convert to 10-bit LSB, ScalePlane_16, then back to MSB.
UVScale_16 cannot general-scale, so UV is split, scaled as planes, and merged.
Packed-plane layout, odd-width strides, ConvertTo/From round-trips, and
the existing scale factor/size/swap tests cover the new APIs.
Bug: 537818862
Change-Id: I1ae5e6cad5e6d83acf1d0df69f94c23688bbe0e8
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8346569
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Add matrix and BT.601 wrappers that convert ARGB to 16-bit biplanar
YUV in P010 (4:2:0), P210 (4:2:2), and P410 (4:4:4) layouts.
Reuse optimized ARGB matrix row functions for 8-bit Y and chroma,
merge UV with MergeUVRow, then expand with Convert8To16Row (bits=10)
and MultiplyRow_16 (scale=64) so 10-bit samples are stored in the
high bits of each uint16.
Reject width > INT_MAX / 2 so UV width (halfwidth * 2 or width * 2)
does not overflow Convert8To16Row.
Bug: 537818862
Change-Id: I7a776fb3d6f52ce46a0f6825903bfdb9c6dcfbe2
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8314189
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
- avoids intrinsics API changes
- build options for rvv not needed for inline instructions
- declare vl and vtype clobbers instead of cc
- tune for SiFive X280 using 16/32-bit loads and stores instead of
vlseg/vsseg where more efficient
- tune LMUL for SpacemiT K3 (256-bit vectors) with max m4 (m2 where
constrained by register pressure or vlseg/vsseg field rules)
- YUVTORGB use vmulhu
NV12ToARGB performance estimates:
CPU VLEN C Intrinsics Inline ASM Speedup (Inline)
------------------------------------------------------------------------
SiFive X390 1024-bit 360 ms 29 ms 9.5 ms 37.89x (3.05x)
SiFive X280 512-bit 980 ms 115 ms 38.0 ms 25.79x (3.03x)
SpacemiT K3 256-bit 420 ms 68 ms 28.0 ms 15.00x (2.43x)
Google Coral 256-bit 1120 ms 190 ms 58.0 ms 19.31x (3.28x)
T-Head C920 128-bit 510 ms 88 ms 39.0 ms 13.08x (2.26x)
T-Head C908 128-bit 860 ms 145 ms 62.0 ms 13.87x (2.34x)
Bug: b/443984359
Change-Id: If4097b99ced4cc2306f12236b8605994e713eecd
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8426529
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- AVX2 uses vpmulhrsw to compute ((128 * a * b) + 16384) >> 15.
- RVV uses e8, m2 (widening to e16, m4) with vwmulu.vv, vwaddu.wx 128,
and vnsrl.wi by 8.
CPU C Intrinsics Inline ASM Speedup (Inline)
------------------------------------------------------------------------
SiFive X390 gen 2 (est.) 2150 ms 29 ms 27 ms 79.63x (1.07x)
SpacemiT K3 (est.) 980 ms 41 ms 38 ms 25.79x (1.08x)
T-Head C908 (est.) 2060 ms 100 ms 94 ms 21.91x (1.06x)
Google Coral (est.) 2680 ms 94 ms 88 ms 30.45x (1.07x)
Test: libyuv_unittest --gtest_filter=*ARGBMultiply*
Bug: None
Change-Id: Iff65794623a6f29e5cb6bda7bd3084c8a5f676d1
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8422278
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Apple processors (such as A18/A19) may report
hw.optional.arm.FEAT_SME = 1 even when the XNU kernel on iOS disables
EL0 (user-mode) SME execution (hw.optional.arm.sme_max_svl_b == 0),
which causes SMSTART or RDSVL to fault with EXC_BAD_INSTRUCTION.
Gate SME/SME2 flags on have_feature("hw.optional.arm.sme_max_svl_b")
being nonzero. See https://github.com/pytorch/cpuinfo/issues/432 and
https://github.com/pytorch/cpuinfo/pull/433.
Also add SME2 detection test in cpu_test.cc and LIBYUV_DISABLE_SME2
environment variable support in unit_test.cc.
Test: libyuv_unittest --gtest_filter=*TestLinuxAArch64*
Bug: 8339302
Change-Id: If3e4d425ebaa89bf27965295ebdb43b844797ca1
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8428226
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Add SwapUVRow_RVV using RVV intrinsics by default and unit-stride
e16, m4 inline assembly (vle16.v, vsrl.vi, vsll.vi, vor.vv, vse16.v)
under #ifdef RVV_ASM.
CPU C Intrinsics Inline ASM Speedup (Inline)
--------------------------------------------------------------------
SiFive X280 (est.) 980 ms 115 ms 38 ms 25.79x (3.03x)
SpacemiT K3 (est.) 420 ms 68 ms 28 ms 15.00x (2.43x)
T-Head C908 (est.) 860 ms 145 ms 62 ms 13.87x (2.34x)
Google Coral (est.) 1120 ms 190 ms 58 ms 19.31x (3.28x)
Original author: Matteo Forzan <matteo.forzan95@gmail.com>
R=wtc@google.comCC=matteo.forzan95@gmail.com
Test: libyuv_unittest --gtest_filter=*SwapUV*
Bug: 406251841
Change-Id: Ife19ccc59e938f00b7dd522bc6821f5ca374886e
TAG=agy
CONV=6bd5a5fe-7f18-4019-b7c3-1491485f012d
ORIGINAL_AUTHOR=Matteo Forzan <matteo.forzan95@gmail.com>
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8339302
Reviewed-by: David Gao <davidgao@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
The existing SVE2 path only computes the average for a pair of input
values instead of groups of four as done in the Neon implementation.
The average of two values must be retained for the final odd element if
present. Existing code just handles this by falling back to the C
implementation, so mirror that for SVE2 as well.
Reduction in time taken relative to the existing Neon code, the SVE2
code remains faster even with the correct averaging:
Cortex-A510: -12.66%
Lumex C1-Nano: -6.38%
Cortex-A720: -9.73%
Lumex C1-Pro: +0.99%
Cortex-X4: -3.05%
Lumex C1-Ultra: -18.98%
Bug: 546962730
Change-Id: I7ab348e64a9d079b81d070cea5fd575243b037d2
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8252281
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Justin Green <greenjustin@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Add SwapUVRow_AVX2 implementation in row_win.cc for MSVC and enable
HAS_SWAPUVROW_AVX2 under USE_ROW_WIN.
Share kShuffleUVToVU shuffle table between SSSE3 and AVX2 in row_gcc.cc.
Make NV12ToNV21 the primary implementation and alias NV21ToNV12 to it.
Add NV12ToNV21 test in convert_test.cc and benchmark dispatch for
SSSE3 and AVX2 in planar_test.cc.
CPU Before After Speedup
----------------------------------------------------------------
Intel Granite Rapids 282 ms 125 ms 2.26x
Intel Emerald Rapids 264 ms 108 ms 2.44x
Intel Sapphire Rapids 276 ms 122 ms 2.26x
Intel Ice Lake 310 ms 122 ms 2.54x
Intel Skylake-X 573 ms 157 ms 3.65x
Intel Haswell 620 ms 162 ms 3.83x
AMD Zen 4 (Genoa) 266 ms 52 ms 5.12x
AMD Zen 6 (Venice) 229 ms 43 ms 5.33x
ARM Neoverse-N1 140 ms 71 ms 1.97x
NVIDIA Grace (Neoverse-V2) 62 ms 69 ms 0.90x
R=wtc@google.com
Test: libyuv_unittest --gtest_filter=*SwapUV*:*NV12ToNV21*:*NV21ToNV12*
Bug: 406251841
Change-Id: I9bed5522aedadea13cdf69c27e3cbe633e31dd5b
TAG=agy
CONV=bb50ac62-0c55-4969-8fff-c71ab3d9a79f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8356968
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- instead of 64 bit load, use vlseg2 with 32 bit
Test build with:
-fsanitize=alignment -fsanitize-trap=alignment
Test: libyuv_unittest --gtest_filter=*UVScale*:*ScalePlane*:*ARGBScale*
Bug: 553156658
Change-Id: Ib6febe49f085f5b3deb33adb6b6f82f7d836c3fa
TAG=agy
CONV=b29f62f9-15a4-4f38-8414-86c8330e61fb
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8315435
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- Also remove redundant neon check for I8MM kernels
======================================================================
CPU Before After Speedup
----------------------------------------------------------------------
Intel Granite Rapids (AVX10.1) 251 ms 141 ms 1.78x (+43.8%)
Intel Emerald Rapids (AVX512) 199 ms 108 ms 1.84x (+45.7%)
Intel Sapphire Rapids (AVX512) 230 ms 112 ms 2.05x (+51.3%)
AMD Turin Zen 5 (AVX512) 156 ms 99 ms 1.58x (+36.5%)
AMD Venice Zen 6 (AVX512) 204 ms 106 ms 1.92x (+48.0%)
Intel Skylake-X (AVX512) 320 ms 156 ms 2.05x (+51.2%)
AMD Genoa Zen 4 (AVX512) 236 ms 346 ms 0.68x (-46.6%)
Intel Ice Lake (AVX512) 242 ms 316 ms 0.77x (-30.6%)
AMD Milan Zen 3 (AVX2) 252 ms 230 ms 1.10x (+8.7%)
Intel Haswell (AVX2) 369 ms 456 ms 0.81x (-23.6%)
NVIDIA Grace (SVE2) 336 ms 337 ms 1.00x (-0.3%)
ARM Neoverse-N1 (NEON) 1081 ms 1105 ms 0.98x (-2.2%)
======================================================================
Bug: 537818862
Change-Id: I3ddeddb040fc52a30dabe7ae2c71d16dda021568
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8343467
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- NV16/24 FOURCC and document
- Rotate take uint8* for ubsan
Bug: 537818862
Change-Id: I2bc7109abaffe2464e031e33c22296c007077d82
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8339427
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Add matrix and BT.601 wrapper APIs for converting ARGB to NV16 and
NV24. Reuse the optimized ARGB Y and UV row functions, then merge
the planar chroma rows into the interleaved destination plane.
NV16 performs horizontal 2:1 chroma subsampling on every source row.
NV24 preserves full-resolution 4:4:4 chroma. Both paths support odd
widths, negative heights, and the existing matrix constants.
Add tests that compare scalar and optimized paths at an odd width and
height through the matrix API and its BT.601 wrapper.
Bug: 537818862
Change-Id: Iddefc714a800cdec4dd1394ac7a52bf7b51fd350
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8314188
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- Add avx512bw, neon 32 bit and RVV versions
- Adapt SSE2 to psrlw instead of pmulhuw
CPU Before After Improvement
---------------------------------------------------------------
Granite Rapids (AVX10) 77 ms 75 ms +2.6% (1.03x)
Emerald Rapids (AVX512) 75 ms 67 ms +10.7% (1.12x)
Sapphire Rapids (AVX512) 91 ms 85 ms +6.6% (1.07x)
AMD Venice Zen 6 (AVX512) 30 ms 30 ms 0.0% (1.00x)
AMD Turin Zen 5 (AVX512) 32 ms 32 ms 0.0% (1.00x)
AMD Genoa Zen 4 (AVX512) 35 ms 36 ms -2.9% (0.97x)
Skylake-X (AVX512) 119 ms 119 ms 0.0% (1.00x)
Ice Lake (AVX512) 93 ms 104 ms -11.8% (0.89x)
AMD Milan Zen 3 (AVX2) 43 ms 44 ms -2.3% (0.98x)
Haswell (AVX2) 129 ms 138 ms -7.0% (0.93x)
NVIDIA Grace (SVE2) 42 ms 43 ms -2.4% (0.98x)
Neoverse-N1 (NEON) 87 ms 87 ms 0.0% (1.00x)
---------------------------------------------------------------
Bug: None
Change-Id: Ia8f310a70c8b1229927fcdf117fe22e131d70ea9
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8330912
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Add missing RGBToYMatrixRow_Any_{LSX,LASX} and kAddY offset fix so
RAW/RGB24 to YUV works on LoongArch.
Change-Id: I4c990a3705d7f521fa6d0c9e7e01f1e36f81c379
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8292090
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
1. source/convert_from_argb.cc should include libyuv/convert.h because
it calls ARGBToI420Matrix().
2. Include <string.h> for memset().
Change-Id: I3f4841fe604639c7357148ef4788fe0415f0b81b
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8255654
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reduces include/libyuv/basic_types.h to only define LIBYUV_API.
Directly includes <stdint.h> and <stddef.h> where needed in public
headers.
Replaces LIBYUV_BOOL, LIBYUV_FALSE, and LIBYUV_TRUE with C++ bool,
true, and false in C++ code (MJpegDecoder, unit tests) and int (0, 1)
for C API functions.
Updates documentation and unit tests.
Test: ninja -C out/Release libyuv_unittest && ./out/Release/libyuv_unittest --gtest_filter=*
Change-Id: I2d58e872f0f976263bcda546dcc3b1b5f7e53d58
TAG=agy
CONV=2730bc56-93ff-48ed-b615-dcde1b834394
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280662
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
CPU / Architecture Before (C) After (NEON) Improvement
----------------------------------------------------------------------
Cortex-A53 (Pixel 2 XL Little) 3248 ms 792 ms 4.10x
Cortex-A73 (Pixel 2 XL Big) 432 ms 392 ms 1.10x
Cortex-A55 (Pixel 7 Little) 780 ms 563 ms 1.39x
Cortex-A78 (Pixel 7 Mid) 254 ms 170 ms 1.49x
Cortex-X1 (Pixel 7 Big) 158 ms 129 ms 1.22x
Cortex-A510 (Galaxy S22 Little) 816 ms 441 ms 1.85x
Cortex-A710 (Galaxy S22 Mid) 270 ms 129 ms 2.09x
Cortex-X2 (Galaxy S22 Big) 116 ms 80 ms 1.45x
Cortex-A510 (Galaxy S23 Little) 723 ms 373 ms 1.94x
Cortex-A715 (Galaxy S23 Mid) 183 ms 119 ms 1.54x
Cortex-X3 (Galaxy S23 Big) 122 ms 72 ms 1.69x
Lumix C1 Pro (PD2509 Little) 246 ms 156 ms 1.58x
Lumix C1 Premium (PD2509 Mid) 211 ms 112 ms 1.88x
Lumix C1 Ultra (PD2509 Big) 82 ms 50 ms 1.64x
Oryon-v3M (OnePlus Mid) 233 ms 150 ms 1.55x
Oryon-v3L (OnePlus Big) 103 ms 85 ms 1.21x
On Icelake
Function (1280x720) C SSSE3 AVX2 AVX-512BW Gain
----------------------------------------------------------------------
BlendPlane 901 ms 165 ms 135 ms 115 ms 1.17x
I420Blend 1398 ms 287 ms 198 ms 187 ms 1.06x
Bug: 546077674
Change-Id: I189c44cc757c0e26838a7b6fdb2db1cbe5f96f20
TAG=agy
CONV=0f82c25c-616e-4bb3-beeb-981decb0f40c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8297264
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
CPU / Arch Before (Kernel) After (Kernel) Delta
----------------------------------------------------------------------
Haswell (x86_64) 263 ms (SSSE3) 263 ms (AVX2) 0.0%
Milan (Zen 3) 115 ms (SSSE3) 138 ms (AVX2) -20.0%
SapphRap (x86_64) 173 ms (SSSE3) 178 ms (AVX512VBMI) -2.9%
GNR (Granite R) 191 ms (SSSE3) 189 ms (AVX512VBMI) +1.0%
Turin (Zen 5) 69 ms (SSSE3) 55 ms (AVX512VBMI) +20.3%
Icelake (x86_64) 204 ms (SSSE3) 186 ms (AVX512VBMI) +8.8%
Cascade Lake 237 ms (SSSE3) 229 ms (AVX512BW) +3.4%
Neoverse (aarch64)* 103 ms (SVE2) 103 ms (SVE2) 0.0%
Bug: None
Change-Id: Ib218632e16b3308a214e9ffcbc745d6c41116c6e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280536
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
ScalePlane's same-width vertical fast path used the endpoint-oriented
FixedDiv1 step for all upsampling. That stepping is appropriate for
bilinear interpolation, but not for point sampling, where source rows
should cover equally sized destination cells.
Use the same centered FixedDiv stepping as the generic point sampler.
Allow the point path to select the final source row while passing a
zero interpolation stride so row kernels cannot form a pointer beyond
it. Add 8-bit and 16-bit point regression coverage and verify that
bilinear upsampling remains unchanged.
Bug: 537818862
Change-Id: I7bee21b85913b31c146ecf7fe9f5f08608a8a17d
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8264449
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
The READYUV444, READYUVA444, READYUVA422, and STOREARGB macros are
defined but not used.
Change-Id: Ic1174b5b9c7dc8554f362871de73080c32a58537
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8279223
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
- reduced shifts to isolate 10 bits
- permi2w instead of packlwd to combine words (ARG) (GB) into ARGB
- vpternlogd to combine A,R,G into 1 register
Saphhire Rapids 10000 iterations of 1280x720P:
Conversion Kernel │ Was │ Now │ Speedup
──────────────────────┼────────────────────────┼────────────────
I420ToAR30_Opt │ 1675 ms │ 1568 ms │ +7.1%
H420ToAR30_Opt │ 1632 ms │ 1555 ms │ +5.2%
I210ToAR30_Opt │ 2459 ms │ 2167 ms │ +13.5%
P010ToAR30_Opt │ 2054 ms │ 1926 ms │ +6.6%
P016ToAR30_Opt │ 2043 ms │ 1938 ms │ +5.4%
Total Test Suite Time │ 9,866 ms │ 9,155 ms │ +7.8% overall
Bug: None
Change-Id: I7972f98408a65a2bb51c506145e4c0c63824c424
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8259796
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
The INCP instructions are particularly slow on some CPUs and in this
case can be replaced by a simple ADD instruction, so use that instead.
Reduction in time taken relative to the existing SVE2 code for
{YUY2,UYVY}ToUVRow_SVE2:
Cortex-A510: −12.1%
Lumex C1-Nano: 0.0%
Cortex-A720: 0.0%
Lumex C1-Pro: 0.0%
Cortex-X4: −20.9%
Lumex C1-Ultra: −20.3%
Bug: 546962730
Change-Id: Iea3d75f7b70fe8a29f12afdd877ac02d43e49f04
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8252282
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
- improves I420ToRAW performance
I420ToRAW on Skylake
Was AVX2
I420ToRAW_Opt (262 ms)
Now AVX512
I420ToRAW_Opt (212 ms)
I420ToRAW on Icelake with avx512vbmi
Was I420ToRAW_Opt (200 ms)
Now I420ToRAW_Opt (187 ms)
Bug: None
Change-Id: If5b171801ba931557d379696fc128d9b155f7c44
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8267746
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
The tail LD2B predicate previously used the raw remaining width, however
for odd widths this is incorrect. We already calculate the correct
halved predicate for the UV store part so just compute the load
predicate from that directly.
Bug: 546962730
Change-Id: I59654208ea48eb5bcc6261a63cb90abae58c1668
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8252280
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
This reverts commit 87c797b3c29cc6fb233fc329a93aace2942ba2d9.
Reason for revert: Only commit 8aeb3a9 needs the workaround
Original change's description:
> Add the ARGBToRGB565Row_SSE2() function back
>
> Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
> back because they hide the unaligned access in ARGBToRGB565Row_C().
>
> To reproduce the unalgined access error, run the following command in
> Google's internal repository:
> blaze test --config=asan third_party/libyuv:libyuv_test \
> --test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
>
> Bug: b:547803993
> Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
> Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
> Commit-Queue: Wan-Teh Chang <wtc@google.com>
> Reviewed-by: Frank Barchard <fbarchard@google.com>
Bug: b:547803993
Change-Id: I904126f032f4ed37c470d8972152a137ca315168
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8258919
Reviewed-by: Frank Barchard <fbarchard@google.com>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
back because they hide the unaligned access in ARGBToRGB565Row_C().
To reproduce the unalgined access error, run the following command in
Google's internal repository:
blaze test --config=asan third_party/libyuv:libyuv_test \
--test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
Bug: b:547803993
Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
It is the temporary input buffer, not the temporary output buffer, that
needs to be zero-initialized for msan.
This mistake may have been introduced because dst_temp was declared
before src_temp. So swap their declarations.
Change-Id: I74691afe51a7d84187fc0decbc83f2c970cf52b7
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8261587
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Copy dither4x4 (16 bytes) into an aligned local uint32_t dither4[4] array
using memcpy and index using dither4[y & 3] to avoid unaligned 32-bit
pointer dereference traps in fastbuild / UB sanitizers.
Bug: None
Test: LibYUVConvertTest.TestNoDither, LibYUVConvertTest.TestDither
Change-Id: I4447cd2924b16739a9bce2792d57a550dedeb345
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8262354
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
- Add register z2 to the inline assembly clobber list in ARGBToRGB565DitherRow_SVE2.
- Fix operand constraints and register indexes for src_argb, dst_rgb, and width in ARGBToRGB565DitherRow_NEON to properly use '+r' for modified
operands.
Bug: None
Change-Id: I521c5b37dc37e2bcbe910f72c389144b1491e9dc
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8261579
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
In ScaleARGBBilinearUp, ScalePlaneBilinearUp, ScalePlaneBilinearUp_16,
and ScaleUVBilinearUp, dy is strictly < 65536 (dy < 1.0 in 16.16 fixed
point), so yi (y >> 16) advances strictly in unit increments (0 -> 1 ->
2 -> ... -> H - 1).
Because yi transitions across each integer row in sequence:
- src is primed at row 2 and incrementally advanced by 1 row on each
integer transition while (y + 65536) < max_y.
- When y >= max_y - 65536, src increments stop, leaving src pointing to
the final row (H - 1).
- Therefore, src is always pointing to row yi + 1 before each buffer
swap, making the inner 'src = src_ptr + yi * src_stride' recomputation
and nested 'if (yi != lasty)' checks redundant.
Bug: 540206290, 539110937
Test: LibYUVScaleTest.*
Change-Id: Ic5cc5c601d4771e9df8b9a882df2f7f3e6777b23
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8252878
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
In ScaleARGBBilinearDown and ScaleARGBBilinearUp, adding dy to y can
overflow signed 32-bit int when scaling down large height images (e.g.,
src_height = 32768 down to 2). Check if max_y - dy < y before adding dy
to y to prevent signed integer overflow.
R=wtc
Fixed: 540206290, 539110937
Test: LibYUVScaleTest.ARGBScale_LargeHeightOverflow
Change-Id: If353d002714481ce0018653798047066663f1fb9
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8175358
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
