Add AVX-512 VNNI (VPDPBUSD / _mm512_dpbusd_epi32) row kernels for 64-bit
x86 targets alongside existing AVX-512BW implementations in both
row_gcc.cc (inline assembly) and row_win.cc (C++ intrinsics):
- ARGBToYMatrixRow_AVX512VNNI and ARGBToYMatrixRow_Any_AVX512VNNI:
Pre-computes the 32-bit signed-pixel bias once per row and replaces
vpmaddubsw + vpmaddwd + vpaddw with a single vpdpbusd per 16 pixels.
- ARGBToUV444MatrixRow_AVX512VNNI and
ARGBToUV444MatrixRow_Any_AVX512VNNI: Loads 64 ARGB pixels once instead
of twice and computes 32-bit U and V dot products directly via
vpdpbusd with signed UV matrix coefficients.
- ARGBToUVMatrixRow_AVX512VNNI: Computes 2x2 box-filtered ARGB to U and
V conversion using vpdpbusd with k-masked tail handling for arbitrary
widths.
- ARGBGrayRow_AVX512VNNI: Computes BT.601 grayscale luminance using a
single vpdpbusd per 16 pixels, broadcasts Y to RGB via vpshufb, and
preserves alpha via vpternlogd with k-masked tail handling.
- Wire runtime dispatch via TestCpuFlag(kCpuHasAVX512VNNI) across
convert.cc, convert_from_argb.cc, and planar_functions.cc.
Add AVX512VNNIVBMI variants of the Y, UV444 and UV kernels. These use
both AVX-512 VNNI (vpdpbusd) and AVX-512 VBMI (vpermt2b) and are only
selected when the CPU has both features; there are no VBMI versions
without VNNI. The AVX512VNNI kernels remain for CPUs with VNNI but no
VBMI (Cascade Lake, Cooper Lake).
- The 32-bit sums are always in [0, 65535], so Y, U and V are byte 1 of
each int32. Two vpermt2b gather 64 results from 4 accumulators,
replacing vpackusdw/vpackssdw + vpsrlw + vpackuswb + vpermd.
- U and V accumulators start at ~kAddUV so they hold ~(kAddUV - dot).
The inverse is folded into the vpternlogd that merges the two
256-bit halves. kAddUV is loaded from ArgbConstants rather than
hard coded as 0x8000.
- Y folds the source loads into vpxorq.
- row_win.cc hides the vpermt2b indices from clang, which otherwise
constant folds them into a longer vpshufb + vpor + vpermd sequence.
Performance (1280x720, 5000 iterations, AVX512BW -> AVX512VNNI):
uArch / Benchmark AVX512BW AVX512VNNI Speedup
--------------------------- -------- ---------- -------
Intel Ice Lake ARGBToI420 941 ms 862 ms +9.2%
Intel Ice Lake ARGBToI422 1059 ms 947 ms +11.8%
Intel Ice Lake ARGBToI444 1140 ms 1038 ms +9.8%
Intel Ice Lake ARGBGray 357 ms 273 ms +30.8%
Intel Sapphire Rapids GrayTo 317 ms 218 ms +45.4%
Intel Emerald Rapids GrayTo 310 ms 213 ms +45.5%
Intel Granite Rapids GrayTo 349 ms 241 ms +44.8%
AMD Zen 4 ARGBToI420 648 ms 517 ms +25.3%
AMD Zen 4 ARGBToI444 695 ms 526 ms +32.1%
AMD Zen 4 ARGBGray 403 ms 227 ms +77.5%
AMD Zen 5 ARGBToI422 460 ms 403 ms +14.1%
AMD Zen 5 ARGBToI444 382 ms 321 ms +19.0%
Performance (row_gcc, AVX512VNNI -> AVX512VNNIVBMI):
uArch / Benchmark AVX512VNNI VNNIVBMI Speedup
---------------------------- ---------- -------- -------
AMD Zen 6 ARGBToI420 1280x720 327 ms 284 ms +15.1%
AMD Zen 6 ARGBToI422 1280x720 372 ms 330 ms +12.7%
AMD Zen 6 ARGBToI444 1280x720 357 ms 334 ms +6.9%
AMD Zen 6 ARGBToI400 256x24 94 ms 76 ms +23.7%
AMD Zen 5 ARGBToI422 1280x720 406 ms 350 ms +16.0%
AMD Zen 5 ARGBToI444 1280x720 330 ms 301 ms +9.6%
AMD Zen 5 ARGBToI400 256x24 100 ms 90 ms +11.1%
AMD Zen 4 ARGBToI420 1280x720 631 ms 540 ms +16.9%
AMD Zen 4 ARGBToI422 1280x720 993 ms 825 ms +20.4%
AMD Zen 4 ARGBToI400 256x24 245 ms 218 ms +12.4%
Intel GNR ARGBToI422 256x24 654 ms 616 ms +6.2%
Intel SPR ARGBToI422 256x24 653 ms 613 ms +6.5%
Intel GNR, EMR, SPR and Ice Lake are within +/-3% at 1280x720.
R=rrwinterton@gmail.com
Bug: none
Test: out/Release/libyuv_unittest --gtest_filter=*ARGBToI4*:*ARGBGray*
Test: LIBYUV_DISABLE_AVX512VBMI=1 out/Release/libyuv_unittest
Test: sde64 -gnr / -clx with is_msan=true (row_win.cc)
Change-Id: Ia7a37b39dd2948fe4055a8a7694dd1ff8ac3e55c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8647811
Reviewed-by: richard winterton <rrwinterton@gmail.com>
1. Fix multi-bit TestCpuFlag(A | B) checks in AYUVToNV12, AYUVToNV21,
RAWToRGBA, NV12ToRGB24Matrix, NV21ToRGB24Matrix, MergeRGBPlane, and
MergeXR30Plane to compare == (A | B) so AVX-512 VBMI/BW kernels are
not dispatched when AVX512VBMI or AVX512VL is absent.
2. Fix 2-byte SVE2 heap buffer overflow on odd widths in
I422ToARGB1555Row_SVE_SC and I422ToARGB4444Row_SVE_SC using
trn1/trn2 + st2b under .b predicates.
3. Fix ARGBPolynomialRow_SSE2 to use packssdw before packuswb, and add
missing vzeroupper to SplitRGBRow_AVX2.
4. Fix CalcFrameSsim to return 0.0 instead of 0.0 / 0 (NaN) when
width <= 8 || height <= 8 so I420Ssim stays finite on small frames.
5. Fix negative-height vertical inversion in ConvertFromI420,
SplitARGBPlane (when dst_a is NULL), I422Rotate, I010Rotate, and
I210Rotate.
6. Fix horizontal flip (negative x/dx) in ScaleFilterCols_NEON (signed
sbfx #16, #16 offset extraction) and ScaleAddCols1_16_C (shift
x >>= 16 before indexing src_ptr).
7. Fix ConvertToARGB odd crop_x chroma alignment in NV12/NV21, src_u
offset in I420/YV12/J420/H420/U420, and FOURCC_U422 calling
U422ToARGB instead of H422ToARGB.
8. Remove dead 14-instruction block in ScaleRowUp2_Bilinear_16_SSE2.
9. Fix UVScale_16 for src_width == 1 (P010ToP410/P210ToP410), enable
and fix column order in TransposeWx4_32_SVE2, fix ByteToFloatRow
NEON Any BPP (3 -> 4), fix AB64ToARGBRow_Any_*, ScaleColsUp2_SSE2,
ScaleUVRowDown2Box_SME, and UYVYToARGBRow_SME macro guards, fix SVE2
loop tail branches (b.lt/b.ge), and remove duplicate HAS_* macros.
10. Fix ConvertFP16ToFP32Column_NEON loop condition and unsigned b.lo
underflow on widths < 8 or not multiples of 8.
11. Fix UYVYToNV12 IS_ALIGNED(awidth, ...) checks and WASM SIMD
IS_ALIGNED(width, 16) checks in I420ToRGB24Matrix /
I422ToRGB24Matrix.
12. Wire SplitUVRow_SVE2, SplitUVRow_LSX, and SplitUVRow_LASX in
NV24ToARGB.
13. Fix MJpegDecoder heap buffer overflows in LoadFrame (reallocate
databuf_[i] when scanlines_size changes), DecodeToBuffers, and
DecodeToCallback (clamp scanlines_to_copy to lines_left and align
skip to max_v_samp_factor).
14. Replace width & (vl - 1) tail calculations with post-loop remainder
across 18 SVE2 functions in row_sve.h for non-power-of-2 SVE VL,
and remove duplicate label 2: in UYVYToARGBRow_SVE_SC.
15. Handle width & 15 tail pixels in DetileToYUY2_C and validate
IS_POWEROFTWO(tile_height) in DetileSplitUVPlane and DetileToYUY2.
16. Avoid signed 32-bit overflow in DivideRow_16_C and C16TO8, fix
ClampFloatToZero_SSE2 pointer increment (lea 0x4) and loop counter
(sub $0x1), and initialize xmm3 in ARGBCopyYToAlphaRow_SSE2.
17. Check max leaf 7 subleaf and max extended CPUID leaf in GetCpuFlags
before querying subleaf 7:1 and extended leaf 0x80000021.
18. Free plane_u on error in I422ToNV21, and fix 4x/2x scratch row
over-allocation in ScaleARGBBilinearDown and ScaleUVBilinearDown.
19. Add ARGBSetRow_RVV and ARGBShuffleRow_RVV in row_rvv.cc.
20. Fix NEON mask (15 -> 7) in ScaleUVRowUp2_Linear_Any_NEON /
_16_Any_NEON, use SIMD InterpolateRow + SplitUVRow in NV16ToI420,
wire SSSE3/SSE2 in ARGBToI400Matrix / ARGBToYUY2Matrix /
ARGBToUYVYMatrix and RGB24Mirror, and split HalfFloatRow multiply
into (src * scale) * 2^-112 to avoid 50x Intel denormal stalls.
Bug: b/574714081
Test: libyuv_unittest
Change-Id: Iec3ae5c58d52bee3ef13cf9b765ac84b733aa576
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8601931
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Add matrix and BT.601 wrappers that convert ARGB to 16-bit biplanar
YUV in P010 (4:2:0), P210 (4:2:2), and P410 (4:4:4) layouts.
Reuse optimized ARGB matrix row functions for 8-bit Y and chroma,
merge UV with MergeUVRow, then expand with Convert8To16Row (bits=10)
and MultiplyRow_16 (scale=64) so 10-bit samples are stored in the
high bits of each uint16.
Reject width > INT_MAX / 2 so UV width (halfwidth * 2 or width * 2)
does not overflow Convert8To16Row.
Bug: 537818862
Change-Id: I7a776fb3d6f52ce46a0f6825903bfdb9c6dcfbe2
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8314189
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
- Also remove redundant neon check for I8MM kernels
======================================================================
CPU Before After Speedup
----------------------------------------------------------------------
Intel Granite Rapids (AVX10.1) 251 ms 141 ms 1.78x (+43.8%)
Intel Emerald Rapids (AVX512) 199 ms 108 ms 1.84x (+45.7%)
Intel Sapphire Rapids (AVX512) 230 ms 112 ms 2.05x (+51.3%)
AMD Turin Zen 5 (AVX512) 156 ms 99 ms 1.58x (+36.5%)
AMD Venice Zen 6 (AVX512) 204 ms 106 ms 1.92x (+48.0%)
Intel Skylake-X (AVX512) 320 ms 156 ms 2.05x (+51.2%)
AMD Genoa Zen 4 (AVX512) 236 ms 346 ms 0.68x (-46.6%)
Intel Ice Lake (AVX512) 242 ms 316 ms 0.77x (-30.6%)
AMD Milan Zen 3 (AVX2) 252 ms 230 ms 1.10x (+8.7%)
Intel Haswell (AVX2) 369 ms 456 ms 0.81x (-23.6%)
NVIDIA Grace (SVE2) 336 ms 337 ms 1.00x (-0.3%)
ARM Neoverse-N1 (NEON) 1081 ms 1105 ms 0.98x (-2.2%)
======================================================================
Bug: 537818862
Change-Id: I3ddeddb040fc52a30dabe7ae2c71d16dda021568
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8343467
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Add matrix and BT.601 wrapper APIs for converting ARGB to NV16 and
NV24. Reuse the optimized ARGB Y and UV row functions, then merge
the planar chroma rows into the interleaved destination plane.
NV16 performs horizontal 2:1 chroma subsampling on every source row.
NV24 preserves full-resolution 4:4:4 chroma. Both paths support odd
widths, negative heights, and the existing matrix constants.
Add tests that compare scalar and optimized paths at an odd width and
height through the matrix API and its BT.601 wrapper.
Bug: 537818862
Change-Id: Iddefc714a800cdec4dd1394ac7a52bf7b51fd350
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8314188
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Add missing RGBToYMatrixRow_Any_{LSX,LASX} and kAddY offset fix so
RAW/RGB24 to YUV works on LoongArch.
Change-Id: I4c990a3705d7f521fa6d0c9e7e01f1e36f81c379
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8292090
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
1. source/convert_from_argb.cc should include libyuv/convert.h because
it calls ARGBToI420Matrix().
2. Include <string.h> for memset().
Change-Id: I3f4841fe604639c7357148ef4788fe0415f0b81b
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8255654
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reduces include/libyuv/basic_types.h to only define LIBYUV_API.
Directly includes <stdint.h> and <stddef.h> where needed in public
headers.
Replaces LIBYUV_BOOL, LIBYUV_FALSE, and LIBYUV_TRUE with C++ bool,
true, and false in C++ code (MJpegDecoder, unit tests) and int (0, 1)
for C API functions.
Updates documentation and unit tests.
Test: ninja -C out/Release libyuv_unittest && ./out/Release/libyuv_unittest --gtest_filter=*
Change-Id: I2d58e872f0f976263bcda546dcc3b1b5f7e53d58
TAG=agy
CONV=2730bc56-93ff-48ed-b615-dcde1b834394
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280662
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
This reverts commit 87c797b3c29cc6fb233fc329a93aace2942ba2d9.
Reason for revert: Only commit 8aeb3a9 needs the workaround
Original change's description:
> Add the ARGBToRGB565Row_SSE2() function back
>
> Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
> back because they hide the unaligned access in ARGBToRGB565Row_C().
>
> To reproduce the unalgined access error, run the following command in
> Google's internal repository:
> blaze test --config=asan third_party/libyuv:libyuv_test \
> --test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
>
> Bug: b:547803993
> Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
> Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
> Commit-Queue: Wan-Teh Chang <wtc@google.com>
> Reviewed-by: Frank Barchard <fbarchard@google.com>
Bug: b:547803993
Change-Id: I904126f032f4ed37c470d8972152a137ca315168
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8258919
Reviewed-by: Frank Barchard <fbarchard@google.com>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions
back because they hide the unaligned access in ARGBToRGB565Row_C().
To reproduce the unalgined access error, run the following command in
Google's internal repository:
blaze test --config=asan third_party/libyuv:libyuv_test \
--test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned
Bug: b:547803993
Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Copy dither4x4 (16 bytes) into an aligned local uint32_t dither4[4] array
using memcpy and index using dither4[y & 3] to avoid unaligned 32-bit
pointer dereference traps in fastbuild / UB sanitizers.
Bug: None
Test: LibYUVConvertTest.TestNoDither, LibYUVConvertTest.TestDither
Change-Id: I4447cd2924b16739a9bce2792d57a550dedeb345
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8262354
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
- MergeUVRow_SVE2: Interleaves U and V
- SplitUVRow_SVE2: De-interleaves UV
- InterpolateRow_SVE2: Bilinear row interpolation
- YUY2ToYRow_SVE2, YUY2ToUVRow_SVE2, UYVYToYRow_SVE2, UYVYToUVRow_SVE2
- HalfMergeUVRow_SVE2: 2:1 vertical averaging of U and V
OnePlus 15 with Qualcomm Snapdragon 8 Elite Gen 5 Oryon V3
Benchmark (1280x720 1000x) | SVE2 | NEON | Result
-------------------------------+--------+--------+------------------
TestInterpolatePlane | 119 ms | 164 ms | SVE2 is 1.38x win
TestInterpolatePlane_16 | 138 ms | 172 ms | SVE2 is 1.25x win
SplitUVPlane_Opt | 68 ms | 69 ms | SVE2 is 1.01x win
MergeUVPlane_Opt | 40 ms | 40 ms | Tied
SwapUVPlane_Opt | 72 ms | 72 ms | Tied
HalfMergeUVPlane_Opt | 50 ms | 50 ms | Tied
I420ToNV12_Opt | 25 ms | 26 ms | SVE2 is faster
I420ToNV21_Opt | 26 ms | 26 ms | Tied
YUY2ToNV12_Opt | 79 ms | 81 ms | SVE2 is faster
NV21ToNV12_Opt | 36 ms | 35 ms | Tied
Android420ToI420ToNV12_Opt | 53 ms | 53 ms | Tied
Android420ToI420ToNV21_Opt | 35 ms | 33 ms | NEON is faster
YUY2ToI420_Opt | 87 ms | 84 ms | NEON is faster
UYVYToI420_Opt | 87 ms | 83 ms | NEON is faster
YUY2ToI422_Opt | 101 ms | 97 ms | NEON is faster
UYVYToI422_Opt | 101 ms | 97 ms | NEON is faster
UYVYToNV12_Opt | 111 ms | 108 ms | NEON is faster
ARGBToNV21_Opt | 279 ms | 279 ms | Tied
ABGRToNV21_Opt | 279 ms | 280 ms | SVE2 is faster
ARGBToNV12_Opt | 285 ms | 280 ms | NEON is faster
ABGRToNV12_Opt | 284 ms | 281 ms | NEON is faster
Bug: 42280902
Change-Id: I53f633fe453f2853f7bc9fe17280e938912c4e8f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8196068
Reviewed-by: Justin Green <greenjustin@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
It should be named dst_stride_vu to match the dst_vu parameter.
Change-Id: I3a4835eafa358787a858b896060e6c715813bbec
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8135083
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Fix the chroma plane order passed to MergeUVRow in ARGBToNV21Matrix.
MergeUVRow(a, b, dst, ...) writes a into byte 0 and b into byte 1.
Passing (row_u, row_v) produced NV12 format (UV interleaved) instead
of NV21 format (VU interleaved). Swapping to (row_v, row_u) ensures
the chroma channels are correctly ordered for NV21 output.
Test: libyuv_unittest --gtest_filter=*NV21*
Bug: None
Change-Id: I3b78246c635ce515923739298013a61203cabb72
TAG=agy
CONV=5daf08a8-11bd-4d47-8c71-c5b7a882a788
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8133203
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Treat height == INT_MIN as invalid. Omit explicit height == INT_MIN
check if we disallow height < 32768.
Perform multiplications of stride in the ptrdiff_t type.
Add checks for invalid width and height to some functions.
Bug: 518806561
Change-Id: I5e39fffed7f806852a8758d4b59df919839c0a3b
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7891415
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
- Standardize libyuv ARGB-family (ARGB, ABGR, RGBA, BGRA) to YUV conversion by utilizing the generic MatrixRow architecture and explicit ArgbConstants.
- Consolidated ARGBToI420, ABGRToI420, BGRAToI420, and RGBAToI420 as wrappers for ARGBToI420Matrix.
- Refactored ABGRToJ420, ABGRToJ422, and ABGRToI422 to use generic matrix functions.
- Added matrix-based versions for NV21, I400, YUY2, and UYVY.
- Updated RAW and RGB24 to I420/I422/I444 dispatchers to use MatrixRow logic and explicit constants.
- Fixed parameter swap bugs in ARGBToI422, ARGBToJ422, and ABGRToJ422.
- Fixed a bug in the generic C implementation of matrix row functions ensuring all 4 channels are processed correctly for all ARGB-family formats.
- Moved kShuffleAARRGGBB in row_gcc.cc to the top of the libyuv namespace for visibility.
- Cleaned up redundant format-specific row implementations.
Bug: libyuv:42280902
Change-Id: I67ffa4c476abc0d2dcc4650510d7bda91b65988e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7830291
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
This consolidation standardizes conversion logic, improves code
maintainability, and provides flexible support for various color spaces
(e.g., BT.601, JPEG full
range).
Key Modifications:
- Function Consolidation: Refactored several high-level conversion functions into lightweight wrappers around generic Matrix variants:
- ARGBToI420 → ARGBToI420Matrix
- ARGBToI444 → ARGBToI444Matrix
- ARGBToI422 → ARGBToI422Matrix
- ARGBToNV12 → ARGBToNV12Matrix
- RAWToJ400, RGB24ToJ400 → RGBToI400Matrix
- RAWToI444, RAWToJ444 → RGBToI444Matrix
- 2-Pass Conversions: Updated RGB565ToI420, ARGB1555ToI420, and ARGB4444ToI420 to utilize 2-pass conversions via RGBToI420Matrix.
- Standardization: Refactored ARGBToNV21, ARGBToYUY2, and ARGBToUYVY to use parameterized matrix row functions (ARGBToYMatrixRow,
ARGBToUVMatrixRow).
- Legacy Cleanup: Replaced legacy calls to ARGBToYJRow with the parameterized ARGBToYMatrixRow in the ARGBSobelize helper.
- Internal Integration: Included libyuv/convert_from_argb.h in planar_functions.cc and ensured all new matrix symbols are properly
declared/exported (LIBYUV_API).
Bug: libyuv:42280902
Change-Id: Ied5fd9899767427e3a03cdcfbeaff3e9d502374a
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7822033
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- implement wrappers with RAW, RGB24, NV21 and JNV21 to call it.
Zen5
Was [ OK ] LibYUVConvertTest.RAWToJNV21_Opt (1146 ms)
Now [ OK ] LibYUVConvertTest.RAWToJNV21_Opt (1446 ms)
reason - the new code uses 1 pass for RAWToY but 2 pass for RAWToARGB,ARGBToUV. needs 1 RGBToUV
Bug: libyuv:42280902
Change-Id: Ife6fbed0829484045409e6d42b85cec1d1fd6052
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7780026
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Adds RGBToYMatrixRow_AVX2 which reads 24 bit RGB values by reading 3 vectors instead of 4 and permutes them into 4 ARGB vectors before conversion.
Also adds RGBToYMatrixRow_Opt and RGBToYMatrixRow_2Step_Opt to convert_argb_test.cc to benchmark and compare the direct AVX2 conversion vs a 2-step approach.
./libyuv_test '--gunit_filter=*RAWToJ400_Opt' --libyuv_width=1280 --libyuv_height=720 --libyuv_repeat=10000 --libyuv_flags=-1 --libyuv_cpu_info=-1
AMD Zen 5
Was LibYUVConvertTest.RAWToJ400_Opt (757 ms)
Now LibYUVConvertTest.RAWToJ400_Opt (699 ms)
Intel Skylake
Was LibYUVConvertTest.RAWToJ400_Opt (1705 ms)
Now LibYUVConvertTest.RAWToJ400_Opt (1426 ms)
Bug: 477295731
Change-Id: I29866baf4ad5fe7a3725e4a01f2fe24649510a7d
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7777325
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Justin Green <greenjustin@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
This change implements ARGBToUV444MatrixRow_RVV, ARGBToUVMatrixRow_RVV,
and their wrappers (ARGBToUVRow_RVV, ARGBToUVJRow_RVV, etc.) using RVV
intrinsics, mirroring the NEON/AVX2 designs. It wires them into the
build and dispatch systems.
LIBYUV_RVV_HAS_TUPLE_TYPE is always true on new compilers. This macro
has been removed, assuming it is true everywhere, reducing the amount of
code in row_rvv.cc, scale_rvv.cc, and row.h.
Tested via: ~/bin/doyuv3v && ~/bin/runyuv3v TestARGBToI444Matrix
~/bin/doyuv3av
Bug: libyuv:42280902
Change-Id: I36d305386b297d69023c068aa9c62ab6b2ad039c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7769956
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Benchmark on Icelake Xeon
Now AVX512BW:
[ OK ] LibYUVConvertTest.ARGBToNV12_Opt (1723 ms)
Was AVX2:
[ OK ] LibYUVConvertTest.ARGBToNV12_Opt (2144 ms)
- Added `ARGBToUVMatrixRow_AVX512BW` implementation in `source/row_gcc.cc`.
- Added corresponding `ARGBToUVRow_AVX512BW` and `ABGRToUVRow_AVX512BW` functions.
- Added unaligned wrappers `ARGBToUVRow_Any_AVX512BW` and `ABGRToUVRow_Any_AVX512BW` in `source/row_any.cc`.
- Updated `source/row_any.cc` to correctly size `vin` and `vout` buffers for AVX512BW width and adjusted the `ANY12MS` and `ANY12S` macros to handle `MASK=63`.
- Updated `include/libyuv/row.h` with the required AVX512BW headers and definitions, scoped appropriately.
- Wired all callers of `ARGBToUVRow_AVX2` and related functions in `source/convert.cc` and `source/convert_from_argb.cc` to dynamically use the `AVX512BW` implementations if the CPU flag indicates AVX-512BW support.
- Optimized AVX-512 code to generate the `-1` multiplier in a single instruction (`vpternlogd`) and reused it across word (`vpmaddwd`) dot products. Handled the resulting negation by replacing a subtraction with `vpaddw` offset adjustment.
Bug: 477295731
R=dalecurtis@chromium.org, rrwinterton@gmail.com
Change-Id: Ida5fb27e59ae4c1c3824737f009b80549cd20a06
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7763257
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
I have successfully ported the usage of ARGBToYRow_AVX2 to dynamically detect and utilize ARGBToYRow_AVX512BW when available.
Here's a summary of the changes:
1. Source Modifications: In both source/convert.cc and source/convert_from_argb.cc, I searched for all references where ARGBToYRow_AVX2 was
being conditionally used (which operates on 32 pixels).
2. AVX512BW Detection: Immediately following those blocks, I injected a new check for kCpuHasAVX512BW. If the CPU flag is present, the logic
now utilizes ARGBToYRow_Any_AVX512BW by default, falling back to the fully aligned ARGBToYRow_AVX512BW when the width is aligned to 64
bytes.
3. Profiling: After building and compiling the tests (doyuv3x), I validated the change using perfyuv3 ARGBToNV12_Opt | cat. The test
successfully executed and the performance profile indicated that ARGBToYRow_AVX512BW successfully executed (taking up ~18% of CPU cycles,
replacing the previous AVX2 specific instruction overhead for the Y row extraction).
The HAS_ARGBTOYROW_AVX512BW macro implementation now fully supports all AVX2 conversion paths to utilize AVX512BW when the system processor
flags allow it!
R=richard, rrwinterton@gmail.com
Change-Id: Iad811e12d301f5621e6f6d039105420861ade43e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7760779
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- add ARGBToYMatrixRow_AVX512BW
- refactor SSE and AVX to use Matrix functions, making old functions
call the new ones.
Zen5 1280x720
Was AVX2 LibYUVConvertTest.ARGBToI444_Opt (1125 ms)
Now AVX512 LibYUVConvertTest.ARGBToI444_Opt (641 ms)
Details by Gemini:
1. Created 3 new Matrix functions:
Added ARGBToYMatrixRow_SSSE3, ARGBToYMatrixRow_AVX2, and
ARGBToYMatrixRow_AVX512BW to source/row_gcc.cc. These take the
const struct ArgbConstants* c parameter similarly to
ARGBToUV444MatrixRow_*. The x86 vector instructions dynamically
calculate the needed values using the properties of the constants
struct, including using vpmaddwd inside the AVX512 code to offset
the lack of a native vphaddw.
2. Replaced Old Functions with Wrappers:
Modified the existing implementations of ARGBToYRow_SSSE3,
ARGBToYJRow_SSSE3, ABGRToYRow_SSSE3, ABGRToYJRow_SSSE3,
RGBAToYRow_SSSE3, RGBAToYJRow_SSSE3, BGRAToYRow_SSSE3 (and their
_AVX2 equivalents) in source/row_gcc.cc to act as inline wrappers
calling the new ARGBToYMatrixRow_* functions, passing the right
matrix parameters (e.g. &kArgbI601Constants, &kArgbJPEGConstants,
&kAbgrI601Constants).
3. Added row_any.cc Handlers:
Added ANY11MC definitions to source/row_any.cc to autogenerate
ARGBToYMatrixRow_Any_SSSE3, ARGBToYMatrixRow_Any_AVX2, and
ARGBToYMatrixRow_Any_AVX512BW which safely handles non-aligned
tails.
4. Updated include/libyuv/row.h:
Updated the headers with the proper void declarations for all newly
generated Matrix and Any_ variants. Also defined
HAS_ARGBTOYROW_AVX512BW in the CPU macros.
5. Tested the Implementations:
Compiled and tested on Linux x86, which resulted in all tests passing
cleanly. Also successfully completed all Windows 32-bit build checks
ensuring 32-bit regression prevention without issues.
Bug: 477295731
Change-Id: I4f5eec9a961e24a9d760d0a1c0810fb5e29a0bd1
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7759494
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
These are about 25% faster than the C versions.
Bug: libyuv:42280902
Change-Id: I8b298670ee5f3ed5db35527fc41d6d9a51b020a1
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7573682
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
This one reuses the SIMD implementations for MergeUVRow_ from the
existing ARGBToNV12 functions.
Bug: libyuv:42280902
Change-Id: If0a4be133d657ed0262f29fdd568dac90b49636c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7564317
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
This allows for ABGR conversion using the same methods
Bug: libyuv:42280902
Change-Id: I5566e3150b30573a2326a900ce31ab095f8935f9
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7564316
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
