Add AVX-512 VNNI (VPDPBUSD / _mm512_dpbusd_epi32) row kernels for 64-bit
x86 targets alongside existing AVX-512BW implementations in both
row_gcc.cc (inline assembly) and row_win.cc (C++ intrinsics):
- ARGBToYMatrixRow_AVX512VNNI and ARGBToYMatrixRow_Any_AVX512VNNI:
Pre-computes the 32-bit signed-pixel bias once per row and replaces
vpmaddubsw + vpmaddwd + vpaddw with a single vpdpbusd per 16 pixels.
- ARGBToUV444MatrixRow_AVX512VNNI and
ARGBToUV444MatrixRow_Any_AVX512VNNI: Loads 64 ARGB pixels once instead
of twice and computes 32-bit U and V dot products directly via
vpdpbusd with signed UV matrix coefficients.
- ARGBToUVMatrixRow_AVX512VNNI: Computes 2x2 box-filtered ARGB to U and
V conversion using vpdpbusd with k-masked tail handling for arbitrary
widths.
- ARGBGrayRow_AVX512VNNI: Computes BT.601 grayscale luminance using a
single vpdpbusd per 16 pixels, broadcasts Y to RGB via vpshufb, and
preserves alpha via vpternlogd with k-masked tail handling.
- Wire runtime dispatch via TestCpuFlag(kCpuHasAVX512VNNI) across
convert.cc, convert_from_argb.cc, and planar_functions.cc.
Add AVX512VNNIVBMI variants of the Y, UV444 and UV kernels. These use
both AVX-512 VNNI (vpdpbusd) and AVX-512 VBMI (vpermt2b) and are only
selected when the CPU has both features; there are no VBMI versions
without VNNI. The AVX512VNNI kernels remain for CPUs with VNNI but no
VBMI (Cascade Lake, Cooper Lake).
- The 32-bit sums are always in [0, 65535], so Y, U and V are byte 1 of
each int32. Two vpermt2b gather 64 results from 4 accumulators,
replacing vpackusdw/vpackssdw + vpsrlw + vpackuswb + vpermd.
- U and V accumulators start at ~kAddUV so they hold ~(kAddUV - dot).
The inverse is folded into the vpternlogd that merges the two
256-bit halves. kAddUV is loaded from ArgbConstants rather than
hard coded as 0x8000.
- Y folds the source loads into vpxorq.
- row_win.cc hides the vpermt2b indices from clang, which otherwise
constant folds them into a longer vpshufb + vpor + vpermd sequence.
Performance (1280x720, 5000 iterations, AVX512BW -> AVX512VNNI):
uArch / Benchmark AVX512BW AVX512VNNI Speedup
--------------------------- -------- ---------- -------
Intel Ice Lake ARGBToI420 941 ms 862 ms +9.2%
Intel Ice Lake ARGBToI422 1059 ms 947 ms +11.8%
Intel Ice Lake ARGBToI444 1140 ms 1038 ms +9.8%
Intel Ice Lake ARGBGray 357 ms 273 ms +30.8%
Intel Sapphire Rapids GrayTo 317 ms 218 ms +45.4%
Intel Emerald Rapids GrayTo 310 ms 213 ms +45.5%
Intel Granite Rapids GrayTo 349 ms 241 ms +44.8%
AMD Zen 4 ARGBToI420 648 ms 517 ms +25.3%
AMD Zen 4 ARGBToI444 695 ms 526 ms +32.1%
AMD Zen 4 ARGBGray 403 ms 227 ms +77.5%
AMD Zen 5 ARGBToI422 460 ms 403 ms +14.1%
AMD Zen 5 ARGBToI444 382 ms 321 ms +19.0%
Performance (row_gcc, AVX512VNNI -> AVX512VNNIVBMI):
uArch / Benchmark AVX512VNNI VNNIVBMI Speedup
---------------------------- ---------- -------- -------
AMD Zen 6 ARGBToI420 1280x720 327 ms 284 ms +15.1%
AMD Zen 6 ARGBToI422 1280x720 372 ms 330 ms +12.7%
AMD Zen 6 ARGBToI444 1280x720 357 ms 334 ms +6.9%
AMD Zen 6 ARGBToI400 256x24 94 ms 76 ms +23.7%
AMD Zen 5 ARGBToI422 1280x720 406 ms 350 ms +16.0%
AMD Zen 5 ARGBToI444 1280x720 330 ms 301 ms +9.6%
AMD Zen 5 ARGBToI400 256x24 100 ms 90 ms +11.1%
AMD Zen 4 ARGBToI420 1280x720 631 ms 540 ms +16.9%
AMD Zen 4 ARGBToI422 1280x720 993 ms 825 ms +20.4%
AMD Zen 4 ARGBToI400 256x24 245 ms 218 ms +12.4%
Intel GNR ARGBToI422 256x24 654 ms 616 ms +6.2%
Intel SPR ARGBToI422 256x24 653 ms 613 ms +6.5%
Intel GNR, EMR, SPR and Ice Lake are within +/-3% at 1280x720.
R=rrwinterton@gmail.com
Bug: none
Test: out/Release/libyuv_unittest --gtest_filter=*ARGBToI4*:*ARGBGray*
Test: LIBYUV_DISABLE_AVX512VBMI=1 out/Release/libyuv_unittest
Test: sde64 -gnr / -clx with is_msan=true (row_win.cc)
Change-Id: Ia7a37b39dd2948fe4055a8a7694dd1ff8ac3e55c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8647811
Reviewed-by: richard winterton <rrwinterton@gmail.com>
1. Fix multi-bit TestCpuFlag(A | B) checks in AYUVToNV12, AYUVToNV21,
RAWToRGBA, NV12ToRGB24Matrix, NV21ToRGB24Matrix, MergeRGBPlane, and
MergeXR30Plane to compare == (A | B) so AVX-512 VBMI/BW kernels are
not dispatched when AVX512VBMI or AVX512VL is absent.
2. Fix 2-byte SVE2 heap buffer overflow on odd widths in
I422ToARGB1555Row_SVE_SC and I422ToARGB4444Row_SVE_SC using
trn1/trn2 + st2b under .b predicates.
3. Fix ARGBPolynomialRow_SSE2 to use packssdw before packuswb, and add
missing vzeroupper to SplitRGBRow_AVX2.
4. Fix CalcFrameSsim to return 0.0 instead of 0.0 / 0 (NaN) when
width <= 8 || height <= 8 so I420Ssim stays finite on small frames.
5. Fix negative-height vertical inversion in ConvertFromI420,
SplitARGBPlane (when dst_a is NULL), I422Rotate, I010Rotate, and
I210Rotate.
6. Fix horizontal flip (negative x/dx) in ScaleFilterCols_NEON (signed
sbfx #16, #16 offset extraction) and ScaleAddCols1_16_C (shift
x >>= 16 before indexing src_ptr).
7. Fix ConvertToARGB odd crop_x chroma alignment in NV12/NV21, src_u
offset in I420/YV12/J420/H420/U420, and FOURCC_U422 calling
U422ToARGB instead of H422ToARGB.
8. Remove dead 14-instruction block in ScaleRowUp2_Bilinear_16_SSE2.
9. Fix UVScale_16 for src_width == 1 (P010ToP410/P210ToP410), enable
and fix column order in TransposeWx4_32_SVE2, fix ByteToFloatRow
NEON Any BPP (3 -> 4), fix AB64ToARGBRow_Any_*, ScaleColsUp2_SSE2,
ScaleUVRowDown2Box_SME, and UYVYToARGBRow_SME macro guards, fix SVE2
loop tail branches (b.lt/b.ge), and remove duplicate HAS_* macros.
10. Fix ConvertFP16ToFP32Column_NEON loop condition and unsigned b.lo
underflow on widths < 8 or not multiples of 8.
11. Fix UYVYToNV12 IS_ALIGNED(awidth, ...) checks and WASM SIMD
IS_ALIGNED(width, 16) checks in I420ToRGB24Matrix /
I422ToRGB24Matrix.
12. Wire SplitUVRow_SVE2, SplitUVRow_LSX, and SplitUVRow_LASX in
NV24ToARGB.
13. Fix MJpegDecoder heap buffer overflows in LoadFrame (reallocate
databuf_[i] when scanlines_size changes), DecodeToBuffers, and
DecodeToCallback (clamp scanlines_to_copy to lines_left and align
skip to max_v_samp_factor).
14. Replace width & (vl - 1) tail calculations with post-loop remainder
across 18 SVE2 functions in row_sve.h for non-power-of-2 SVE VL,
and remove duplicate label 2: in UYVYToARGBRow_SVE_SC.
15. Handle width & 15 tail pixels in DetileToYUY2_C and validate
IS_POWEROFTWO(tile_height) in DetileSplitUVPlane and DetileToYUY2.
16. Avoid signed 32-bit overflow in DivideRow_16_C and C16TO8, fix
ClampFloatToZero_SSE2 pointer increment (lea 0x4) and loop counter
(sub $0x1), and initialize xmm3 in ARGBCopyYToAlphaRow_SSE2.
17. Check max leaf 7 subleaf and max extended CPUID leaf in GetCpuFlags
before querying subleaf 7:1 and extended leaf 0x80000021.
18. Free plane_u on error in I422ToNV21, and fix 4x/2x scratch row
over-allocation in ScaleARGBBilinearDown and ScaleUVBilinearDown.
19. Add ARGBSetRow_RVV and ARGBShuffleRow_RVV in row_rvv.cc.
20. Fix NEON mask (15 -> 7) in ScaleUVRowUp2_Linear_Any_NEON /
_16_Any_NEON, use SIMD InterpolateRow + SplitUVRow in NV16ToI420,
wire SSSE3/SSE2 in ARGBToI400Matrix / ARGBToYUY2Matrix /
ARGBToUYVYMatrix and RGB24Mirror, and split HalfFloatRow multiply
into (src * scale) * 2^-112 to avoid 50x Intel denormal stalls.
Bug: b/574714081
Test: libyuv_unittest
Change-Id: Iec3ae5c58d52bee3ef13cf9b765ac84b733aa576
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8601931
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
1275x1650 (US Letter 150 DPI, 1000 iterations) speedup:
CPU ARGBExtractAlpha ARGBCopyYToAlpha ARGBMirror RGB24Mirror
-----------------------------------------------------------------------
Intel Granite Rapids 1.00x 1.00x 1.00x 1.04x
Intel Emerald Rapids 1.01x 0.97x 1.03x 1.07x
Intel Sapphire Rapids 0.97x 1.01x 1.08x 1.26x
Intel Ice Lake 0.99x 1.02x 0.97x 1.08x
Intel Skylake-X 1.04x 1.03x 1.01x NA
Intel Haswell NA NA NA NA
AMD Turin (Zen 5) 1.01x 1.10x 1.25x 1.31x
AMD Genoa (Zen 4) 0.97x 1.04x 1.09x 1.42x
AMD Milan (Zen 3) NA NA NA NA
AMD Rome (Zen 2) NA NA NA NA
NVIDIA Grace (V2) 1.57x 2.51x NA NA
Google Axion 2 1.12x 3.17x NA NA
Ampere Siryn 1.06x 1.70x NA NA
ARM Neoverse-N1 1.49x 2.01x NA NA
Google Axion (Maple) 1.47x 2.03x NA NA
ARM Cortex-A53 1.06x 1.69x NA NA
Add AVX512BW ARGBExtractAlphaRow, ARGBCopyYToAlphaRow and
ARGBMirrorRow, and AVX512VBMI RGB24MirrorRow, in row_gcc.cc and as
intrinsics in row_win.cc. They are x64 only. The remainder is done
with a k-mask (bzhi) zero-masked load and a masked store, so they
handle any width without an Any wrapper and never access memory
outside the row.
RGB24MirrorRow_AVX512VBMI loads and stores 3 full zmm vectors per
64 pixels using 3 vpermi2b. The one output byte that comes from the
third input vector is merged with a masked vpbroadcastb. The 1 to 63
pixel tail rebases the same tables by 192 - 3 * r; indices >= 128 are
taken from the third vector with vpmovb2m and a masked vpermb.
Add ARGBCopyYToAlphaRow_NEON for Arm 32 bit (vld4 + vst4) and
AArch64 (zip + sli), and rewrite AArch64 ARGBExtractAlphaRow_NEON
with ldp + uzp2. Both prefetch source and destination, which is
2.5x faster on Neoverse-N1 and 1.2x to 1.5x faster on Cortex-A53
for 1280x720. Add WASM SIMD ARGBExtractAlphaRow and
ARGBCopyYToAlphaRow, using a swizzle table for CopyYToAlpha. Rework
the RVV versions to use unit stride loads with vnsrl and vzext.vf4.
R=rrwinterton@gmail.com
Bug: None
Change-Id: I436f27d70a3db0a2b176c91eb789a9491133cfcb
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8523531
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
I420ScaleTo569x480_Bilinear (LIBYUV_WIDTH=569 LIBYUV_REPEAT=5000):
CPU AVX2 AVX512 Speedup
Intel Emerald Rapids 124 ms 63 ms 1.97x
Intel Sapphire Rapids 134 ms 69 ms 1.94x
Intel Skylake-X 219 ms 129 ms 1.70x
Intel Granite Rapids 124 ms 76 ms 1.63x
Intel Ice Lake 164 ms 101 ms 1.62x
Bug: None
Change-Id: I7a10446eb4dbcebae68380c2a186d4e3badec551
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8483371
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
P010 and P210 were auxiliary FourCCs and P410 did not exist, so
ConvertToI420, ConvertToARGB, and ConvertFromI420 could not consume or
produce them. Callers had to pick I010/I210 helpers and pack UV by hand.
NV12Scale existed, but NV16/NV24 and P010/P210/P410 had no Scale APIs.
Promote P010 and P210 to primary FourCCs, add FOURCC_P410, and add the
I4xx/Px10 converters those entry points need: I410ToP410, I412ToP412,
P210ToI210, P410ToI410, P010/P210/P410ToI420, I420ToP010/P210/P410,
I422ToP210, I444ToP410, and P410ToARGB/AR30Matrix.
Add NV16Scale (same path as NV12Scale: ScalePlane + UVScale).
Add P010Scale, P210Scale, and P410Scale: 10-bit samples stay in the MSBs, so Y/UV convert to 10-bit LSB, ScalePlane_16, then back to MSB.
UVScale_16 cannot general-scale, so UV is split, scaled as planes, and merged.
Packed-plane layout, odd-width strides, ConvertTo/From round-trips, and
the existing scale factor/size/swap tests cover the new APIs.
Bug: 537818862
Change-Id: I1ae5e6cad5e6d83acf1d0df69f94c23688bbe0e8
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8346569
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- Also remove redundant neon check for I8MM kernels
======================================================================
CPU Before After Speedup
----------------------------------------------------------------------
Intel Granite Rapids (AVX10.1) 251 ms 141 ms 1.78x (+43.8%)
Intel Emerald Rapids (AVX512) 199 ms 108 ms 1.84x (+45.7%)
Intel Sapphire Rapids (AVX512) 230 ms 112 ms 2.05x (+51.3%)
AMD Turin Zen 5 (AVX512) 156 ms 99 ms 1.58x (+36.5%)
AMD Venice Zen 6 (AVX512) 204 ms 106 ms 1.92x (+48.0%)
Intel Skylake-X (AVX512) 320 ms 156 ms 2.05x (+51.2%)
AMD Genoa Zen 4 (AVX512) 236 ms 346 ms 0.68x (-46.6%)
Intel Ice Lake (AVX512) 242 ms 316 ms 0.77x (-30.6%)
AMD Milan Zen 3 (AVX2) 252 ms 230 ms 1.10x (+8.7%)
Intel Haswell (AVX2) 369 ms 456 ms 0.81x (-23.6%)
NVIDIA Grace (SVE2) 336 ms 337 ms 1.00x (-0.3%)
ARM Neoverse-N1 (NEON) 1081 ms 1105 ms 0.98x (-2.2%)
======================================================================
Bug: 537818862
Change-Id: I3ddeddb040fc52a30dabe7ae2c71d16dda021568
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8343467
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Add missing RGBToYMatrixRow_Any_{LSX,LASX} and kAddY offset fix so
RAW/RGB24 to YUV works on LoongArch.
Change-Id: I4c990a3705d7f521fa6d0c9e7e01f1e36f81c379
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8292090
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reduces include/libyuv/basic_types.h to only define LIBYUV_API.
Directly includes <stdint.h> and <stddef.h> where needed in public
headers.
Replaces LIBYUV_BOOL, LIBYUV_FALSE, and LIBYUV_TRUE with C++ bool,
true, and false in C++ code (MJpegDecoder, unit tests) and int (0, 1)
for C API functions.
Updates documentation and unit tests.
Test: ninja -C out/Release libyuv_unittest && ./out/Release/libyuv_unittest --gtest_filter=*
Change-Id: I2d58e872f0f976263bcda546dcc3b1b5f7e53d58
TAG=agy
CONV=2730bc56-93ff-48ed-b615-dcde1b834394
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280662
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
- MergeUVRow_SVE2: Interleaves U and V
- SplitUVRow_SVE2: De-interleaves UV
- InterpolateRow_SVE2: Bilinear row interpolation
- YUY2ToYRow_SVE2, YUY2ToUVRow_SVE2, UYVYToYRow_SVE2, UYVYToUVRow_SVE2
- HalfMergeUVRow_SVE2: 2:1 vertical averaging of U and V
OnePlus 15 with Qualcomm Snapdragon 8 Elite Gen 5 Oryon V3
Benchmark (1280x720 1000x) | SVE2 | NEON | Result
-------------------------------+--------+--------+------------------
TestInterpolatePlane | 119 ms | 164 ms | SVE2 is 1.38x win
TestInterpolatePlane_16 | 138 ms | 172 ms | SVE2 is 1.25x win
SplitUVPlane_Opt | 68 ms | 69 ms | SVE2 is 1.01x win
MergeUVPlane_Opt | 40 ms | 40 ms | Tied
SwapUVPlane_Opt | 72 ms | 72 ms | Tied
HalfMergeUVPlane_Opt | 50 ms | 50 ms | Tied
I420ToNV12_Opt | 25 ms | 26 ms | SVE2 is faster
I420ToNV21_Opt | 26 ms | 26 ms | Tied
YUY2ToNV12_Opt | 79 ms | 81 ms | SVE2 is faster
NV21ToNV12_Opt | 36 ms | 35 ms | Tied
Android420ToI420ToNV12_Opt | 53 ms | 53 ms | Tied
Android420ToI420ToNV21_Opt | 35 ms | 33 ms | NEON is faster
YUY2ToI420_Opt | 87 ms | 84 ms | NEON is faster
UYVYToI420_Opt | 87 ms | 83 ms | NEON is faster
YUY2ToI422_Opt | 101 ms | 97 ms | NEON is faster
UYVYToI422_Opt | 101 ms | 97 ms | NEON is faster
UYVYToNV12_Opt | 111 ms | 108 ms | NEON is faster
ARGBToNV21_Opt | 279 ms | 279 ms | Tied
ABGRToNV21_Opt | 279 ms | 280 ms | SVE2 is faster
ARGBToNV12_Opt | 285 ms | 280 ms | NEON is faster
ABGRToNV12_Opt | 284 ms | 281 ms | NEON is faster
Bug: 42280902
Change-Id: I53f633fe453f2853f7bc9fe17280e938912c4e8f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8196068
Reviewed-by: Justin Green <greenjustin@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
RAW format in libyuv is Red-first ([R, G, B] in memory), whereas RGB24
is Blue-first ([B, G, R] in memory).
RGBToYMatrixRow and RGBToUVMatrixRow internally expand 3-byte input
into 4-byte [byte0, byte1, byte2, 255]. For RAW, this buffer becomes
[R, G, B, 255] (matching ABGR byte order). RAWToI420 previously passed
&kArgbI601Constants (which expects [B, G, R, 255]) instead of
&kAbgrI601Constants, causing Red and Blue channels to be swapped.
Pass &kAbgrI601Constants in RAWToI420, fix parameter names in
source/convert.cc, and add a unit test.
Test: libyuv_unittest --gtest_filter=*RAWToI420*
Bug: libyuv:537503370
TAG=agy
CONV=2c54c4b5-f515-421b-80b3-079caa9f948b
Change-Id: Ic72470813eaf15e80046c2c01cf804cb827039ab
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8133048
Reviewed-by: richard winterton <rrwinterton@gmail.com>
The Convert16To8Row function pointer in I010ToNV12 is only ever called
with halfwidth for the chroma planes (the Y plane goes through
Convert16To8Plane which has its own dispatch), but its SIMD variants
were selected based on the alignment of the full luma width. When width
is a multiple of 32 but halfwidth is not, the bare AVX2 kernel was
selected and over-read the chroma sources and over-wrote the temporary
row buffer. Match the MergeUVRow dispatch in the same function and key
on halfwidth.
Bug: chromium:514748734
Change-Id: I4c2a42857828cdcc6505ebaa65d5b6a95b0f7e55
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8031724
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Validate that an int parameter is not equal to INT_MIN before negating
it.
Remove redundant src_width > 32768 || src_height > 32768 checks in
callers of ScalePlane(), ScalePlane_16(), ScalePlane_12(), and
UVScale().
Change UVScale() to validate its parameters in the same way as
ScalePlane(), ScalePlane_16(), and ScalePlane_12().
Change-Id: I64e03257cf090760030c966b49c4d23e4cec25e5
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7902889
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Treat height == INT_MIN as invalid. Omit explicit height == INT_MIN
check if we disallow height < 32768.
Perform multiplications of stride in the ptrdiff_t type.
Add checks for invalid width and height to some functions.
Bug: 518806561
Change-Id: I5e39fffed7f806852a8758d4b59df919839c0a3b
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7891415
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Adds a safety check to prevent signed integer overflow in the UV
plane coalescing logic within NV12ToI420. This ensures that
halfwidth * halfheight does not overflow INT_MAX, matching the Y
plane coalescing check and preventing potential undefined behavior
(signed integer overflow) which could lead to negative widths being
passed to SIMD functions.
Test: libyuv_unittest --gtest_filter=*NV12Crop*
Bug: None
CONV=6401df25-4d5d-4595-a231-f72c2c8e78df
TAG=agy
R=wtc@google.com
Change-Id: I15a51609a1e000a82f4b6958b4ada444efb1f2f4
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7886824
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
- Standardize libyuv ARGB-family (ARGB, ABGR, RGBA, BGRA) to YUV conversion by utilizing the generic MatrixRow architecture and explicit ArgbConstants.
- Consolidated ARGBToI420, ABGRToI420, BGRAToI420, and RGBAToI420 as wrappers for ARGBToI420Matrix.
- Refactored ABGRToJ420, ABGRToJ422, and ABGRToI422 to use generic matrix functions.
- Added matrix-based versions for NV21, I400, YUY2, and UYVY.
- Updated RAW and RGB24 to I420/I422/I444 dispatchers to use MatrixRow logic and explicit constants.
- Fixed parameter swap bugs in ARGBToI422, ARGBToJ422, and ABGRToJ422.
- Fixed a bug in the generic C implementation of matrix row functions ensuring all 4 channels are processed correctly for all ARGB-family formats.
- Moved kShuffleAARRGGBB in row_gcc.cc to the top of the libyuv namespace for visibility.
- Cleaned up redundant format-specific row implementations.
Bug: libyuv:42280902
Change-Id: I67ffa4c476abc0d2dcc4650510d7bda91b65988e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7830291
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
This consolidation standardizes conversion logic, improves code
maintainability, and provides flexible support for various color spaces
(e.g., BT.601, JPEG full
range).
Key Modifications:
- Function Consolidation: Refactored several high-level conversion functions into lightweight wrappers around generic Matrix variants:
- ARGBToI420 → ARGBToI420Matrix
- ARGBToI444 → ARGBToI444Matrix
- ARGBToI422 → ARGBToI422Matrix
- ARGBToNV12 → ARGBToNV12Matrix
- RAWToJ400, RGB24ToJ400 → RGBToI400Matrix
- RAWToI444, RAWToJ444 → RGBToI444Matrix
- 2-Pass Conversions: Updated RGB565ToI420, ARGB1555ToI420, and ARGB4444ToI420 to utilize 2-pass conversions via RGBToI420Matrix.
- Standardization: Refactored ARGBToNV21, ARGBToYUY2, and ARGBToUYVY to use parameterized matrix row functions (ARGBToYMatrixRow,
ARGBToUVMatrixRow).
- Legacy Cleanup: Replaced legacy calls to ARGBToYJRow with the parameterized ARGBToYMatrixRow in the ARGBSobelize helper.
- Internal Integration: Included libyuv/convert_from_argb.h in planar_functions.cc and ensured all new matrix symbols are properly
declared/exported (LIBYUV_API).
Bug: libyuv:42280902
Change-Id: Ied5fd9899767427e3a03cdcfbeaff3e9d502374a
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7822033
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- implement wrappers with RAW, RGB24, NV21 and JNV21 to call it.
Zen5
Was [ OK ] LibYUVConvertTest.RAWToJNV21_Opt (1146 ms)
Now [ OK ] LibYUVConvertTest.RAWToJNV21_Opt (1446 ms)
reason - the new code uses 1 pass for RAWToY but 2 pass for RAWToARGB,ARGBToUV. needs 1 RGBToUV
Bug: libyuv:42280902
Change-Id: Ife6fbed0829484045409e6d42b85cec1d1fd6052
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7780026
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
Adds RGBToYMatrixRow_AVX2 which reads 24 bit RGB values by reading 3 vectors instead of 4 and permutes them into 4 ARGB vectors before conversion.
Also adds RGBToYMatrixRow_Opt and RGBToYMatrixRow_2Step_Opt to convert_argb_test.cc to benchmark and compare the direct AVX2 conversion vs a 2-step approach.
./libyuv_test '--gunit_filter=*RAWToJ400_Opt' --libyuv_width=1280 --libyuv_height=720 --libyuv_repeat=10000 --libyuv_flags=-1 --libyuv_cpu_info=-1
AMD Zen 5
Was LibYUVConvertTest.RAWToJ400_Opt (757 ms)
Now LibYUVConvertTest.RAWToJ400_Opt (699 ms)
Intel Skylake
Was LibYUVConvertTest.RAWToJ400_Opt (1705 ms)
Now LibYUVConvertTest.RAWToJ400_Opt (1426 ms)
Bug: 477295731
Change-Id: I29866baf4ad5fe7a3725e4a01f2fe24649510a7d
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7777325
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Justin Green <greenjustin@google.com>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
This change implements ARGBToUV444MatrixRow_RVV, ARGBToUVMatrixRow_RVV,
and their wrappers (ARGBToUVRow_RVV, ARGBToUVJRow_RVV, etc.) using RVV
intrinsics, mirroring the NEON/AVX2 designs. It wires them into the
build and dispatch systems.
LIBYUV_RVV_HAS_TUPLE_TYPE is always true on new compilers. This macro
has been removed, assuming it is true everywhere, reducing the amount of
code in row_rvv.cc, scale_rvv.cc, and row.h.
Tested via: ~/bin/doyuv3v && ~/bin/runyuv3v TestARGBToI444Matrix
~/bin/doyuv3av
Bug: libyuv:42280902
Change-Id: I36d305386b297d69023c068aa9c62ab6b2ad039c
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7769956
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Benchmark on Icelake Xeon
Now AVX512BW:
[ OK ] LibYUVConvertTest.ARGBToNV12_Opt (1723 ms)
Was AVX2:
[ OK ] LibYUVConvertTest.ARGBToNV12_Opt (2144 ms)
- Added `ARGBToUVMatrixRow_AVX512BW` implementation in `source/row_gcc.cc`.
- Added corresponding `ARGBToUVRow_AVX512BW` and `ABGRToUVRow_AVX512BW` functions.
- Added unaligned wrappers `ARGBToUVRow_Any_AVX512BW` and `ABGRToUVRow_Any_AVX512BW` in `source/row_any.cc`.
- Updated `source/row_any.cc` to correctly size `vin` and `vout` buffers for AVX512BW width and adjusted the `ANY12MS` and `ANY12S` macros to handle `MASK=63`.
- Updated `include/libyuv/row.h` with the required AVX512BW headers and definitions, scoped appropriately.
- Wired all callers of `ARGBToUVRow_AVX2` and related functions in `source/convert.cc` and `source/convert_from_argb.cc` to dynamically use the `AVX512BW` implementations if the CPU flag indicates AVX-512BW support.
- Optimized AVX-512 code to generate the `-1` multiplier in a single instruction (`vpternlogd`) and reused it across word (`vpmaddwd`) dot products. Handled the resulting negation by replacing a subtraction with `vpaddw` offset adjustment.
Bug: 477295731
R=dalecurtis@chromium.org, rrwinterton@gmail.com
Change-Id: Ida5fb27e59ae4c1c3824737f009b80549cd20a06
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7763257
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
I have successfully ported the usage of ARGBToYRow_AVX2 to dynamically detect and utilize ARGBToYRow_AVX512BW when available.
Here's a summary of the changes:
1. Source Modifications: In both source/convert.cc and source/convert_from_argb.cc, I searched for all references where ARGBToYRow_AVX2 was
being conditionally used (which operates on 32 pixels).
2. AVX512BW Detection: Immediately following those blocks, I injected a new check for kCpuHasAVX512BW. If the CPU flag is present, the logic
now utilizes ARGBToYRow_Any_AVX512BW by default, falling back to the fully aligned ARGBToYRow_AVX512BW when the width is aligned to 64
bytes.
3. Profiling: After building and compiling the tests (doyuv3x), I validated the change using perfyuv3 ARGBToNV12_Opt | cat. The test
successfully executed and the performance profile indicated that ARGBToYRow_AVX512BW successfully executed (taking up ~18% of CPU cycles,
replacing the previous AVX2 specific instruction overhead for the Y row extraction).
The HAS_ARGBTOYROW_AVX512BW macro implementation now fully supports all AVX2 conversion paths to utilize AVX512BW when the system processor
flags allow it!
R=richard, rrwinterton@gmail.com
Change-Id: Iad811e12d301f5621e6f6d039105420861ade43e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7760779
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
- add ARGBToYMatrixRow_AVX512BW
- refactor SSE and AVX to use Matrix functions, making old functions
call the new ones.
Zen5 1280x720
Was AVX2 LibYUVConvertTest.ARGBToI444_Opt (1125 ms)
Now AVX512 LibYUVConvertTest.ARGBToI444_Opt (641 ms)
Details by Gemini:
1. Created 3 new Matrix functions:
Added ARGBToYMatrixRow_SSSE3, ARGBToYMatrixRow_AVX2, and
ARGBToYMatrixRow_AVX512BW to source/row_gcc.cc. These take the
const struct ArgbConstants* c parameter similarly to
ARGBToUV444MatrixRow_*. The x86 vector instructions dynamically
calculate the needed values using the properties of the constants
struct, including using vpmaddwd inside the AVX512 code to offset
the lack of a native vphaddw.
2. Replaced Old Functions with Wrappers:
Modified the existing implementations of ARGBToYRow_SSSE3,
ARGBToYJRow_SSSE3, ABGRToYRow_SSSE3, ABGRToYJRow_SSSE3,
RGBAToYRow_SSSE3, RGBAToYJRow_SSSE3, BGRAToYRow_SSSE3 (and their
_AVX2 equivalents) in source/row_gcc.cc to act as inline wrappers
calling the new ARGBToYMatrixRow_* functions, passing the right
matrix parameters (e.g. &kArgbI601Constants, &kArgbJPEGConstants,
&kAbgrI601Constants).
3. Added row_any.cc Handlers:
Added ANY11MC definitions to source/row_any.cc to autogenerate
ARGBToYMatrixRow_Any_SSSE3, ARGBToYMatrixRow_Any_AVX2, and
ARGBToYMatrixRow_Any_AVX512BW which safely handles non-aligned
tails.
4. Updated include/libyuv/row.h:
Updated the headers with the proper void declarations for all newly
generated Matrix and Any_ variants. Also defined
HAS_ARGBTOYROW_AVX512BW in the CPU macros.
5. Tested the Implementations:
Compiled and tested on Linux x86, which resulted in all tests passing
cleanly. Also successfully completed all Windows 32-bit build checks
ensuring 32-bit regression prevention without issues.
Bug: 477295731
Change-Id: I4f5eec9a961e24a9d760d0a1c0810fb5e29a0bd1
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7759494
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: richard winterton <rrwinterton@gmail.com>
These are about 25% faster than the C versions.
Bug: libyuv:42280902
Change-Id: I8b298670ee5f3ed5db35527fc41d6d9a51b020a1
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7573682
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
This allows for ABGR conversion using the same methods
Bug: libyuv:42280902
Change-Id: I5566e3150b30573a2326a900ce31ab095f8935f9
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7564316
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
