380 Commits

Author	SHA1	Message	Date
Frank Barchard	d2a6496353	[libyuv] Add LoongArch LASX parity, cpucfg CPUID, and 38 convert rows ... Follow up on the LoongArch *_la.cc consolidation with runtime CPU detection, build/test disable flags, full LASX parity for all LSX functions, and 38 new LSX/LASX convert and planar row functions: - Update LoongArchCpuCaps() in cpu_id.cc to detect LSX and LASX via the unprivileged cpucfg instruction combined with getauxval(AT_HWCAP) OS support, and wire GetCpuFlags() for __loongarch__. - Support LIBYUV_DISABLE_LSX, LIBYUV_DISABLE_LASX, and LIBYUV_DISABLE_ASM at build time in cpu_support.h and CMakeLists.txt, and fix the __loongarch__ environment variable check in unit_test.cc. - Port all 24 LSX-only Rotate and Scale kernels to 256-bit LASX so every LSX function has a matching LASX implementation. - Move ScaleMaxSamples, ScaleSumSamples, and ScaleSamples from row_* to scale_* across all platforms and add LSX and LASX implementations in scale_la.cc. - Add 38 LSX and LASX row implementations (and _Any wrappers): SwapUVRow, ARGBCopyYToAlphaRow, HalfMergeUVRow, Convert16To8Row, Convert8To16Row, MultiplyRow_16, DivideRow_16, MergeUVRow_16, SplitUVRow_16, CopyRow_16, HalfRow, HalfRow_16, ARGBCopyAlphaRow, YUY2ToNVUVRow, AR64ToARGBRow, AB64ToARGBRow, ARGBToAR64Row, ARGBToAB64Row, Convert8To8Row, HalfRow_16To8, AR30ToAB30Row, ARGBToAR30Row, ABGRToAR30Row, AR30ToARGBRow, AR30ToABGRRow, AYUVToYRow, AYUVToUVRow, AYUVToVURow, DetileRow, DetileRow_16, MergeARGBRow, MergeXRGBRow, SplitARGBRow, SplitXRGBRow, RAWToRGBARow, RGB24MirrorRow, BlendPlaneRow, and InterpolateRow_16. Compare SIMD ISA per *_C: ------------------------------------------------------------------------ C Function x86 LoongArch ------------------------------------------------------------------------ ARGBDetectRow_C AVX2 LSX+LASX HammingDistance_C SSSE3+AVX2+AVX512BW+AVX512VNNI LSX+LASX HashDjb2_C SSE41+AVX2+AVX512BW LSX+LASX Ssim4x4Row_C SSE2+AVX2+AVX512BW LSX+LASX SsimWindowRow_C SSE2+AVX2+AVX512BW LSX+LASX SumSquareError_C SSE2+AVX2+AVX512BW+AVX512VNNI LSX+LASX ------------------------------------------------------------------------ Compare: 6 (100%) 6 (100%) 6 (100%) ------------------------------------------------------------------------ Convert SIMD ISA per *_C: ------------------------------------------------------------------------ C Function x86 LoongArch ------------------------------------------------------------------------ AB64ToARGBRow_C SSSE3+AVX2 LSX+LASX ABGRToAR30Row_C SSSE3+AVX2+AVX512BW LSX+LASX AR30ToAB30Row_C AVX2+AVX512BW LSX+LASX AR30ToABGRRow_C AVX2+AVX512BW LSX+LASX AR30ToARGBRow_C AVX2+AVX512BW LSX+LASX AR64ShuffleRow_C SSSE3+AVX2+AVX512BW LSX+LASX AR64ToAB64Row_C SSSE3+AVX2+AVX512BW LSX+LASX AR64ToARGBRow_C SSSE3+AVX2 LSX+LASX ARGB1555ToARGBRow_C AVX2+AVX512BW LSX+LASX ARGB1555ToUVMatrixRow_C AVX2 ARGB1555ToUVRow_C AVX2 LSX+LASX ARGB1555ToYMatrixRow_C AVX2 ARGB1555ToYRow_C AVX2 LSX+LASX ARGB4444ToARGBRow_C AVX2 LSX+LASX ARGB4444ToUVMatrixRow_C AVX2 ARGB4444ToUVRow_C AVX2 ARGB4444ToYMatrixRow_C AVX2 ARGB4444ToYRow_C AVX2 ARGBAddRow_C SSE2+AVX2 LSX+LASX ARGBAffineRow_C AVX2 ARGBAlphaBlendRow_C SSSE3+AVX2+AVX512BW LSX+LASX ARGBAttenuateRow_C SSSE3+AVX2+AVX512BW LSX+LASX ARGBBlendRow_C AVX2+AVX512BW LSX+LASX ARGBColorMatrixRow_C AVX2+AVX512VNNI LSX+LASX ARGBColorTableRow_C AVX2 ARGBCopyAlphaRow_C SSE2+AVX2 LSX+LASX ARGBCopyYToAlphaRow_C SSE2+AVX2+AVX512BW LSX+LASX ARGBExtractAlphaRow_C SSE2+AVX2+AVX512BW LSX+LASX ARGBGrayRow_C AVX2+AVX512BW+AVX512VNNI LSX+LASX ARGBLumaColorTableRow_C AVX2 ARGBMirrorRow_C SSE2+AVX2+AVX512BW LSX+LASX ARGBMultiplyRow_C SSE2+AVX2+AVX512BW LSX+LASX ARGBOrRow_C SSE2+AVX2+AVX512BW LSX+LASX ARGBPolynomialRow_C SSE2+AVX2 ARGBQuantizeRow_C AVX2 LSX+LASX ARGBSepiaRow_C AVX2+AVX512VNNI LSX+LASX ARGBSetRow_C AVX2 LSX+LASX ARGBShadeRow_C AVX2 LSX+LASX ARGBShuffleRow_C SSSE3+AVX2+AVX512BW LSX+LASX ARGBSubtractRow_C SSE2+AVX2 LSX+LASX ARGBToAB64Row_C SSSE3+AVX2 LSX+LASX ARGBToABGRRow_C SSSE3+AVX2+AVX512BW LSX+LASX ARGBToAR30Row_C SSSE3+AVX2+AVX512BW LSX+LASX ARGBToAR64Row_C SSSE3+AVX2 LSX+LASX ARGBToARGB1555Row_C AVX2 LSX+LASX ARGBToARGB4444Row_C AVX2 LSX+LASX ARGBToBGRARow_C SSSE3+AVX2+AVX512BW LSX+LASX ARGBToRAWRow_C SSSE3+AVX2+AVX512BW LSX+LASX ARGBToRGB24Row_C SSSE3+AVX2+AVX512BW LSX+LASX ARGBToRGB565DitherRow_C AVX2+AVX512BW LSX+LASX ARGBToRGB565Row_C AVX2+AVX512BW LSX+LASX ARGBToRGBARow_C SSSE3+AVX2+AVX512BW LSX+LASX ARGBToUV444MatrixRow_C SSSE3+AVX2+AVX512BW+AVX512VNNI+AVX512VNNIVBMI ARGBToUVMatrixRow_C SSSE3+AVX2+AVX512BW+AVX512VNNI+AVX512VNNIVBMI ARGBToYMatrixRow_C SSSE3+AVX2+AVX512BW+AVX512VNNI+AVX512VNNIVBMI LSX+LASX ARGBUnattenuateRow_C SSE2+AVX2 AYUVToUVRow_C AVX2+AVX512VBMI LSX+LASX AYUVToVURow_C AVX2+AVX512VBMI LSX+LASX AYUVToYRow_C AVX2+AVX512VBMI LSX+LASX BlendPlaneRow_C SSSE3+AVX2+AVX512BW LSX+LASX ByteToFloatRow_C AVX2 ComputeCumulativeSumRow_C AVX2 Convert16To8Row_C SSSE3+AVX2+AVX512BW LSX+LASX Convert8To16Row_C SSE2+AVX2+AVX512BW LSX+LASX Convert8To8Row_C AVX2+AVX512BW LSX+LASX CopyRow_16_C AVX2 LSX+LASX CopyRow_C ERMS+SSE2+AVX+AVX512BW LSX+LASX CumulativeSumToAverageRow_C AVX2 DetileRow_16_C AVX2 LSX+LASX DetileRow_C AVX2 LSX+LASX DetileSplitUVRow_C AVX2 DetileToYUY2_C AVX2 DivideRow_16_C AVX2+AVX512BW LSX+LASX GaussCol_C AVX2 GaussCol_F32_C AVX2 GaussRow_C AVX2 GaussRow_F32_C AVX2 HalfFloatRow_C F16C+AVX2+AVX512BW LSX+LASX HalfMergeUVRow_C SSSE3+AVX2+AVX512BW LSX+LASX HalfRow_16To8_C SSSE3+AVX2+AVX512BW LSX+LASX HalfRow_16_C AVX2 LSX+LASX HalfRow_C AVX2 LSX+LASX HalfWidthRow_16To8_C SSSE3+AVX2+AVX512BW HalfWidthRow_16To8_Odd_C AVX2 I210AlphaToARGBRow_C SSSE3+AVX2+AVX512BW I210ToAR30Row_C SSSE3+AVX2+AVX512BW I210ToARGBRow_C SSSE3+AVX2+AVX512BW I212ToAR30Row_C SSSE3+AVX2+AVX512BW I212ToARGBRow_C SSSE3+AVX2+AVX512BW I400ToARGBRow_C SSE2+AVX2+AVX512BW LSX+LASX I410AlphaToARGBRow_C SSSE3+AVX2+AVX512BW I410ToAR30Row_C SSSE3+AVX2+AVX512BW I410ToARGBRow_C SSSE3+AVX2+AVX512BW I422AlphaToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX I422ToAR30Row_C SSSE3+AVX2+AVX512BW I422ToARGB1555Row_C SSSE3+AVX2+AVX512BW LSX+LASX I422ToARGB4444Row_C SSSE3+AVX2+AVX512BW LSX+LASX I422ToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX I422ToRGB24Row_C SSSE3+AVX2+AVX512BW+AVX512VBMI LSX+LASX I422ToRGB565Row_C SSSE3+AVX2+AVX512BW LSX+LASX I422ToRGBARow_C SSSE3+AVX2+AVX512BW LSX+LASX I422ToUYVYRow_C SSE2+AVX2 LSX+LASX I422ToYUY2Row_C SSE2+AVX2 LSX+LASX I444AlphaToARGBRow_C SSSE3+AVX2+AVX512BW I444ToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX I444ToRGB24Row_C SSSE3+AVX2+AVX512VBMI InterpolateRow_16_C AVX2+AVX512BW LSX+LASX InterpolateRow_C AVX2+AVX512BW LSX+LASX J400ToARGBRow_C AVX2+AVX512BW LSX+LASX MergeAR64Row_C AVX2 MergeARGB16To8Row_C AVX2 MergeARGBRow_C SSE2+AVX2 LSX+LASX MergeRGBRow_C AVX2+AVX512VBMI MergeUVRow_16_C AVX2+AVX512BW LSX+LASX MergeUVRow_C SSE2+AVX2+AVX512BW LSX+LASX MergeXR30Row_C AVX2 MergeXR64Row_C AVX2 MergeXRGB16To8Row_C AVX2 MergeXRGBRow_C SSE2+AVX2 LSX+LASX MirrorRow_16_C SSSE3+AVX2+AVX512BW LSX+LASX MirrorRow_C SSSE3+AVX2+AVX512BW LSX+LASX MirrorSplitUVRow_C SSSE3+AVX2+AVX512BW LSX+LASX MirrorUVRow_C SSSE3+AVX2+AVX512BW LSX+LASX MultiplyRow_16_C AVX2+AVX512BW LSX+LASX NV12ToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX NV12ToRGB24Row_C SSSE3+AVX2+AVX512VBMI NV12ToRGB565Row_C SSSE3+AVX2 LSX+LASX NV21ToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX NV21ToRGB24Row_C SSSE3+AVX2+AVX512VBMI NV21ToYUV24Row_C SSSE3+AVX2+AVX512VBMI P210ToAR30Row_C SSSE3+AVX2+AVX512BW P210ToARGBRow_C SSSE3+AVX2+AVX512BW P410ToAR30Row_C SSSE3+AVX2+AVX512BW P410ToARGBRow_C SSSE3+AVX2+AVX512BW RAWToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX RAWToRGB24Row_C SSSE3+AVX2+AVX512BW+AVX512VBMI LSX+LASX RAWToRGBARow_C AVX2+AVX512VBMI LSX+LASX RAWToUVRow LSX+LASX RGB24MirrorRow_C SSSE3+AVX2+AVX512VBMI LSX+LASX RGB24ToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX RGB24ToUVRow LSX+LASX RGB565ToARGBRow_C AVX2 LSX+LASX RGB565ToUVMatrixRow_C AVX2 RGB565ToUVRow_C AVX2 LSX+LASX RGB565ToYMatrixRow_C AVX2 RGB565ToYRow_C AVX2 LSX+LASX RGBAToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX RGBColorTableRow_C AVX2 RGBToUV444MatrixRow_C SSSE3+AVX2+AVX512BW RGBToUVMatrixRow_C SSSE3+AVX2+AVX512BW RGBToYMatrixRow_C SSSE3+AVX2+AVX512BW LSX+LASX SetRow_C ERMS+AVX2 LSX+LASX SobelRow_C AVX2 LSX+LASX SobelToPlaneRow_C AVX2 LSX+LASX SobelXRow_C AVX2 SobelXYRow_C AVX2 LSX+LASX SobelYRow_C AVX2 SplitARGBRow_C SSE2+SSSE3+AVX2 LSX+LASX SplitRGBRow_C SSSE3+SSE41+AVX2 SplitUVRow_16_C AVX2 LSX+LASX SplitUVRow_C SSE2+AVX2+AVX512BW LSX+LASX SplitXRGBRow_C SSE2+SSSE3+AVX2 LSX+LASX SwapUVRow_C SSSE3+AVX2+AVX512BW LSX+LASX UYVYToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX UYVYToUV422Row_C SSE2+AVX2+AVX512BW LSX+LASX UYVYToUVRow_C SSE2+AVX2+AVX512BW LSX+LASX UYVYToYRow_C SSE2+AVX2+AVX512BW LSX+LASX UnpackMT2T_C AVX2 YUY2ToARGBRow_C SSSE3+AVX2+AVX512BW LSX+LASX YUY2ToNVUVRow_C SSE2+AVX2 LSX+LASX YUY2ToUV422Row_C SSE2+AVX2+AVX512BW LSX+LASX YUY2ToUVRow_C SSE2+AVX2+AVX512BW LSX+LASX YUY2ToYRow_C SSE2+AVX2+AVX512BW LSX+LASX ------------------------------------------------------------------------ Convert: 173 (100%) 171 (99%) 116 (67%) ------------------------------------------------------------------------ Rotate SIMD ISA per *_C: ------------------------------------------------------------------------ C Function x86 LoongArch ------------------------------------------------------------------------ TransposeUVWx16_C AVX2+AVX512BW LSX+LASX TransposeUVWx8_C AVX2 LSX+LASX TransposeWx16_C AVX2+AVX512BW LSX+LASX TransposeWx4_32_C AVX2 LSX+LASX TransposeWx8_16_C AVX2 LSX+LASX TransposeWx8_C AVX2 LSX+LASX TransposeWxH_32_C AVX2 ------------------------------------------------------------------------ Rotate: 7 (100%) 7 (100%) 6 (86%) ------------------------------------------------------------------------ Scale SIMD ISA per *_C: ------------------------------------------------------------------------ C Function x86 LoongArch ------------------------------------------------------------------------ FixedDiv1_C X86 FixedDiv_C X86 ScaleARGBCols64_C ScaleARGBColsUp2_C AVX2 ScaleARGBCols_C AVX2 LSX+LASX ScaleARGBFilterCols64_C ScaleARGBFilterCols_C AVX2 LSX+LASX ScaleARGBRowDown2Box_C AVX2 LSX+LASX ScaleARGBRowDown2Linear_C AVX2 LSX+LASX ScaleARGBRowDown2_C AVX2 LSX+LASX ScaleARGBRowDownEvenBox_C AVX2 LSX+LASX ScaleARGBRowDownEven_C AVX2 LSX+LASX ScaleAddCols0_C ScaleAddCols1_16_C ScaleAddCols1_C AVX512BW ScaleAddCols2_16_C ScaleAddCols2_C AVX512BW ScaleAddRow_16_C ScaleAddRow_C SSE2+AVX2+AVX512BW LSX+LASX ScaleColsUp2_16_C ScaleColsUp2_C AVX2 ScaleCols_16_C ScaleCols_C ScaleFilterCols64_16_C ScaleFilterCols64_C ScaleFilterCols_16_C ScaleFilterCols_C AVX2 LSX+LASX ScaleMaxSamples_C AVX2 LSX+LASX ScaleRowDown2Box_16_C AVX2 ScaleRowDown2Box_C SSSE3+AVX2+AVX512BW LSX+LASX ScaleRowDown2Box_Odd_C SSSE3+AVX2+AVX512BW ScaleRowDown2Linear_16_C AVX2 ScaleRowDown2Linear_C SSSE3+AVX2+AVX512BW LSX+LASX ScaleRowDown2_16_C AVX2 ScaleRowDown2_C SSSE3+AVX2+AVX512BW LSX+LASX ScaleRowDown34_0_Box_16_C ScaleRowDown34_0_Box_C AVX2 LSX+LASX ScaleRowDown34_16_C ScaleRowDown34_1_Box_16_C ScaleRowDown34_1_Box_C AVX2 LSX+LASX ScaleRowDown34_C AVX2 LSX+LASX ScaleRowDown38_16_C ScaleRowDown38_2_Box_16_C ScaleRowDown38_2_Box_C AVX2 LSX+LASX ScaleRowDown38_3_Box_16_C ScaleRowDown38_3_Box_C AVX2 LSX+LASX ScaleRowDown38_C AVX2 LSX+LASX ScaleRowDown4Box_16_C ScaleRowDown4Box_C SSSE3+AVX2 LSX+LASX ScaleRowDown4_16_C ScaleRowDown4_C SSSE3+AVX2 LSX+LASX ScaleRowUp2_Bilinear_16_Any_C SSE2+AVX2 ScaleRowUp2_Bilinear_16_C SSE2+AVX2 ScaleRowUp2_Bilinear_Any_C SSE2+SSSE3+AVX2 ScaleRowUp2_Bilinear_C SSE2+SSSE3+AVX2 ScaleRowUp2_Linear_16_Any_C SSE2+AVX2 ScaleRowUp2_Linear_16_C SSE2+AVX2 ScaleRowUp2_Linear_Any_C SSE2+SSSE3+AVX2 ScaleRowUp2_Linear_C SSE2+SSSE3+AVX2 ScaleSamples_C AVX2 LSX+LASX ScaleSumSamples_C AVX2 LSX+LASX ScaleUVAddCols1_16_C ScaleUVAddCols1_C ScaleUVAddCols2_16_C ScaleUVAddCols2_C ScaleUVAddCols3_16_C ScaleUVAddCols3_C AVX2+AVX512VBMI LSX+LASX ScaleUVCols64_16_C ScaleUVCols64_C ScaleUVColsUp2_16_C ScaleUVColsUp2_C ScaleUVCols_16_C ScaleUVCols_C ScaleUVFilterCols64_16_C ScaleUVFilterCols64_C ScaleUVFilterCols_16_C ScaleUVFilterCols_C ScaleUVRowDown2Box_16_C ScaleUVRowDown2Box_C AVX2 ScaleUVRowDown2Linear_16_C ScaleUVRowDown2Linear_C AVX2 ScaleUVRowDown2_16_C ScaleUVRowDown2_C AVX2 ScaleUVRowDownEvenBox_16_C ScaleUVRowDownEvenBox_C ScaleUVRowDownEven_16_C ScaleUVRowDownEven_C AVX2 ScaleUVRowUp2_Bilinear_16_Any_C SSE41+AVX2 ScaleUVRowUp2_Bilinear_16_C SSE41+AVX2 ScaleUVRowUp2_Bilinear_Any_C SSSE3+AVX2 ScaleUVRowUp2_Bilinear_C SSSE3+AVX2 ScaleUVRowUp2_Linear_16_Any_C SSE41+AVX2 ScaleUVRowUp2_Linear_16_C SSE41+AVX2 ScaleUVRowUp2_Linear_Any_C SSSE3+AVX2 ScaleUVRowUp2_Linear_C SSSE3+AVX2 ------------------------------------------------------------------------ Scale: 95 (100%) 54 (57%) 24 (25%) ------------------------------------------------------------------------ Bug: 42280942 Change-Id: I663f277db5e8a6a73319df7e3153f9f78a147a7a Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8718861 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-11-04 11:55:35 -08:00
Frank Barchard	d088b8f2bd	[libyuv] Add GCC 15 SME support, SVE2/SME ports, and next 10 SIMD fixes ... 1. Enable ARMv9 SVE2 and SME compilation with GCC 15 and Clang across Android.bp, BUILD.gn, CMakeLists.txt, and linux.mk using portable LIBYUV_SVE_ATTR, LIBYUV_SME_ATTR, and STREAMING_COMPATIBLE macros. 2. Port 46 SVE2 kernels to streaming-compatible SVE_SC functions in row_sve.h and expose _SME entry points for processors with SME but no non-streaming SVE2. Upgrade 15 1x SVE2 kernels to 2x unrolled, use full-vector st1b in ARGBToRGB24Row/ARGBToRAWRow_SVE2, contiguous ld1h/st1h in HalfFloatRow_SVE_SC, and frecpe+frecps reciprocal in ARGBAlphaBlendRow_SVE_SC. 3. Add missing kCpuHasNEON runtime checks in planar_test.cc and define missing HAS_*_NEON macros in row.h for existing NEON kernels. 4. Handle odd width directly in AYUVToUVRow_SVE_SC and AYUVToVURow_SVE_SC and remove redundant width alignment guards in AYUVToNV12 and AYUVToNV21. 5. Implement TransposeWx8_NEON for AArch64 in rotate_neon64.cc so TransposePlane and RotatePlane90/270 cascade 16x -> 8x -> C tail. 6. Add _Any_ wrappers for 9 ARGB/float filter functions (ARGBGrayTo, ARGBGray, ARGBSepia, ARGBColorMatrix, ARGBQuantize, ARGBShade, GaussRow_F32, GaussCol_F32, and ARGBLumaColorTable). 7. Add HalfRow and HalfRow_16 wrappers for AVX2, NEON, SVE2, and SME delegating to InterpolateRow{,_16} with fraction 0x80. 8. Add YUY2ToUV422Row_AVX512BW and UYVYToUV422Row_AVX512BW in row_gcc.cc and row_win.cc. 9. Add HalfWidthRow_16To8_Odd_NEON in row_any.cc and wire into I41xToI420 and Px10ToI420 for odd chroma widths. 10. Fix inline asm clobbers and vzeroupper placement in TransposeWx32_16_AVX512BW and TransposeWx16_32_AVX512BW, and re-enable 36 passing unit tests. Test: libyuv_unittest Bug: 579659306 Change-Id: I6b808f8b03ed424d3af4bb7941c34af05c1cb774 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8701301 Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-11-02 13:46:12 -08:00
Frank Barchard	38ca05f1eb	[libyuv] Fix color matrix, blend, NV layout and null pointer bugs ... Correctness fixes: - U010ToAB30 and U210ToAB30 swap U and V but passed kYuv2020Constants; use kYvu2020Constants as I010ToAB30 does. - H420ToAR30 passed kYvuH709Constants without swapping U and V, so red and blue were swapped; use kYuvH709Constants. - I420Blend with negative height flipped Y but not U and V. - BlendPlaneRow_WASMSIMD processes 16 pixels per loop but was dispatched for widths that are a multiple of 8. - I420Blend used ScaleRowDown2Box_RVV for odd widths, reading 1 byte past the row. Add ScaleRowDown2Box_Odd_RVV. - ConvertFromI420 to NV12/NV21/NV16 with odd width used a UV stride of width, overlapping UV rows and writing 1 byte past the plane. Use ((width + 1) / 2) * 2. - ConvertToI420 (NV16/NV24) and ConvertToARGB (NV12/NV21/NV16/ NV24) computed the Y plane size with the aligned width; the Y stride is src_width. NV24 UV stride is src_width * 2. - I420Rect and I420Interpolate skipped a UV row for negative odd heights. - DetilePlane and DetilePlane_16 accepted tile_height 0, reading past the source. - Add null checks to NV12ToNV21, IxxxToPxxx, I422ToI210, I422ToNV21, YUY2ToI420, UYVYToI420, AYUVToNV12, AYUVToNV21, DetileSplitUVPlane and DetileToYUY2, and avoid pointer arithmetic on optional NULL src_y/dst_y with negative height. MT2TToP010 now returns -1 for NULL src_y with non-NULL dst_y. - Replace unaligned uint32_t stores in row_common.cc with memcpy. - Add LIBYUV_TARGET_AVX512VNNI to HammingDistance_AVX512VNNI so GCC accepts the k1 clobber on non-AVX512 builds. - Check kCpuHasWASMSIMD instead of kCpuHasWASM in scale.cc and scale_uv.cc. - Avoid the 16.16 fixed point NEON/LSX/RVV column scalers for src_width >= 32768, matching the existing AVX2 guard. Performance: - NV24ToI420 uses InterpolateRow, ScaleUVRowDown2Linear and SplitUVRow instead of a scalar loop (bit exact). - Add HalfMergeUVRow_Any_NEON and MirrorSplitUVRow_Any_{NEON, LSX,LASX,WASMSIMD} so non-multiple-of-16 widths use SIMD. - Enable BlendPlaneRow_SSSE3/AVX2 for row_win builds. - Add missing LSX/LASX/RVV/SME dispatch for YUY2ToY, I422ToYUY2/UYVY, ARGBToY/UVMatrix, RAWToNVMatrix, AR64Shuffle, ARGBSobelize and Convert16To8Row in I010ToNV12, and coalesce rows in ARGBToI400Matrix. Cleanup: - UYVYToYRow_Any_AVX2 uses the same UVSHIFT/SBPP as YUY2. - Add #ifdef HAS_* guards in compare_gcc.cc and row_gcc.cc. - Remove unused clobbers and dead cnth/cmp/subs instructions in row_neon.cc, row_neon64.cc, row_sve.h, row_sve.cc and row_sme.cc, and order clobbers "memory", "cc" first. - Re-enable the I420Blend_Any unit test. Test: libyuv_unittest (x86-64 Release, rowwin, ASan, UBSan, MSan; Arm64 SME, SVE2 and NEON on Samsung S26) Bug: 579659306 Change-Id: Ic1afddbf961f7ef7deeba124a55907a31e09822d Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8702355 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-11-02 10:06:18 -08:00
Frank Barchard	79ca9187e5	[libyuv] Fix x86 SIMD OOB, math, alignment, MSan & GCC bugs ... Address x86 SIMD correctness, sanitizer, alignment, and compiler guard bugs across row_gcc.cc, row_win.cc, row_any.cc, row_common.cc, and caller dispatchers: - MSan & row_win.cc: Replace _mm512_ternarylogic_epi32 in row_win.cc (ARGBGrayRow_AVX512VNNI, ARGBToUV444MatrixRow_AVX512VNNIVBMI, and I422ToAR30Row_AVX512BW) with bitwise intrinsics modeled bit-by-bit by LLVM MSan, document the MSan intrinsic rules in GEMINI.md, and fix standalone MSan ELF PT_INTERP in BUILD.gn. Add missing USE_ROW_WIN DivideRow_16_AVX2, HalfMergeUVRow_AVX2, and Convert8To8Row_AVX2. - Out-of-bounds & AVX-512 opmask tails: Fix RGBToUVMatrixRow_Any_SSSE3 MASK from 7 to 15 (24-byte overread), guard UnpackMT2T_C and UnpackMT2T_AVX2 against partial < 80-byte tail blocks, add inline k1-k7 opmask tails to all 17 remaining AVX-512 row kernels (removing all *_Any_AVX512* wrappers), and enable opmask-tailed NV21ToYUV24Row_AVX512VBMI. - Math, saturation & pointer bugs: Fix MirrorRow_16_AVX512BW 64-bit width sign extension, ARGBPolynomialRow_AVX2 vpackssdw clamping, CumulativeSumToAverageRow_AVX2 small-area (area <= 1) handling and vdivss precision, HalfWidthRow_16To8_{SSSE3,AVX2,AVX512BW} unsigned pair summation for samples >= 0x8000, HalfRow_16To8_AVX512BW non-power-of-two scale multiplication, DivideRow_16 SIMD scale == 65536 (16-bit depth) truncation, ARGBLumaColorTableRow_AVX2 unsigned luma weight accumulation, Merge{ARGB,XRGB}16To8Row_AVX2 depth == 8 clamping, custom ArgbConstants::kAddUV loading, 16-bit YUV unsigned shifts/ORs, and null-pointer checks in NV16ToI420/NV24ToI420. - Alignment (#GP), ISA (#UD), UBSan & 32-bit x86: Use unaligned loads for ArgbConstants, YuvConstants, and ARGBPolynomialRow_SSE2 float coefficients; remove AVX2 instructions from HalfFloat{,1}Row_F16C; fix Merge{AR,XR}64Row_AVX2 depth == 16 shift UB and ByteToFloatRow_C / TestByteToFloat float alignment UB; and fix 32-bit -m32 -fPIC ASan inline asm register pressure and operand suffixes. - GCC & compiler guards: Enable GCC_HAS_AVX512 for GCC >= 9 on x86_64 with per-function target attributes, gate HAS_*_AVX2 and HAS_*_AVX512* macros on compiler version macros, add missing #ifdef HAS_* guards in row_gcc.cc, fix _Any / IS_ALIGNED step mismatches, wire missing dispatchers (ARGBSobelize, SplitRotateUV180, HalfMergeUVPlane, ARGBPolynomial), and silence GCC 15 -Werror=stringop-overflow=. Convert SIMD ISA per *_C (* = uncalled in library): ------------------------------------------------------------------------ C Function arm x86 riscv hexagon wasm loong ------------------------------------------------------------------------ AB64ToARGBRow_C NEON AVX2 RVV HVX - - ABGRToAR30Row_C NEON AVX512BW - - - - AR30ToAB30Row_C NEON AVX512BW - - - - AR30ToABGRRow_C NEON AVX512BW - - - - AR30ToARGBRow_C NEON AVX512BW - - - - AR64ShuffleRow_C NEON AVX512BW RVV - RELAXED LASX AR64ToAB64Row_C NEON AVX512BW RVV HVX RELAXED LASX AR64ToARGBRow_C NEON AVX2 RVV HVX - - ARGB1555ToARGBRow_C SVE2 AVX512BW - - SIMD LASX ARGB1555ToUVMatrixRow_C NEON AVX2 - - SIMD - ARGB1555ToUVRow_C * NEON AVX2 - - - LASX ARGB1555ToYMatrixRow_C NEON AVX2 - - SIMD - ARGB1555ToYRow_C * NEON AVX2 - - - LASX ARGB4444ToARGBRow_C NEON AVX2 - - SIMD LASX ARGB4444ToUVMatrixRow_C NEON AVX2 - - SIMD - ARGB4444ToUVRow_C * NEON AVX2 - - - - ARGB4444ToYMatrixRow_C NEON AVX2 - - SIMD - ARGB4444ToYRow_C * NEON AVX2 - - - - ARGBAddRow_C NEON AVX2 - HVX - LASX ARGBAffineRow_C * NEON AVX2 - - - - ARGBAlphaBlendRow_C SME AVX512BW RVV HVX SIMD LASX ARGBAttenuateRow_C NEON AVX512BW RVV HVX - LASX ARGBBlendRow_C NEON AVX512BW RVV HVX RELAXED LASX ARGBColorMatrixRow_C I8MM AVX512VNNI - - - LASX ARGBColorTableRow_C NEON AVX2 - - - - ARGBCopyAlphaRow_C NEON AVX2 - HVX - - ARGBCopyYToAlphaRow_C NEON AVX512BW RVV HVX RELAXED - ARGBExtractAlphaRow_C NEON AVX512BW RVV HVX SIMD LASX ARGBGrayRow_C DOTPROD AVX512VNNI - - - LASX ARGBLumaColorTableRow_C NEON AVX2 - - - - ARGBMirrorRow_C NEON AVX512BW RVV HVX SIMD LASX ARGBMultiplyRow_C SME AVX512BW RVV HVX - LASX ARGBOrRow_C SME AVX512BW RVV HVX SIMD LASX ARGBPolynomialRow_C NEON AVX2 - - - - ARGBQuantizeRow_C NEON AVX2 - - - LASX ARGBSepiaRow_C DOTPROD AVX512VNNI - - - LASX ARGBSetRow_C NEON AVX2 RVV HVX - LASX ARGBShadeRow_C NEON AVX2 - - - LASX ARGBShuffleRow_C NEON AVX512BW RVV - RELAXED LASX ARGBSubtractRow_C NEON AVX2 - HVX - LASX ARGBToAB64Row_C NEON AVX2 RVV HVX - - ARGBToABGRRow_C NEON AVX512BW RVV HVX RELAXED LASX ARGBToAR30Row_C NEON AVX512BW - - - - ARGBToAR64Row_C NEON AVX2 RVV HVX - - ARGBToARGB1555Row_C NEON AVX2 - - SIMD LASX ARGBToARGB4444Row_C NEON AVX2 - - SIMD LASX ARGBToBGRARow_C NEON AVX512BW RVV HVX RELAXED LASX ARGBToRAWRow_C SVE2 AVX512BW RVV - - LASX ARGBToRGB24Row_C SVE2 AVX512BW RVV - - LASX ARGBToRGB565DitherRow_C SVE2 AVX512BW - - SIMD LASX ARGBToRGB565Row_C SVE2 AVX512BW - - SIMD LASX ARGBToRGBARow_C NEON AVX512BW RVV HVX RELAXED LASX ARGBToUV444MatrixRow_C I8MM AVX512VNNI RVV HVX RELAXED - ARGBToUVMatrixRow_C SME AVX512VNNI RVV HVX SIMD - ARGBToYMatrixRow_C SME AVX512VNNI RVV HVX SIMD LASX ARGBUnattenuateRow_C NEON AVX2 - - - - AYUVToUVRow_C SVE2 AVX512VBMI - - - - AYUVToVURow_C SVE2 AVX512VBMI - - - - AYUVToYRow_C NEON AVX512VBMI - - - - BlendPlaneRow_C NEON AVX512BW RVV HVX SIMD - ByteToFloatRow_C NEON AVX2 - - - - ComputeCumulativeSumRow_C NEON AVX2 - - - - Convert16To8Row_C SME AVX512BW RVV HVX - - Convert8To16Row_C SME AVX512BW RVV HVX SIMD - Convert8To8Row_C SME AVX512BW - HVX - - CopyRow_16_C NEON AVX2 - HVX - - CopyRow_C SME AVX512BW RVV HVX - LASX CumulativeSumToAverageRow_C NEON AVX2 - - - - DetileRow_16_C NEON AVX2 - - - - DetileRow_C NEON AVX2 - - - - DetileSplitUVRow_C NEON AVX2 - - - - DetileToYUY2_C NEON AVX2 - - - - DivideRow_16_C SVE2 AVX512BW - HVX - - GaussCol_C * NEON AVX2 - - - - GaussCol_F32_C NEON AVX2 - - - - GaussRow_C * NEON AVX2 - - - - GaussRow_F32_C NEON AVX2 - - - - HalfFloatRow_C SVE2 AVX512BW - - - LASX HalfMergeUVRow_C SVE2 AVX512BW - - - - HalfRow_16To8_C SME AVX512BW RVV HVX - - HalfRow_16_C - AVX2 - - - - HalfRow_C - AVX2 - - - - HalfWidthRow_16To8_C SME AVX512BW RVV - - - HalfWidthRow_16To8_Odd_C - AVX2 - - - - I210AlphaToARGBRow_C SME AVX512BW - - - - I210ToAR30Row_C SME AVX512BW - - - - I210ToARGBRow_C SME AVX512BW - - - - I212ToAR30Row_C SME AVX512BW - - - - I212ToARGBRow_C SME AVX512BW - - - - I400ToARGBRow_C SME AVX512BW RVV HVX - LASX I410AlphaToARGBRow_C SME AVX512BW - - - - I410ToAR30Row_C SME AVX512BW - - - - I410ToARGBRow_C SME AVX512BW - - - - I422AlphaToARGBRow_C SME AVX512BW RVV HVX - LASX I422ToAR30Row_C SME AVX512BW RVV - SIMD - I422ToARGB1555Row_C SME AVX512BW - - SIMD LASX I422ToARGB4444Row_C SME AVX512BW - - SIMD LASX I422ToARGBRow_C SME AVX512BW RVV HVX SIMD LASX I422ToRGB24Row_C SME AVX512VBMI RVV - RELAXED LASX I422ToRGB565Row_C SME AVX512BW - - SIMD LASX I422ToRGBARow_C SME AVX512BW RVV HVX - LASX I422ToUYVYRow_C NEON AVX2 - - - LASX I422ToYUY2Row_C NEON AVX2 - - - LASX I444AlphaToARGBRow_C SME AVX512BW RVV HVX - - I444ToARGBRow_C SME AVX512BW RVV HVX - LASX I444ToRGB24Row_C SME AVX512VBMI RVV - - - InterpolateRow_16_C SME AVX512BW - HVX SIMD - InterpolateRow_C SME AVX512BW RVV HVX SIMD LASX J400ToARGBRow_C NEON AVX512BW RVV HVX RELAXED LASX MergeAR64Row_C NEON AVX2 - - - - MergeARGB16To8Row_C NEON AVX2 - - - - MergeARGBRow_C NEON AVX2 RVV HVX - - MergeRGBRow_C NEON AVX512VBMI RVV - - - MergeUVRow_16_C SME AVX512BW - HVX - - MergeUVRow_C SME AVX512BW RVV HVX SIMD LASX MergeXR30Row_C NEON AVX512BW - - - - MergeXR64Row_C NEON AVX2 - - - - MergeXRGB16To8Row_C NEON AVX2 - - - - MergeXRGBRow_C NEON AVX2 RVV HVX - - MirrorRow_16_C NEON AVX512BW RVV HVX RELAXED LASX MirrorRow_C NEON AVX512BW - HVX RELAXED LASX MirrorSplitUVRow_C NEON AVX512BW - HVX RELAXED LASX MirrorUVRow_C NEON AVX512BW RVV HVX RELAXED LASX MultiplyRow_16_C SME AVX512BW RVV HVX SIMD - NV12ToARGBRow_C SME AVX512BW RVV HVX SIMD LASX NV12ToRGB24Row_C SME AVX512VBMI RVV - - - NV12ToRGB565Row_C NEON AVX2 - - SIMD LASX NV21ToARGBRow_C SME AVX512BW RVV HVX SIMD LASX NV21ToRGB24Row_C SME AVX512VBMI RVV - - - NV21ToYUV24Row_C NEON AVX512VBMI - - - - P210ToAR30Row_C SME AVX512BW - - - - P210ToARGBRow_C SME AVX512BW - - - - P410ToAR30Row_C SME AVX512BW - - - - P410ToARGBRow_C SME AVX512BW - - - - RAWToARGBRow_C SVE2 AVX512BW RVV - RELAXED LASX RAWToRGB24Row_C SVE2 AVX512VBMI RVV - RELAXED LASX RAWToRGBARow_C SVE2 AVX512VBMI RVV - - - RGB24MirrorRow_C NEON AVX512VBMI - - RELAXED - RGB24ToARGBRow_C SVE2 AVX512BW RVV - RELAXED LASX RGB565ToARGBRow_C NEON AVX2 - - SIMD LASX RGB565ToUVMatrixRow_C NEON AVX2 - - SIMD - RGB565ToUVRow_C * NEON AVX2 - - - LASX RGB565ToYMatrixRow_C NEON AVX2 - - SIMD - RGB565ToYRow_C * NEON AVX2 - - - LASX RGBAToARGBRow_C NEON AVX512BW RVV HVX RELAXED LASX RGBColorTableRow_C NEON AVX2 - - - - RGBToUV444MatrixRow_C NEON AVX512BW RVV - RELAXED - RGBToUVMatrixRow_C NEON AVX512BW RVV - RELAXED - RGBToYMatrixRow_C DOTPROD AVX512BW RVV - RELAXED LASX ScaleMaxSamples_C * NEON AVX2 - - - - ScaleSamples_C * NEON AVX2 - - - - ScaleSumSamples_C * NEON AVX2 - - - - SetRow_C NEON AVX2 - HVX - LASX SobelRow_C NEON AVX2 - - - LASX SobelToPlaneRow_C NEON AVX2 - - - LASX SobelXRow_C NEON AVX2 - - - - SobelXYRow_C NEON AVX2 - - - LASX SobelYRow_C NEON AVX2 - - - - SplitARGBRow_C NEON AVX2 RVV HVX - - SplitRGBRow_C NEON AVX2 RVV - - - SplitUVRow_16_C NEON AVX2 - HVX - - SplitUVRow_C SVE2 AVX512BW RVV HVX - LASX SplitXRGBRow_C NEON AVX2 RVV HVX - - SwapUVRow_C NEON AVX512BW RVV HVX SIMD - UYVYToARGBRow_C SME AVX512BW - - - LASX UYVYToUV422Row_C NEON AVX2 - HVX - LASX UYVYToUVRow_C SVE2 AVX512BW - HVX - LASX UYVYToYRow_C SVE2 AVX512BW - HVX - LASX UnpackMT2T_C NEON AVX2 - - - - YUY2ToARGBRow_C SME AVX512BW - - - LASX YUY2ToNVUVRow_C NEON AVX2 - - - - YUY2ToUV422Row_C NEON AVX2 - HVX - LASX YUY2ToUVRow_C SVE2 AVX512BW - HVX - LASX YUY2ToYRow_C SVE2 AVX512BW - HVX - LASX ------------------------------------------------------------------------ SIMD (174 convert) 171 98% 174 100% 65 37% 64 37% 57 33% 76 44% ------------------------------------------------------------------------ Rotate SIMD ISA per *_C (* = uncalled in library): ------------------------------------------------------------------------ C Function arm x86 riscv hexagon wasm loong ------------------------------------------------------------------------ TransposeUVWx16_C - AVX512BW - - SIMD LASX TransposeUVWx8_C NEON AVX512BW - HVX SIMD LASX TransposeWx16_C NEON AVX512BW - - SIMD LASX TransposeWx4_32_C * SVE2 AVX512BW RVV HVX SIMD LASX TransposeWx8_16_C SVE2 AVX512BW RVV HVX SIMD LASX TransposeWx8_C NEON AVX512BW - HVX SIMD LASX TransposeWxH_32_C NEON AVX2 RVV HVX - - ------------------------------------------------------------------------ SIMD (7 rotate) 6 86% 7 100% 3 43% 5 71% 6 86% 6 86% ------------------------------------------------------------------------ Compare SIMD ISA per *_C: ------------------------------------------------------------------------ C Function arm x86 riscv hexagon wasm loong ------------------------------------------------------------------------ ARGBDetectRow_C - AVX2 - - - - HammingDistance_C DOTPROD AVX512BW - HVX SIMD - HashDjb2_C NEON AVX512BW - HVX SIMD - Ssim4x4Row_C SME AVX512BW RVV HVX SIMD LASX SsimWindowRow_C SME AVX512BW RVV - - - SumSquareError_C DOTPROD AVX512VNNI - HVX SIMD - ------------------------------------------------------------------------ SIMD (6 compare) 5 83% 6 100% 2 33% 4 67% 4 67% 1 17% ------------------------------------------------------------------------ Scale SIMD ISA per *_C: ------------------------------------------------------------------------ C Function arm x86 riscv hexagon wasm loong ------------------------------------------------------------------------ FixedDiv1_C - X86 - - - - FixedDiv_C - X86 - - - - ScaleARGBCols64_C - - - - - - ScaleARGBColsUp2_C - AVX2 - - SIMD - ScaleARGBCols_C NEON AVX2 - - SIMD LSX ScaleARGBFilterCols64_C - - - - - - ScaleARGBFilterCols_C NEON AVX2 RVV - SIMD LSX ScaleARGBRowDown2Box_C SME AVX2 RVV HVX SIMD LSX ScaleARGBRowDown2Linear_C SME AVX2 RVV HVX SIMD LSX ScaleARGBRowDown2_C SME AVX2 RVV HVX SIMD LSX ScaleARGBRowDownEvenBox_C NEON AVX2 RVV - SIMD LSX ScaleARGBRowDownEven_C NEON AVX2 - - SIMD LSX ScaleAddCols0_C - - - - - - ScaleAddCols1_16_C - - - - - - ScaleAddCols1_C - AVX512BW - - - - ScaleAddCols2_16_C - - - - - - ScaleAddCols2_C - AVX512BW - - - - ScaleAddRow_16_C - - - - - - ScaleAddRow_C NEON AVX512BW RVV HVX SIMD LSX ScaleColsUp2_16_C - - - - - - ScaleColsUp2_C - AVX2 - - - - ScaleCols_16_C - - - - - - ScaleCols_C - - - - - - ScaleFilterCols64_16_C - - - - - - ScaleFilterCols64_C - - - - - - ScaleFilterCols_16_C - - - - - - ScaleFilterCols_C NEON AVX2 - - SIMD LSX ScaleRowDown2Box_16_C SME AVX2 - HVX - - ScaleRowDown2Box_C SME AVX512BW RVV HVX SIMD LSX ScaleRowDown2Box_Odd_C - AVX512BW - HVX - - ScaleRowDown2Linear_16_C SME AVX2 - HVX - - ScaleRowDown2Linear_C SME AVX512BW RVV HVX SIMD LSX ScaleRowDown2_16_C SME AVX2 - HVX - - ScaleRowDown2_C SME AVX512BW RVV HVX SIMD LSX ScaleRowDown34_0_Box_16_C - - - - - - ScaleRowDown34_0_Box_C NEON AVX2 RVV - RELAXED LSX ScaleRowDown34_16_C - - - - - - ScaleRowDown34_1_Box_16_C - - - - - - ScaleRowDown34_1_Box_C NEON AVX2 RVV - RELAXED LSX ScaleRowDown34_C NEON AVX2 RVV - RELAXED LSX ScaleRowDown38_16_C - - - - - - ScaleRowDown38_2_Box_16_C - - - - - - ScaleRowDown38_2_Box_C NEON AVX2 RVV - RELAXED LSX ScaleRowDown38_3_Box_16_C - - - - - - ScaleRowDown38_3_Box_C NEON AVX2 RVV - RELAXED LSX ScaleRowDown38_C NEON AVX2 RVV - RELAXED LSX ScaleRowDown4Box_16_C - - - - - - ScaleRowDown4Box_C NEON AVX2 RVV HVX SIMD LSX ScaleRowDown4_16_C - - - - - - ScaleRowDown4_C NEON AVX2 RVV HVX SIMD LSX ScaleRowUp2_Bilinear_16_Any_C NEON AVX2 - - - - ScaleRowUp2_Bilinear_16_C NEON AVX2 - - - - ScaleRowUp2_Bilinear_Any_C NEON AVX2 RVV - - - ScaleRowUp2_Bilinear_C NEON AVX2 RVV - - - ScaleRowUp2_Linear_16_Any_C NEON AVX2 - - - - ScaleRowUp2_Linear_16_C NEON AVX2 - - - - ScaleRowUp2_Linear_Any_C NEON AVX2 RVV - - - ScaleRowUp2_Linear_C NEON AVX2 RVV - - - ScaleUVAddCols1_16_C - - - - - - ScaleUVAddCols1_C - - - - - - ScaleUVAddCols2_16_C - - - - - - ScaleUVAddCols2_C - - - - - - ScaleUVAddCols3_16_C - - - - - - ScaleUVAddCols3_C SME AVX512VBMI RVV HVX SIMD LSX ScaleUVCols64_16_C - - - - - - ScaleUVCols64_C - - - - - - ScaleUVColsUp2_16_C - - - - - - ScaleUVColsUp2_C - - - - - - ScaleUVCols_16_C - - - - - - ScaleUVCols_C - - - - - - ScaleUVFilterCols64_16_C - - - - - - ScaleUVFilterCols64_C - - - - - - ScaleUVFilterCols_16_C - - - - - - ScaleUVFilterCols_C - - - - - - ScaleUVRowDown2Box_16_C - - - - - - ScaleUVRowDown2Box_C SME AVX2 RVV HVX - - ScaleUVRowDown2Linear_16_C - - - - - - ScaleUVRowDown2Linear_C SME AVX2 RVV HVX - - ScaleUVRowDown2_16_C - - - - - - ScaleUVRowDown2_C SME AVX2 RVV HVX - - ScaleUVRowDownEvenBox_16_C - - - - - - ScaleUVRowDownEvenBox_C - - - - - - ScaleUVRowDownEven_16_C - - - - - - ScaleUVRowDownEven_C NEON AVX2 RVV - - - ScaleUVRowUp2_Bilinear_16_Any_C NEON AVX2 - - - - ScaleUVRowUp2_Bilinear_16_C NEON AVX2 - - - - ScaleUVRowUp2_Bilinear_Any_C NEON AVX2 RVV - - - ScaleUVRowUp2_Bilinear_C NEON AVX2 RVV - - - ScaleUVRowUp2_Linear_16_Any_C NEON AVX2 - - - - ScaleUVRowUp2_Linear_16_C NEON AVX2 - - - - ScaleUVRowUp2_Linear_Any_C NEON AVX2 RVV - - - ScaleUVRowUp2_Linear_C NEON AVX2 RVV - - - ------------------------------------------------------------------------ SIMD (92 scale) 44 48% 51 55% 30 33% 17 18% 22 24% 21 23% ------------------------------------------------------------------------ Test: libyuv_unittest (Release, row_win, x86_32, Sanitizers, GCC 15) Bug: 388428508, 578615529, 579659306 Change-Id: Id4eaf5f7fd85c75a60984a3b1559bfdb07b21afc Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8686033 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-10-30 14:51:16 -07:00
Frank Barchard	8d440c181c	[libyuv] Upgrade x86 SIMD to AVX2/AVX-512 with 32-bit & row_win parity ... Upgrade all remaining SSE2 and SSSE3 max-ISA kernels across row, scale, rotate, and compare to AVX2 while removing superseded pre-AVX2 versions. Implement AVX2 for ARGBDetectRow, HalfRow, HalfRow_16, HalfWidthRow_16To8_Odd, AYUVTo{Y,UV,VU}Row, AR30To{ARGB,ABGR,AB30}Row, and TransposeWx8_16, and port all remaining Arm NEON (or higher) kernels to AVX2 (TransposeWxH_32, ScaleRowDown2*_16, ScaleUVRowDown2*, ScaleUVRowDownEven, ScaleColsUp2, GaussCol*, Scale*Samples, and UnpackMT2T). Port 32-bit x86 (-m32 -fPIC) inline assembly across row_gcc.cc, rotate_gcc.cc, and compare_gcc.cc (including 5-GPR PIC support for I422ToAR30Row and I444/I422/I210/I410AlphaToARGBRow) so x86_32 matches x64 SIMD coverage across all 187 supported row/rotate/compare kernels. Handle 1..7 remainder elements directly in inline assembly without calling *_C in ARGBDetectRow_AVX2, ARGBSepiaRow_AVX2, ARGBColorMatrixRow_AVX2, ARGBQuantizeRow_AVX2, ARGBShadeRow_AVX2, ARGBAffineRow_AVX2, GaussRow*_AVX2, GaussCol*_AVX2, and *ColorTableRow_AVX2. Fix unsigned 16-bit packing in ScaleRowUp2_{Linear,Bilinear}_16_SSE2. Add 64-bit AVX-512 (AVX512BW, AVX512VBMI, and AVX512VNNI, including SumSquareError_AVX512VNNI) implementations in *_gcc.cc and *_win.cc wherever Arm64 exceeds NEON, and handle arbitrary tail widths via opmasks in Mirror*_AVX512BW without _Any_ wrappers. Bring row_win.cc, scale_win.cc, rotate_win.cc, and compare_win.cc C++ intrinsics (and 64-bit VISUALC_HAS_AVX2) to 100% parity with *_gcc.cc for MSVC and MemorySanitizer. Wire YUY2ToUVRow_HVX and UYVYToUVRow_HVX in YUY2ToI420 and UYVYToI420, implement all 13 missing ScaleUV*_16_C kernels so UVScale_16 supports downscaling and general upscaling, make Ssim8x8, TransposeWxH, TransposeUVWxH, and TransposeWxH_16 static internal helpers, clean up HAS_* guards, enable direct row unittests so all 279 *_C functions have active test coverage, and add Intel AMX-TF32 (kCpuHasAMXTF32, CPUID leaf 7 subleaf 1 EAX bit 4, LIBYUV_DISABLE_AMXTF32) detection. Mark 16-bit RGB formats (RGB565, ARGB1555, and ARGB4444) and legacy fixed-matrix 16-bit RGB-to-YUV row functions as deprecated in headers and GEMINI.md, using two-step ARGB wrapper row functions in row_common.cc. AMD Zen 6 Benchmark (1280 width, 10000 iterations): ---------------------------------------------------------------- Function Before (ms) After (ms) Improvement ---------------------------------------------------------------- ARGB1555ToARGBRow 0.83 0.49 1.69x ARGB1555ToUVRow 16.00 3.21 4.99x ARGB1555ToYRow 7.82 1.38 5.66x ARGB4444ToUVRow 15.04 2.58 5.83x ARGB4444ToYRow 7.42 1.07 6.94x ARGBAffineRow 3.51 2.75 1.28x ARGBColorMatrixRow 4.09 1.13 3.63x ARGBColorTableRow 11.21 8.93 1.26x ARGBLumaColorTableRow 10.01 10.21 0.98x ARGBQuantizeRow 2.23 1.21 1.84x ARGBSepiaRow 3.29 0.81 4.04x ARGBSetRow 0.21 0.21 1.02x ARGBShadeRow 1.66 0.87 1.91x ARGBToRGB565DitherRow 1.22 0.50 2.43x ARGBToRGB565Row 1.09 0.44 2.47x ByteToFloatRow 0.88 0.41 2.14x Convert8To8Row 0.21 0.11 1.88x CopyRow_16 0.11 0.11 0.97x DetileRow 0.21 0.12 1.75x DetileRow_16 0.23 0.14 1.67x DetileSplitUVRow 0.29 0.21 1.38x DivideRow_16 0.14 0.11 1.30x GaussRow 3.29 1.27 2.59x GaussRow_F32 1.93 0.89 2.17x HalfFloatRow 1.16 0.74 1.57x I210AlphaToARGBRow 1.83 0.92 2.00x I210ToAR30Row 2.21 0.96 2.30x I210ToARGBRow 1.75 0.87 2.00x I212ToAR30Row 2.21 0.96 2.31x I212ToARGBRow 1.75 0.87 2.01x I410AlphaToARGBRow 1.73 0.86 2.00x I410ToAR30Row 2.04 0.89 2.28x I410ToARGBRow 1.64 0.82 2.00x I422ToARGB1555Row 2.94 2.27 1.30x I422ToARGB4444Row 2.94 2.27 1.29x I422ToRGB565Row 2.63 1.27 2.07x I422ToRGBARow 1.51 0.82 1.83x I444AlphaToARGBRow 1.52 0.87 1.73x I444ToARGBRow 1.48 0.81 1.82x I444ToRGB24Row 2.13 0.72 2.98x MergeUVRow_16 0.87 0.32 2.74x P210ToAR30Row 1.77 0.70 2.54x P210ToARGBRow 1.38 0.62 2.24x P410ToAR30Row 1.94 0.85 2.28x P410ToARGBRow 1.45 0.77 1.90x RGB565ToUVRow 15.84 2.88 5.49x RGB565ToYRow 7.82 1.22 6.42x RGBColorTableRow 8.42 6.95 1.21x SetRow 0.11 0.11 1.04x SobelRow 0.80 0.43 1.87x SobelToPlaneRow 0.27 0.12 2.25x SobelXRow 1.58 0.85 1.85x SobelXYRow 0.80 0.55 1.44x SobelYRow 1.58 0.85 1.85x UYVYToARGBRow 1.28 0.65 1.96x UYVYToUVRow 0.38 0.21 1.78x UYVYToYRow 0.21 0.11 1.87x YUY2ToARGBRow 1.28 0.65 1.96x YUY2ToUVRow 0.40 0.21 1.87x YUY2ToYRow 0.21 0.11 1.88x ---------------------------------------------------------------- Row/Rotate/Compare SIMD ISA per *_C (* = uncalled in library): ------------------------------------------------------------------------ C Function arm64 x86_32 x64 row_win ------------------------------------------------------------------------ AB64ToARGBRow_C NEON AVX2 AVX2 AVX2 ABGRToAR30Row_C NEON AVX2 AVX512BW AVX512BW AR30ToAB30Row_C NEON AVX2 AVX512BW AVX512BW AR30ToABGRRow_C NEON AVX2 AVX512BW AVX512BW AR30ToARGBRow_C NEON AVX2 AVX512BW AVX512BW AR64ShuffleRow_C NEON AVX2 AVX512BW AVX512BW AR64ToAB64Row_C NEON AVX2 AVX512BW AVX512BW AR64ToARGBRow_C NEON AVX2 AVX2 AVX2 ARGB1555ToARGBRow_C SVE2 AVX2 AVX512BW AVX512BW ARGB1555ToUVMatrixRow_C NEON AVX2 AVX2 AVX2 ARGB1555ToUVRow_C * NEON AVX2 AVX2 AVX2 ARGB1555ToYMatrixRow_C NEON AVX2 AVX2 AVX2 ARGB1555ToYRow_C * NEON AVX2 AVX2 AVX2 ARGB4444ToARGBRow_C NEON AVX2 AVX2 AVX2 ARGB4444ToUVMatrixRow_C NEON AVX2 AVX2 AVX2 ARGB4444ToUVRow_C * NEON AVX2 AVX2 AVX2 ARGB4444ToYMatrixRow_C NEON AVX2 AVX2 AVX2 ARGB4444ToYRow_C * NEON AVX2 AVX2 AVX2 ARGBAddRow_C NEON AVX2 AVX2 AVX2 ARGBAffineRow_C * NEON AVX2 AVX2 AVX2 ARGBAlphaBlendRow_C SME AVX2 AVX512BW AVX512BW ARGBAttenuateRow_C NEON AVX2 AVX512BW AVX512BW ARGBBlendRow_C NEON AVX2 AVX512BW AVX512BW ARGBColorMatrixRow_C I8MM AVX2 AVX512VNNI AVX512VNNI ARGBColorTableRow_C NEON AVX2 AVX2 AVX2 ARGBCopyAlphaRow_C NEON AVX2 AVX2 AVX2 ARGBCopyYToAlphaRow_C NEON AVX2 AVX512BW AVX512BW ARGBDetectRow_C - AVX2 AVX2 AVX2 ARGBExtractAlphaRow_C NEON AVX2 AVX512BW AVX512BW ARGBGrayRow_C DOTPROD AVX2 AVX512VNNI AVX512VNNI ARGBLumaColorTableRow_C NEON AVX2 AVX2 AVX2 ARGBMirrorRow_C NEON AVX2 AVX512BW AVX512BW ARGBMultiplyRow_C SME AVX2 AVX512BW AVX512BW ARGBOrRow_C SME AVX2 AVX512BW AVX512BW ARGBPolynomialRow_C NEON AVX2 AVX2 AVX2 ARGBQuantizeRow_C NEON AVX2 AVX2 AVX2 ARGBSepiaRow_C DOTPROD AVX2 AVX512VNNI AVX512VNNI ARGBSetRow_C NEON AVX2 AVX2 AVX2 ARGBShadeRow_C NEON AVX2 AVX2 AVX2 ARGBShuffleRow_C NEON AVX2 AVX512BW AVX512BW ARGBSubtractRow_C NEON AVX2 AVX2 AVX2 ARGBToAB64Row_C NEON AVX2 AVX2 AVX2 ARGBToABGRRow_C NEON AVX2 AVX512BW AVX512BW ARGBToAR30Row_C NEON AVX2 AVX512BW AVX512BW ARGBToAR64Row_C NEON AVX2 AVX2 AVX2 ARGBToARGB1555Row_C NEON AVX2 AVX2 AVX2 ARGBToARGB4444Row_C NEON AVX2 AVX2 AVX2 ARGBToBGRARow_C NEON AVX2 AVX512BW AVX512BW ARGBToRAWRow_C SVE2 AVX2 AVX512BW AVX512BW ARGBToRGB24Row_C SVE2 AVX2 AVX512BW AVX512BW ARGBToRGB565DitherRow_C SVE2 AVX2 AVX512BW AVX512BW ARGBToRGB565Row_C SVE2 AVX2 AVX512BW AVX512BW ARGBToRGBARow_C NEON AVX2 AVX512BW AVX512BW ARGBToUV444MatrixRow_C I8MM AVX2 AVX512VNNI AVX512VNNI ARGBToUVMatrixRow_C SME AVX2 AVX512VNNI AVX512VNNI ARGBToYMatrixRow_C SME AVX2 AVX512VNNI AVX512VNNI ARGBUnattenuateRow_C NEON AVX2 AVX2 AVX2 AYUVToUVRow_C SVE2 AVX2 AVX512VBMI AVX512VBMI AYUVToVURow_C SVE2 AVX2 AVX512VBMI AVX512VBMI AYUVToYRow_C NEON AVX2 AVX512VBMI AVX512VBMI BlendPlaneRow_C NEON AVX2 AVX512BW AVX512BW ByteToFloatRow_C NEON AVX2 AVX2 AVX2 ComputeCumulativeSumRow_C NEON AVX2 AVX2 AVX2 Convert16To8Row_C SME AVX2 AVX512BW AVX512BW Convert8To16Row_C SME AVX2 AVX512BW AVX512BW Convert8To8Row_C SME AVX2 AVX512BW AVX512BW CopyRow_16_C NEON AVX2 AVX2 AVX2 CopyRow_C SME AVX AVX512BW AVX512BW CumulativeSumToAverageRow_C NEON AVX2 AVX2 AVX2 DetileRow_16_C NEON AVX2 AVX2 AVX2 DetileRow_C NEON AVX2 AVX2 AVX2 DetileSplitUVRow_C NEON AVX2 AVX2 AVX2 DetileToYUY2_C NEON AVX2 AVX2 AVX2 DivideRow_16_C SVE2 AVX2 AVX512BW AVX512BW GaussCol_C * NEON AVX2 AVX2 AVX2 GaussCol_F32_C NEON AVX2 AVX2 AVX2 GaussRow_C * NEON AVX2 AVX2 AVX2 GaussRow_F32_C NEON AVX2 AVX2 AVX2 HalfFloatRow_C SVE2 AVX2 AVX512BW AVX512BW HalfMergeUVRow_C SVE2 AVX2 AVX512BW AVX512BW HalfRow_16To8_C SME AVX2 AVX512BW AVX512BW HalfRow_16_C - AVX2 AVX2 AVX2 HalfRow_C - AVX2 AVX2 AVX2 HalfWidthRow_16To8_C SME AVX2 AVX512BW AVX512BW HalfWidthRow_16To8_Odd_C - AVX2 AVX2 AVX2 HammingDistance_C DOTPROD AVX2 AVX512BW AVX512BW HashDjb2_C NEON AVX2 AVX512BW AVX512BW I210AlphaToARGBRow_C SME AVX2 AVX512BW AVX512BW I210ToAR30Row_C SME AVX2 AVX512BW AVX512BW I210ToARGBRow_C SME AVX2 AVX512BW AVX512BW I212ToAR30Row_C SME AVX2 AVX512BW AVX512BW I212ToARGBRow_C SME AVX2 AVX512BW AVX512BW I400ToARGBRow_C SME AVX2 AVX512BW AVX512BW I410AlphaToARGBRow_C SME AVX2 AVX512BW AVX512BW I410ToAR30Row_C SME AVX2 AVX512BW AVX512BW I410ToARGBRow_C SME AVX2 AVX512BW AVX512BW I422AlphaToARGBRow_C SME AVX2 AVX512BW AVX512BW I422ToAR30Row_C SME AVX2 AVX512BW AVX512BW I422ToARGB1555Row_C SME AVX2 AVX512BW AVX512BW I422ToARGB4444Row_C SME AVX2 AVX512BW AVX512BW I422ToARGBRow_C SME AVX2 AVX512BW AVX512BW I422ToRGB24Row_C SME AVX2 AVX512VBMI AVX512VBMI I422ToRGB565Row_C SME AVX2 AVX512BW AVX512BW I422ToRGBARow_C SME AVX2 AVX512BW AVX512BW I422ToUYVYRow_C NEON AVX2 AVX2 AVX2 I422ToYUY2Row_C NEON AVX2 AVX2 AVX2 I444AlphaToARGBRow_C SME AVX2 AVX512BW AVX512BW I444ToARGBRow_C SME AVX2 AVX512BW AVX512BW I444ToRGB24Row_C SME AVX2 AVX512VBMI AVX512VBMI InterpolateRow_16_C SME AVX2 AVX512BW AVX512BW InterpolateRow_C SME AVX2 AVX512BW AVX512BW J400ToARGBRow_C NEON AVX2 AVX512BW AVX512BW MergeAR64Row_C NEON AVX2 AVX2 AVX2 MergeARGB16To8Row_C NEON AVX2 AVX2 AVX2 MergeARGBRow_C NEON AVX2 AVX2 AVX2 MergeRGBRow_C NEON AVX2 AVX512VBMI AVX512VBMI MergeUVRow_16_C SME AVX2 AVX512BW AVX512BW MergeUVRow_C SME AVX2 AVX512BW AVX512BW MergeXR30Row_C NEON AVX2 AVX512BW AVX512BW MergeXR64Row_C NEON AVX2 AVX2 AVX2 MergeXRGB16To8Row_C NEON AVX2 AVX2 AVX2 MergeXRGBRow_C NEON AVX2 AVX2 AVX2 MirrorRow_16_C NEON AVX2 AVX512BW AVX512BW MirrorRow_C NEON AVX2 AVX512BW AVX512BW MirrorSplitUVRow_C NEON AVX2 AVX512BW AVX512BW MirrorUVRow_C NEON AVX2 AVX512BW AVX512BW MultiplyRow_16_C SME AVX2 AVX512BW AVX512BW NV12ToARGBRow_C SME AVX2 AVX512BW AVX512BW NV12ToRGB24Row_C SME AVX2 AVX512VBMI AVX512VBMI NV12ToRGB565Row_C NEON AVX2 AVX2 AVX2 NV21ToARGBRow_C SME AVX2 AVX512BW AVX512BW NV21ToRGB24Row_C SME AVX2 AVX512VBMI AVX512VBMI NV21ToYUV24Row_C NEON AVX2 AVX2 AVX2 P210ToAR30Row_C SME AVX2 AVX512BW AVX512BW P210ToARGBRow_C SME AVX2 AVX512BW AVX512BW P410ToAR30Row_C SME AVX2 AVX512BW AVX512BW P410ToARGBRow_C SME AVX2 AVX512BW AVX512BW RAWToARGBRow_C SVE2 AVX2 AVX512BW AVX512BW RAWToRGB24Row_C SVE2 AVX2 AVX512VBMI AVX512VBMI RAWToRGBARow_C SVE2 AVX2 AVX512VBMI AVX512VBMI RGB24MirrorRow_C NEON AVX2 AVX512VBMI AVX512VBMI RGB24ToARGBRow_C SVE2 AVX2 AVX512BW AVX512BW RGB565ToARGBRow_C NEON AVX2 AVX2 AVX2 RGB565ToUVMatrixRow_C NEON AVX2 AVX2 AVX2 RGB565ToUVRow_C * NEON AVX2 AVX2 AVX2 RGB565ToYMatrixRow_C NEON AVX2 AVX2 AVX2 RGB565ToYRow_C * NEON AVX2 AVX2 AVX2 RGBAToARGBRow_C NEON AVX2 AVX512BW AVX512BW RGBColorTableRow_C NEON AVX2 AVX2 AVX2 RGBToUV444MatrixRow_C NEON AVX2 AVX512BW AVX512BW RGBToUVMatrixRow_C NEON AVX2 AVX512BW AVX512BW RGBToYMatrixRow_C DOTPROD AVX2 AVX512BW AVX512BW ScaleMaxSamples_C * NEON AVX2 AVX2 AVX2 ScaleSamples_C * NEON AVX2 AVX2 AVX2 ScaleSumSamples_C * NEON AVX2 AVX2 AVX2 SetRow_C NEON AVX2 AVX2 AVX2 SobelRow_C NEON AVX2 AVX2 AVX2 SobelToPlaneRow_C NEON AVX2 AVX2 AVX2 SobelXRow_C NEON AVX2 AVX2 AVX2 SobelXYRow_C NEON AVX2 AVX2 AVX2 SobelYRow_C NEON AVX2 AVX2 AVX2 SplitARGBRow_C NEON AVX2 AVX2 AVX2 SplitRGBRow_C NEON AVX2 AVX2 AVX2 SplitUVRow_16_C NEON AVX2 AVX2 AVX2 SplitUVRow_C SVE2 AVX2 AVX512BW AVX512BW SplitXRGBRow_C NEON AVX2 AVX2 AVX2 Ssim4x4Row_C SME AVX2 AVX512BW AVX512BW SsimWindowRow_C SME AVX2 AVX512BW AVX512BW SumSquareError_C DOTPROD AVX2 AVX512VNNI AVX512VNNI SwapUVRow_C NEON AVX2 AVX512BW AVX512BW TransposeUVWx16_C - AVX2 AVX512BW AVX512BW TransposeUVWx8_C NEON AVX2 AVX2 AVX2 TransposeWx16_C NEON AVX2 AVX512BW AVX512BW TransposeWx4_32_C * SVE2 AVX2 AVX512BW AVX512BW TransposeWx8_16_C SVE2 AVX2 AVX512BW AVX512BW TransposeWx8_C - AVX2 AVX2 AVX2 TransposeWxH_32_C NEON AVX2 AVX2 AVX2 UYVYToARGBRow_C SME AVX2 AVX512BW AVX512BW UYVYToUV422Row_C NEON AVX2 AVX2 AVX2 UYVYToUVRow_C SVE2 AVX2 AVX512BW AVX512BW UYVYToYRow_C SVE2 AVX2 AVX512BW AVX512BW UnpackMT2T_C NEON AVX2 AVX2 AVX2 YUY2ToARGBRow_C SME AVX2 AVX512BW AVX512BW YUY2ToNVUVRow_C NEON AVX2 AVX2 AVX2 YUY2ToUV422Row_C NEON AVX2 AVX2 AVX2 YUY2ToUVRow_C SVE2 AVX2 AVX512BW AVX512BW YUY2ToYRow_C SVE2 AVX2 AVX512BW AVX512BW ------------------------------------------------------------------------ SIMD (187 row/rot/cmp) 181 97% 187 100% 187 100% 187 100% ------------------------------------------------------------------------ Scale SIMD ISA supported per *_C function: ------------------------------------------------------------------------ C Function arm64 x86_32 x64 row_win ------------------------------------------------------------------------ FixedDiv1_C - X86 X86 X86 FixedDiv_C - X86 X86 X86 ScaleARGBCols64_C - - - - ScaleARGBColsUp2_C - AVX2 AVX2 AVX2 ScaleARGBCols_C NEON AVX2 AVX2 AVX2 ScaleARGBFilterCols64_C - - - - ScaleARGBFilterCols_C NEON AVX2 AVX2 AVX2 ScaleARGBRowDown2Box_C SME AVX2 AVX2 AVX2 ScaleARGBRowDown2Linear_C SME AVX2 AVX2 AVX2 ScaleARGBRowDown2_C SME AVX2 AVX2 AVX2 ScaleARGBRowDownEvenBox_C NEON AVX2 AVX2 AVX2 ScaleARGBRowDownEven_C NEON AVX2 AVX2 AVX2 ScaleAddCols0_C - - - - ScaleAddCols1_16_C - - - - ScaleAddCols1_C - - AVX512BW AVX512BW ScaleAddCols2_16_C - - - - ScaleAddCols2_C - - AVX512BW AVX512BW ScaleAddRow_16_C - - - - ScaleAddRow_C NEON AVX2 AVX512BW AVX512BW ScaleColsUp2_16_C - - - - ScaleColsUp2_C - AVX2 AVX2 AVX2 ScaleCols_16_C - - - - ScaleCols_C - - - - ScaleFilterCols64_16_C - - - - ScaleFilterCols64_C - - - - ScaleFilterCols_16_C - - - - ScaleFilterCols_C NEON AVX2 AVX2 AVX2 ScaleRowDown2Box_16_C SME AVX2 AVX2 AVX2 ScaleRowDown2Box_C SME AVX2 AVX512BW AVX512BW ScaleRowDown2Box_Odd_C - - AVX512BW AVX512BW ScaleRowDown2Linear_16_C SME AVX2 AVX2 AVX2 ScaleRowDown2Linear_C SME AVX2 AVX512BW AVX512BW ScaleRowDown2_16_C SME AVX2 AVX2 AVX2 ScaleRowDown2_C SME AVX2 AVX512BW AVX512BW ScaleRowDown34_0_Box_16_C - - - - ScaleRowDown34_0_Box_C NEON AVX2 AVX2 AVX2 ScaleRowDown34_16_C - - - - ScaleRowDown34_1_Box_16_C - - - - ScaleRowDown34_1_Box_C NEON AVX2 AVX2 AVX2 ScaleRowDown34_C NEON AVX2 AVX2 AVX2 ScaleRowDown38_16_C - - - - ScaleRowDown38_2_Box_16_C - - - - ScaleRowDown38_2_Box_C NEON AVX2 AVX2 AVX2 ScaleRowDown38_3_Box_16_C - - - - ScaleRowDown38_3_Box_C NEON AVX2 AVX2 AVX2 ScaleRowDown38_C NEON AVX2 AVX2 AVX2 ScaleRowDown4Box_16_C - - - - ScaleRowDown4Box_C NEON AVX2 AVX2 AVX2 ScaleRowDown4_16_C - - - - ScaleRowDown4_C NEON AVX2 AVX2 AVX2 ScaleRowUp2_Bilinear_16_Any_C NEON AVX2 AVX2 AVX2 ScaleRowUp2_Bilinear_16_C NEON AVX2 AVX2 AVX2 ScaleRowUp2_Bilinear_Any_C NEON AVX2 AVX2 AVX2 ScaleRowUp2_Bilinear_C NEON AVX2 AVX2 AVX2 ScaleRowUp2_Linear_16_Any_C NEON AVX2 AVX2 AVX2 ScaleRowUp2_Linear_16_C NEON AVX2 AVX2 AVX2 ScaleRowUp2_Linear_Any_C NEON AVX2 AVX2 AVX2 ScaleRowUp2_Linear_C NEON AVX2 AVX2 AVX2 ScaleUVAddCols1_16_C - - - - ScaleUVAddCols1_C - - - - ScaleUVAddCols2_16_C - - - - ScaleUVAddCols2_C - - - - ScaleUVAddCols3_16_C - - - - ScaleUVAddCols3_C SME AVX2 AVX512VBMI AVX512VBMI ScaleUVCols64_16_C - - - - ScaleUVCols64_C - - - - ScaleUVColsUp2_16_C - - - - ScaleUVColsUp2_C - - - - ScaleUVCols_16_C - - - - ScaleUVCols_C - - - - ScaleUVFilterCols64_16_C - - - - ScaleUVFilterCols64_C - - - - ScaleUVFilterCols_16_C - - - - ScaleUVFilterCols_C - - - - ScaleUVRowDown2Box_16_C - - - - ScaleUVRowDown2Box_C SME AVX2 AVX2 AVX2 ScaleUVRowDown2Linear_16_C - - - - ScaleUVRowDown2Linear_C SME AVX2 AVX2 AVX2 ScaleUVRowDown2_16_C - - - - ScaleUVRowDown2_C SME AVX2 AVX2 AVX2 ScaleUVRowDownEvenBox_16_C - - - - ScaleUVRowDownEvenBox_C - - - - ScaleUVRowDownEven_16_C - - - - ScaleUVRowDownEven_C NEON AVX2 AVX2 AVX2 ScaleUVRowUp2_Bilinear_16_Any_C NEON AVX2 AVX2 AVX2 ScaleUVRowUp2_Bilinear_16_C NEON AVX2 AVX2 AVX2 ScaleUVRowUp2_Bilinear_Any_C NEON AVX2 AVX2 AVX2 ScaleUVRowUp2_Bilinear_C NEON AVX2 AVX2 AVX2 ScaleUVRowUp2_Linear_16_Any_C NEON AVX2 AVX2 AVX2 ScaleUVRowUp2_Linear_16_C NEON AVX2 AVX2 AVX2 ScaleUVRowUp2_Linear_Any_C NEON AVX2 AVX2 AVX2 ScaleUVRowUp2_Linear_C NEON AVX2 AVX2 AVX2 ------------------------------------------------------------------------ SIMD (92 scale *_C) 44 48% 48 52% 51 55% 51 55% ------------------------------------------------------------------------ Test: libyuv_unittest Bug: none Change-Id: I2b4a832f2a7fe1fe93487b453240cd15c1f5f940 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8676400 Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-10-29 12:46:05 -07:00
Frank Barchard	63e54f7374	[libyuv] Add NEON implementations for all remaining C row functions ... Implement 32-bit and 64-bit Arm NEON versions (in row_neon.cc and row_neon64.cc) for the 12 remaining row functions that previously fell back to C on Arm: - AR30ToAB30Row_NEON (and AR30ToAB30Row_Any_NEON) - AR30ToABGRRow_NEON (and AR30ToABGRRow_Any_NEON) - AR30ToARGBRow_NEON (and AR30ToARGBRow_Any_NEON) - ARGBAffineRow_NEON - ARGBColorTableRow_NEON - ARGBCopyAlphaRow_NEON (and ARGBCopyAlphaRow_Any_NEON) - ARGBLumaColorTableRow_NEON - ARGBPolynomialRow_NEON (and ARGBPolynomialRow_Any_NEON) - ARGBUnattenuateRow_NEON (and ARGBUnattenuateRow_Any_NEON) - CopyRow_16_NEON (and CopyRow_16_Any_NEON) - MirrorRow_16_NEON (and MirrorRow_16_Any_NEON) - RGBColorTableRow_NEON Add HAS_*_NEON macros, prototypes, _Any wrappers, NEON dispatches in convert.cc, convert_argb.cc, and planar_functions.cc, and enable unit tests in planar_test.cc. Arm row function SIMD coverage is now 164/164 (100%). ARM64 (Apple M6) Benchmark (1280 width, 10000 iterations): ---------------------------------------------------------------- Function Before C (ms) After NEON (ms) Improvement ---------------------------------------------------------------- AR30ToAB30Row 1.20 0.70 1.72x AR30ToABGRRow 1.88 1.03 1.83x AR30ToARGBRow 1.88 0.87 2.17x ARGBAffineRow 7.96 2.31 3.44x ARGBColorTableRow 8.17 7.43 1.10x ARGBCopyAlphaRow 1.37 0.53 2.58x ARGBLumaColorTableRow 8.18 5.24 1.56x ARGBPolynomialRow 8.73 7.55 1.16x ARGBUnattenuateRow 6.00 2.05 2.93x CopyRow_16 0.18 0.18 1.00x MirrorRow_16 0.34 0.20 1.73x RGBColorTableRow 6.15 5.58 1.10x ---------------------------------------------------------------- Maximum SIMD ISA supported per C row function: ---------------------------------------------------------------- C Row Function arm32 arm64 x86 riscv ---------------------------------------------------------------- AB64ToARGBRow_C NEON NEON AVX2 RVV ABGRToAR30Row_C - NEON AVX512BW - AR30ToAB30Row_C NEON NEON AVX512BW - AR30ToABGRRow_C NEON NEON AVX512BW - AR30ToARGBRow_C NEON NEON AVX512BW - AR64ShuffleRow_C NEON NEON AVX512BW RVV AR64ToAB64Row_C NEON NEON AVX512BW RVV AR64ToARGBRow_C NEON NEON AVX2 RVV ARGB1555ToARGBRow_C NEON SVE2 AVX2 - ARGB1555ToUVMatrixRow_C NEON NEON AVX2 - ARGB1555ToUVRow_C NEON NEON - - ARGB1555ToYMatrixRow_C NEON NEON AVX2 - ARGB1555ToYRow_C NEON NEON - - ARGB4444ToARGBRow_C NEON NEON AVX2 - ARGB4444ToUVMatrixRow_C NEON NEON AVX2 - ARGB4444ToUVRow_C NEON NEON - - ARGB4444ToYMatrixRow_C NEON NEON AVX2 - ARGB4444ToYRow_C NEON NEON - - ARGBAddRow_C NEON NEON AVX2 - ARGBAffineRow_C NEON NEON SSE2 - ARGBAlphaBlendRow_C NEON SME AVX512BW RVV ARGBAttenuateRow_C NEON NEON AVX512BW RVV ARGBBlendRow_C NEON NEON AVX512BW RVV ARGBColorMatrixRow_C NEON I8MM SSSE3 - ARGBColorTableRow_C NEON NEON X86 - ARGBCopyAlphaRow_C NEON NEON AVX2 - ARGBCopyYToAlphaRow_C NEON NEON AVX512BW RVV ARGBExtractAlphaRow_C NEON NEON AVX512BW RVV ARGBGrayRow_C NEON DOTPROD AVX512VNNI - ARGBLumaColorTableRow_C NEON NEON SSSE3 - ARGBMirrorRow_C NEON NEON AVX512BW RVV ARGBMultiplyRow_C NEON SME AVX512BW RVV ARGBOrRow_C NEON SME AVX512BW RVV ARGBPolynomialRow_C NEON NEON AVX2 - ARGBQuantizeRow_C NEON NEON SSE2 - ARGBSepiaRow_C NEON DOTPROD SSSE3 - ARGBSetRow_C NEON NEON X86 RVV ARGBShadeRow_C NEON NEON SSE2 - ARGBShuffleRow_C NEON NEON AVX512BW RVV ARGBSubtractRow_C NEON NEON AVX2 - ARGBToAB64Row_C NEON NEON AVX2 RVV ARGBToABGRRow_C NEON NEON AVX512BW RVV ARGBToAR30Row_C - NEON AVX512BW - ARGBToAR64Row_C NEON NEON AVX2 RVV ARGBToARGB1555Row_C NEON NEON AVX2 - ARGBToARGB4444Row_C NEON NEON AVX2 - ARGBToBGRARow_C NEON NEON AVX512BW RVV ARGBToRAWRow_C NEON SVE2 AVX512BW RVV ARGBToRGB24Row_C NEON SVE2 AVX512BW RVV ARGBToRGB565DitherRow_C NEON SVE2 AVX2 - ARGBToRGB565Row_C NEON SVE2 AVX2 - ARGBToRGBARow_C NEON NEON AVX512BW RVV ARGBToUV444MatrixRow_C NEON I8MM AVX512VNNI RVV ARGBToUVMatrixRow_C NEON SME AVX512VNNI RVV ARGBToYMatrixRow_C NEON SME AVX512VNNI RVV ARGBUnattenuateRow_C NEON NEON AVX2 - AYUVToUVRow_C NEON SVE2 AVX512VBMI - AYUVToVURow_C NEON SVE2 AVX512VBMI - AYUVToYRow_C NEON NEON AVX512VBMI - BlendPlaneRow_C NEON NEON AVX512BW RVV ByteToFloatRow_C NEON NEON - - ComputeCumulativeSumRow_C - NEON AVX2 - Convert16To8Row_C NEON SME AVX512BW RVV Convert8To16Row_C NEON SME AVX512BW RVV Convert8To8Row_C NEON SME AVX2 - CopyRow_C NEON SME AVX512BW RVV CopyRow_16_C NEON NEON - - CumulativeSumToAverageRow_C - NEON AVX2 - DetileRow_C NEON NEON SSE2 - DetileRow_16_C NEON NEON AVX - DetileSplitUVRow_C NEON NEON SSSE3 - DivideRow_16_C NEON SVE2 AVX2 - GaussRow_C NEON NEON - - GaussRow_F32_C - NEON - - HalfFloatRow_C NEON SVE2 AVX2 - HalfMergeUVRow_C NEON SVE2 AVX512BW - HalfRow_16To8_C NEON SME AVX512BW RVV HalfWidthRow_16To8_C NEON SME AVX512BW RVV I210AlphaToARGBRow_C - SME AVX2 - I210ToAR30Row_C - SME AVX2 - I210ToARGBRow_C - SME AVX2 - I212ToAR30Row_C - SME AVX2 - I212ToARGBRow_C - SME AVX2 - I400ToARGBRow_C NEON SME AVX512BW RVV I410AlphaToARGBRow_C - SME AVX2 - I410ToAR30Row_C - SME AVX2 - I410ToARGBRow_C - SME AVX2 - I422AlphaToARGBRow_C NEON SME AVX512BW RVV I422ToAR30Row_C - SME AVX512BW RVV I422ToARGB1555Row_C NEON SME AVX2 - I422ToARGB4444Row_C NEON SME AVX2 - I422ToARGBRow_C NEON SME AVX512BW RVV I422ToRGB24Row_C NEON SME AVX512VBMI RVV I422ToRGB565Row_C NEON SME AVX2 - I422ToRGBARow_C NEON SME AVX2 RVV I422ToUYVYRow_C NEON NEON AVX2 - I422ToYUY2Row_C NEON NEON AVX2 - I444AlphaToARGBRow_C NEON SME AVX2 RVV I444ToARGBRow_C NEON SME AVX2 RVV I444ToRGB24Row_C NEON SME AVX2 RVV InterpolateRow_C NEON SME AVX512BW RVV InterpolateRow_16_C NEON SME AVX512BW - J400ToARGBRow_C NEON NEON AVX512BW RVV MergeAR64Row_C NEON NEON AVX2 - MergeARGB16To8Row_C NEON NEON AVX2 - MergeARGBRow_C NEON NEON AVX2 RVV MergeRGBRow_C NEON NEON AVX512VBMI RVV MergeUVRow_C NEON SME AVX512BW RVV MergeUVRow_16_C NEON SME AVX2 - MergeXR30Row_C NEON NEON AVX512BW - MergeXR64Row_C NEON NEON AVX2 - MergeXRGB16To8Row_C NEON NEON AVX2 - MergeXRGBRow_C NEON NEON AVX2 RVV MirrorRow_C NEON NEON AVX512BW - MirrorRow_16_C NEON NEON AVX512BW - MirrorSplitUVRow_C NEON NEON AVX512BW - MirrorUVRow_C NEON NEON AVX512BW RVV MultiplyRow_16_C NEON SME AVX512BW RVV NV12ToARGBRow_C NEON SME AVX512BW RVV NV12ToRGB24Row_C NEON SME AVX512VBMI RVV NV12ToRGB565Row_C NEON NEON AVX2 - NV21ToARGBRow_C NEON SME AVX512BW RVV NV21ToRGB24Row_C NEON SME AVX512VBMI RVV NV21ToYUV24Row_C NEON NEON AVX2 - P210ToAR30Row_C - SME AVX2 - P210ToARGBRow_C - SME AVX2 - P410ToAR30Row_C - SME AVX2 - P410ToARGBRow_C - SME AVX2 - RAWToARGBRow_C NEON SVE2 AVX512BW RVV RAWToRGB24Row_C NEON SVE2 AVX512VBMI RVV RAWToRGBARow_C NEON SVE2 AVX512VBMI RVV RGB24MirrorRow_C NEON NEON AVX512VBMI - RGB24ToARGBRow_C NEON SVE2 AVX512BW RVV RGB565ToARGBRow_C NEON NEON AVX2 - RGB565ToUVMatrixRow_C NEON NEON AVX2 - RGB565ToUVRow_C NEON NEON - - RGB565ToYMatrixRow_C NEON NEON AVX2 - RGB565ToYRow_C NEON NEON - - RGBAToARGBRow_C NEON NEON AVX512BW RVV RGBColorTableRow_C NEON NEON X86 - RGBToUV444MatrixRow_C NEON NEON AVX512BW RVV RGBToUVMatrixRow_C NEON NEON AVX512BW RVV RGBToYMatrixRow_C NEON DOTPROD AVX512BW RVV SetRow_C NEON NEON ERMS - SobelRow_C NEON NEON SSE2 - SobelToPlaneRow_C NEON NEON SSE2 - SobelXRow_C NEON NEON SSE2 - SobelXYRow_C NEON NEON SSE2 - SobelYRow_C NEON NEON SSE2 - SplitARGBRow_C NEON NEON AVX2 RVV SplitRGBRow_C NEON NEON AVX2 RVV SplitUVRow_C NEON SVE2 AVX512BW RVV SplitUVRow_16_C NEON NEON AVX2 - SplitXRGBRow_C NEON NEON AVX2 RVV SwapUVRow_C NEON NEON AVX512BW RVV UYVYToARGBRow_C NEON SME AVX2 - UYVYToUV422Row_C NEON NEON AVX2 - UYVYToUVRow_C NEON SVE2 AVX2 - UYVYToYRow_C NEON SVE2 AVX2 - YUY2ToARGBRow_C NEON SME AVX2 - YUY2ToNVUVRow_C NEON NEON AVX2 - YUY2ToUV422Row_C NEON NEON AVX2 - YUY2ToUVRow_C NEON SVE2 AVX2 - YUY2ToYRow_C NEON SVE2 AVX2 - ---------------------------------------------------------------- SIMD (164/164 100%) 146 89% 164 100% 154 94% 64 39% ---------------------------------------------------------------- Test: libyuv_unittest --gtest_filter=*AR30*:*Affine*:*ColorTable*:*CopyAlpha*:*Polynomial*:*Unattenuate*:*MirrorPlane_16* Bug: None Change-Id: Ib8b68beb77b47de6b996d41fd080397c61539ffb Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8666404 Reviewed-by: richard winterton <rrwinterton@gmail.com> Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-10-27 16:51:21 -07:00
Frank Barchard	205022c08c	[libyuv] Add AVX-512 VNNI row functions for ARGB to YUV and Gray ... Add AVX-512 VNNI (VPDPBUSD / _mm512_dpbusd_epi32) row kernels for 64-bit x86 targets alongside existing AVX-512BW implementations in both row_gcc.cc (inline assembly) and row_win.cc (C++ intrinsics): - ARGBToYMatrixRow_AVX512VNNI and ARGBToYMatrixRow_Any_AVX512VNNI: Pre-computes the 32-bit signed-pixel bias once per row and replaces vpmaddubsw + vpmaddwd + vpaddw with a single vpdpbusd per 16 pixels. - ARGBToUV444MatrixRow_AVX512VNNI and ARGBToUV444MatrixRow_Any_AVX512VNNI: Loads 64 ARGB pixels once instead of twice and computes 32-bit U and V dot products directly via vpdpbusd with signed UV matrix coefficients. - ARGBToUVMatrixRow_AVX512VNNI: Computes 2x2 box-filtered ARGB to U and V conversion using vpdpbusd with k-masked tail handling for arbitrary widths. - ARGBGrayRow_AVX512VNNI: Computes BT.601 grayscale luminance using a single vpdpbusd per 16 pixels, broadcasts Y to RGB via vpshufb, and preserves alpha via vpternlogd with k-masked tail handling. - Wire runtime dispatch via TestCpuFlag(kCpuHasAVX512VNNI) across convert.cc, convert_from_argb.cc, and planar_functions.cc. Add AVX512VNNIVBMI variants of the Y, UV444 and UV kernels. These use both AVX-512 VNNI (vpdpbusd) and AVX-512 VBMI (vpermt2b) and are only selected when the CPU has both features; there are no VBMI versions without VNNI. The AVX512VNNI kernels remain for CPUs with VNNI but no VBMI (Cascade Lake, Cooper Lake). - The 32-bit sums are always in [0, 65535], so Y, U and V are byte 1 of each int32. Two vpermt2b gather 64 results from 4 accumulators, replacing vpackusdw/vpackssdw + vpsrlw + vpackuswb + vpermd. - U and V accumulators start at ~kAddUV so they hold ~(kAddUV - dot). The inverse is folded into the vpternlogd that merges the two 256-bit halves. kAddUV is loaded from ArgbConstants rather than hard coded as 0x8000. - Y folds the source loads into vpxorq. - row_win.cc hides the vpermt2b indices from clang, which otherwise constant folds them into a longer vpshufb + vpor + vpermd sequence. Performance (1280x720, 5000 iterations, AVX512BW -> AVX512VNNI): uArch / Benchmark AVX512BW AVX512VNNI Speedup --------------------------- -------- ---------- ------- Intel Ice Lake ARGBToI420 941 ms 862 ms +9.2% Intel Ice Lake ARGBToI422 1059 ms 947 ms +11.8% Intel Ice Lake ARGBToI444 1140 ms 1038 ms +9.8% Intel Ice Lake ARGBGray 357 ms 273 ms +30.8% Intel Sapphire Rapids GrayTo 317 ms 218 ms +45.4% Intel Emerald Rapids GrayTo 310 ms 213 ms +45.5% Intel Granite Rapids GrayTo 349 ms 241 ms +44.8% AMD Zen 4 ARGBToI420 648 ms 517 ms +25.3% AMD Zen 4 ARGBToI444 695 ms 526 ms +32.1% AMD Zen 4 ARGBGray 403 ms 227 ms +77.5% AMD Zen 5 ARGBToI422 460 ms 403 ms +14.1% AMD Zen 5 ARGBToI444 382 ms 321 ms +19.0% Performance (row_gcc, AVX512VNNI -> AVX512VNNIVBMI): uArch / Benchmark AVX512VNNI VNNIVBMI Speedup ---------------------------- ---------- -------- ------- AMD Zen 6 ARGBToI420 1280x720 327 ms 284 ms +15.1% AMD Zen 6 ARGBToI422 1280x720 372 ms 330 ms +12.7% AMD Zen 6 ARGBToI444 1280x720 357 ms 334 ms +6.9% AMD Zen 6 ARGBToI400 256x24 94 ms 76 ms +23.7% AMD Zen 5 ARGBToI422 1280x720 406 ms 350 ms +16.0% AMD Zen 5 ARGBToI444 1280x720 330 ms 301 ms +9.6% AMD Zen 5 ARGBToI400 256x24 100 ms 90 ms +11.1% AMD Zen 4 ARGBToI420 1280x720 631 ms 540 ms +16.9% AMD Zen 4 ARGBToI422 1280x720 993 ms 825 ms +20.4% AMD Zen 4 ARGBToI400 256x24 245 ms 218 ms +12.4% Intel GNR ARGBToI422 256x24 654 ms 616 ms +6.2% Intel SPR ARGBToI422 256x24 653 ms 613 ms +6.5% Intel GNR, EMR, SPR and Ice Lake are within +/-3% at 1280x720. R=rrwinterton@gmail.com Bug: none Test: out/Release/libyuv_unittest --gtest_filter=*ARGBToI4*:*ARGBGray* Test: LIBYUV_DISABLE_AVX512VBMI=1 out/Release/libyuv_unittest Test: sde64 -gnr / -clx with is_msan=true (row_win.cc) Change-Id: Ia7a37b39dd2948fe4055a8a7694dd1ff8ac3e55c Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8647811 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-10-26 14:07:50 -07:00
Frank Barchard	38e21134f7	[libyuv] Add ARGBOr and ARGBAlphaBlend with multi-platform SIMD ... Add two new planar/row compositing functions for 32-bit ARGB/BGRA scanlines: - ARGBOr / ARGBOrRow_*: Applies a bitwise OR with a 32-bit ARGB mask to every pixel (e.g., 0xFF000000 to force opaque alpha for BGRx to BGRA conversion). Implemented for C, SSE2, AVX2, AVX-512BW, NEON (32/64-bit), SVE2, SME, RVV, WASMSIMD, LSX, LASX, HVX, and Windows intrinsics (row_win.cc). - ARGBAlphaBlend / ARGBAlphaBlendRow_*: Implements straight-alpha (unattenuated) Porter-Duff source-over blending matching PDFium's CompositeRowBgra2Bgra: dst_a = back_a + src_a - (back_a * src_a) / 255 ratio = (src_a * 255) / dst_a dst_c = (back_c * (255 - ratio) + src_c * ratio) / 255 Implemented with bit-exact division by 255 across all 65,536 (back_a, src_a) pairs for C, SSSE3, AVX2, AVX-512BW, NEON (32/64-bit), SVE2, SME, RVV, WASMSIMD, LSX, LASX, HVX, and Windows intrinsics (row_win.cc). ARGBOr Performance (2048x64 BGRx->BGRA scanlines, us per call): uArch Before Highway libyuv vs Hwy ------------------------ ------ ------- ------- ------ Qualcomm Hexagon v81 HVX 259.63 260.97 15.78 16.54x SiFive X280 (RVV) 1107.57 72.49 50.70 1.43x SiFive X390 (RVV) 1108.82 70.42 52.26 1.35x ARGBAlphaBlend Performance (2048x64 BGRA scanlines, us per call): uArch Before Highway libyuv vs Hwy ------------------------ ------ ------- ------- ------ Intel Lion Cove (AVX2) 429.79 141.97 41.89 3.39x Intel Cougar Cove (AVX2) 516.83 181.98 51.33 3.55x Intel Golden Cove (AVX2) 560.31 206.28 67.08 3.08x AMD Zen 3 (AVX2) 1025.73 251.09 98.99 2.54x Wasm SIMD128 (V8) 1014.14 964.83 385.80 2.50x Apple M6 Super (NEON) 221.90 118.42 46.35 2.55x Apple M5 Super (NEON) 241.90 130.07 49.60 2.62x ARM C1-Ultra (SME2) 384.63 112.52 63.62 1.77x Qualcomm Oryon V3-L (SME) 388.13 154.87 87.65 1.77x Qualcomm Oryon-L (NEON) 318.28 149.36 57.23 2.61x ARM Cortex-X4 (SVE2) 675.47 213.56 102.53 2.08x ARM Cortex-A720 (NEON) 795.85 586.55 235.81 2.49x ARM Cortex-A520 (NEON) 2360.95 2122.99 787.65 2.70x ARM Cortex-A35 (ARMv7) 4907.31 5895.04 1474.51 4.00x SiFive X280 (RVV) 4145.06 1096.42 596.82 1.84x SiFive X390 (RVV) 4144.57 1084.93 597.62 1.82x Qualcomm Hexagon v81 HVX 4912.03 5786.25 952.19 6.08x Coral V4 RVV (1024 px) 50276 c 48992 c 4299 c 11.40x R=wtc@google.com Bug: none Change-Id: If661239826a033e89d4b5c1adc0bfdf048a7a726 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8647895 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-10-26 11:24:16 -07:00
Frank Barchard	f249eaa72f	[libyuv] Fix top 20 bugs and performance issues ... 1. Fix multi-bit TestCpuFlag(A | B) checks in AYUVToNV12, AYUVToNV21, RAWToRGBA, NV12ToRGB24Matrix, NV21ToRGB24Matrix, MergeRGBPlane, and MergeXR30Plane to compare == (A | B) so AVX-512 VBMI/BW kernels are not dispatched when AVX512VBMI or AVX512VL is absent. 2. Fix 2-byte SVE2 heap buffer overflow on odd widths in I422ToARGB1555Row_SVE_SC and I422ToARGB4444Row_SVE_SC using trn1/trn2 + st2b under .b predicates. 3. Fix ARGBPolynomialRow_SSE2 to use packssdw before packuswb, and add missing vzeroupper to SplitRGBRow_AVX2. 4. Fix CalcFrameSsim to return 0.0 instead of 0.0 / 0 (NaN) when width <= 8 || height <= 8 so I420Ssim stays finite on small frames. 5. Fix negative-height vertical inversion in ConvertFromI420, SplitARGBPlane (when dst_a is NULL), I422Rotate, I010Rotate, and I210Rotate. 6. Fix horizontal flip (negative x/dx) in ScaleFilterCols_NEON (signed sbfx #16, #16 offset extraction) and ScaleAddCols1_16_C (shift x >>= 16 before indexing src_ptr). 7. Fix ConvertToARGB odd crop_x chroma alignment in NV12/NV21, src_u offset in I420/YV12/J420/H420/U420, and FOURCC_U422 calling U422ToARGB instead of H422ToARGB. 8. Remove dead 14-instruction block in ScaleRowUp2_Bilinear_16_SSE2. 9. Fix UVScale_16 for src_width == 1 (P010ToP410/P210ToP410), enable and fix column order in TransposeWx4_32_SVE2, fix ByteToFloatRow NEON Any BPP (3 -> 4), fix AB64ToARGBRow_Any_*, ScaleColsUp2_SSE2, ScaleUVRowDown2Box_SME, and UYVYToARGBRow_SME macro guards, fix SVE2 loop tail branches (b.lt/b.ge), and remove duplicate HAS_* macros. 10. Fix ConvertFP16ToFP32Column_NEON loop condition and unsigned b.lo underflow on widths < 8 or not multiples of 8. 11. Fix UYVYToNV12 IS_ALIGNED(awidth, ...) checks and WASM SIMD IS_ALIGNED(width, 16) checks in I420ToRGB24Matrix / I422ToRGB24Matrix. 12. Wire SplitUVRow_SVE2, SplitUVRow_LSX, and SplitUVRow_LASX in NV24ToARGB. 13. Fix MJpegDecoder heap buffer overflows in LoadFrame (reallocate databuf_[i] when scanlines_size changes), DecodeToBuffers, and DecodeToCallback (clamp scanlines_to_copy to lines_left and align skip to max_v_samp_factor). 14. Replace width & (vl - 1) tail calculations with post-loop remainder across 18 SVE2 functions in row_sve.h for non-power-of-2 SVE VL, and remove duplicate label 2: in UYVYToARGBRow_SVE_SC. 15. Handle width & 15 tail pixels in DetileToYUY2_C and validate IS_POWEROFTWO(tile_height) in DetileSplitUVPlane and DetileToYUY2. 16. Avoid signed 32-bit overflow in DivideRow_16_C and C16TO8, fix ClampFloatToZero_SSE2 pointer increment (lea 0x4) and loop counter (sub $0x1), and initialize xmm3 in ARGBCopyYToAlphaRow_SSE2. 17. Check max leaf 7 subleaf and max extended CPUID leaf in GetCpuFlags before querying subleaf 7:1 and extended leaf 0x80000021. 18. Free plane_u on error in I422ToNV21, and fix 4x/2x scratch row over-allocation in ScaleARGBBilinearDown and ScaleUVBilinearDown. 19. Add ARGBSetRow_RVV and ARGBShuffleRow_RVV in row_rvv.cc. 20. Fix NEON mask (15 -> 7) in ScaleUVRowUp2_Linear_Any_NEON / _16_Any_NEON, use SIMD InterpolateRow + SplitUVRow in NV16ToI420, wire SSSE3/SSE2 in ARGBToI400Matrix / ARGBToYUY2Matrix / ARGBToUYVYMatrix and RGB24Mirror, and split HalfFloatRow multiply into (src * scale) * 2^-112 to avoid 50x Intel denormal stalls. Bug: b/574714081 Test: libyuv_unittest Change-Id: Iec3ae5c58d52bee3ef13cf9b765ac84b733aa576 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8601931 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-10-19 16:46:40 -07:00
Frank Barchard	2404f9730e	[libyuv] Rewrite LoongArch LSX/LASX kernels in compact inline asm ... Reduce loop unrolling across LoongArch LSX (128-bit) and LASX (256-bit) compare, row, rotate, and scale kernels to match SSE2/SSSE3 and AVX2 (1-2 vectors per iteration), rewrite intrinsics/macro implementations into GCC/Clang inline assembly, update all _Any wrapper masks and IS_ALIGNED dispatch checks, and port all remaining LSX Convert (+19) and Rotate (+2) kernels to LASX for full parity: ISA | Compare | Convert | Rotate | Scale -----------------|---------|---------|---------|-------- MIPS/Loong (Any) | 1 | 68 | 4 | 20 LSX | 1 | 68 | 4 | 20 LASX | 1 | 68 | 4 | 0 Removes 1,935 macro/helper lines from loongson_intrinsics.h (-99.3%) and reduces total LoongArch compiled .text size by 9,799 bytes (-25.1% overall, -37.4% row_lsx.cc, -52.3% scale_lsx.cc, -31.2% rotate_lsx.cc, -4.9% row_lasx.cc while adding +25 LASX functions): File | Before | After | Change -------------------|----------|----------|-------- scale_lsx.cc.o | 6,055 B | 2,888 B | -52.3% row_lsx.cc.o | 17,252 B | 10,792 B | -37.4% rotate_lsx.cc.o | 1,488 B | 1,024 B | -31.2% row_lasx.cc.o | 13,740 B | 13,072 B | -4.9% rotate_lasx.cc.o | 544 B | 1,504 B | +176.5% TOTAL .text | 39,079 B | 29,280 B | -25.1% R=wtc@google.com, yuanhecai@loongson.cn Test: qemu-loongarch64 -cpu la464 libyuv_unittest (LASX and LIBYUV_DISABLE_LASX=1) Change-Id: I11c1143ad2aad0399f10b2c3fc9bc4c02c740c59 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8564659 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-10-16 17:25:23 -07:00
Frank Barchard	63b56324de	[libyuv] Replace AVX-512 _Any wrappers with k-mask tail handling ... Replace stack-buffer memcpy _Any wrappers with native AVX-512 k-mask tail handling across 8 AVX-512 row kernels (unlocking composite RGBToUVMatrixRow_AVX512BW as well) and remove IS_ALIGNED dispatch constraints: - MirrorSplitUVRow_AVX512BW: Masked 16-bit load (vmovdqu16 %{k1%}%{z%}), dynamic reverse index (vpsubw + vpermw via kMirrorUVPermute_AVX512), and vpmovwb %{k1%} U/V stores; unlocks unaligned SplitRotateUV180. - ARGBToUVMatrixRow_AVX512BW: Fix MASK=63 vs 32-pixel loop bug; zero-load tail dwords per row, duplicate odd final pixel via knotw + vpshufd, and store masked U/V bytes. - I422ToARGBRow_AVX512BW & I422ToAR30Row_AVX512BW: Add masked Y/U/V loads (READYUV422_MASK_AVX512BW) and masked ARGB/AR30 stores. - CopyRow_AVX512BW: Add 64-byte full vector step + bzhi masked tail. - BlendPlaneRow_AVX512BW, ARGBShuffleRow_AVX512BW, RGBToARGBRow_AVX512BW (RAWToARGBRow_AVX512BW / RGB24ToARGBRow_AVX512BW): Masked loads and stores in both row_gcc.cc inline assembly and row_win.cc intrinsics. Reduces x86-64 binary .text size by 2,592 bytes with 0 bytes added to .rodata. Performance on Intel Sapphire Rapids (1275x1650, 1000 iterations): Function (Test) | Before | After | Speedup -------------------------|---------|---------|-------- NV12Rotate180_Opt | 429 ms | 260 ms | 1.65x RGB24ToI420_Opt | 615 ms | 531 ms | 1.16x I420ToAR30_Opt | 425 ms | 383 ms | 1.11x ARGBToI420_Opt | 427 ms | 411 ms | 1.04x I420ToARGB_Opt | 423 ms | 411 ms | 1.03x RGB24ToARGB_Opt | 493 ms | 490 ms | 1.01x ARGBToBGRA_Opt | 544 ms | 542 ms | 1.00x CopyPlane_Opt | 246 ms | 245 ms | 1.00x BlendPlane_Opt | 259 ms | 258 ms | 1.00x RAWToARGB_Opt | 486 ms | 486 ms | 1.00x R=rrwinterton@gmail.com Test: LIBYUV_WIDTH=1275 LIBYUV_HEIGHT=1650 LIBYUV_REPEAT=1000 out/Release/libyuv_unittest Change-Id: Ib5e5ae52076352649108b20f8df2b12d81860fb1 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8563977 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-10-12 15:33:19 -07:00
Frank Barchard	16af162cf9	[libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes ... Implement SIMD 16-bit plane rotation (TransposePlane_16, RotatePlane90_16, RotatePlane180_16, RotatePlane270_16) used by I010Rotate, I210Rotate, and I410Rotate, keeping tile sizes compact with full-register writes and zero cross-lane shuffles: - AVX2: 8x16 tile in ymm0..ymm15 with _Any tail (low-end x86 SIMD) - AVX-512BW: 8x32 tile in zmm0..zmm15 with k1/k2 right/bottom masks - NEON (ARM64/ARM32): 8x16 tile in v0..v23 / q0..q11 with _Any tail - SME: 8xVL tile using ZA16 horizontal/vertical array slices - RVV: 8xVL tile using widening unpacks with hardware vl tail - WASM SIMD: 8x8 tile in v128 with _Any tail Also update 32-bit ARGBRotate x86 transposes to compact vertical in-lane tiles with full-register writes and zero cross-lane shuffles: - TransposeWxH_32_AVX2: 4x8 tile in ymm0..ymm7 with vpmaskmovd tails - TransposeWxH_32_AVX512BW: 4x16 tile in zmm0..zmm7 with k1/k2 tails Add RotatePlane16_SplitMergeUV_Opt and RotatePlane16_SplitMergeUV_Odd unit tests comparing 16-bit rotation against SplitUVPlane + RotatePlane + MergeUVPlane across 0/90/180/270 degrees. Performance on 1275x1650 (1000 iterations, before -> after): CPU 90 degree 180 degree Coral RVV (Verilator) 7306 -> 1415c (5.16x) 2274 -> 719c (3.16x) AMD Rome (Zen 2) 821 -> 215ms (3.82x) 259 -> 181ms (1.44x) AMD Genoa (Zen 4) 483 -> 129ms (3.73x) 149 -> 111ms (1.34x) AMD Turin (Zen 5) 321 -> 101ms (3.18x) 109 -> 78ms (1.40x) AMD Milan (Zen 3) 505 -> 189ms (2.67x) 199 -> 118ms (1.69x) Intel Skylake-X 1076 -> 447ms (2.41x) 353 -> 317ms (1.11x) WASM SIMD (d8) 415 -> 203ms (2.04x) 370 -> 106ms (3.49x) Intel Emerald Rapids 735 -> 422ms (1.74x) 238 -> 233ms (1.02x) Intel Haswell 1120 -> 648ms (1.73x) 438 -> 393ms (1.11x) Intel Ice Lake 774 -> 474ms (1.63x) 279 -> 290ms (0.96x) ARM Neoverse-N1 864 -> 544ms (1.59x) 404 -> 266ms (1.52x) Ampere Siryn 862 -> 576ms (1.50x) 568 -> 329ms (1.72x) Google Axion (Maple) 554 -> 379ms (1.46x) 199 -> 142ms (1.41x) Intel Sapphire Rapids 1126 -> 772ms (1.46x) 301 -> 288ms (1.04x) Intel Granite Rapids 730 -> 565ms (1.29x) 280 -> 280ms (1.00x) NVIDIA Grace 462 -> 365ms (1.26x) 192 -> 126ms (1.52x) ARM Cortex-A53 28290 -> 24386ms (1.16x) 7178 -> 5075ms (1.41x) Google Axion 2 (Tamar) 523 -> 453ms (1.15x) 217 -> 112ms (1.93x) Test: libyuv_unittest --gtest_filter=*Rotate* Bug: b/42280942 Change-Id: I91ba6cac22185b5d294fcb98344fdb8a23cc94ff Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8545294 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-10-12 10:57:34 -07:00
Frank Barchard	eeb06a6451	[libyuv] Optimize ARGBRotate for Intel SSE2, AVX2, and AVX-512 ... Optimize 32-bit ARGBRotate (0, 90, 180, 270) across Intel x86/x64 architectures (SSE2, AVX2, AVX512BW), with matching kernels for ARM (NEON, SVE2, SME), WASM, LoongArch (LSX, LASX), and RISC-V (RVV): - Replace column-by-column ScaleARGBRowDownEven gather loops in ARGBTranspose (90/270) with full-plane horizontal tile transposes: - AVX512BW (TransposeWxH_32_AVX512BW / TransposeNx4_32_AVX512BW): Transpose 16x16 blocks across zmm0..zmm31 using 4-stage vpunpckldq/hdq, vpunpcklqdq/hqdq, and vshufi32x4 butterflies with k-masked right and bottom tails, eliminating Any wrappers and scalar C edge fallbacks. - AVX2 (TransposeWxH_32_AVX2 / TransposeNx4_32_AVX2): Transpose 8x8 blocks across ymm0..ymm15 using vpunpck* and vperm2i128 with vpmaskmovd masked tails. - SSE2 (Transpose4x4_32_SSE2 / Transpose4x4_32_Any_SSE2): Transpose 4x4 blocks in xmm registers. - Implement ARGBRotate180 directly via ARGBMirror with negative height, avoiding intermediate row buffer allocations and extra copies. - Implement ARGBRotate0 via ARGBCopy. Speedup vs Before (1275x1650, US Letter 150 DPI, 1000 iterations): CPU | 0 deg | 90 deg | 180 deg | 270 deg ---------------------|---------|---------|---------|-------- Intel Ice Lake | 1.02x | 1.34x | 1.18x | 1.33x Intel Skylake-X | 1.02x | 1.20x | 1.07x | 1.13x Intel Emerald Rapids | 1.01x | 1.07x | 1.00x | 1.07x Intel AVX-512 vs C | 1.00x | 2.67x | 2.41x | 2.63x R=rrwinterton@gmail.com Bug: b/42280915 Test: LIBYUV_WIDTH=1275 LIBYUV_HEIGHT=1650 LIBYUV_REPEAT=1000 ./out/Release/libyuv_unittest --gtest_filter=*ARGBRotate* Change-Id: Ib308a225c17e46eab083a9da4033598fcf0c303c Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8538986 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-10-08 16:20:46 -07:00
Frank Barchard	1ddb18fb6b	[libyuv] AVX512, NEON, WASM and RVV alpha copy and mirror rows ... 1275x1650 (US Letter 150 DPI, 1000 iterations) speedup: CPU ARGBExtractAlpha ARGBCopyYToAlpha ARGBMirror RGB24Mirror ----------------------------------------------------------------------- Intel Granite Rapids 1.00x 1.00x 1.00x 1.04x Intel Emerald Rapids 1.01x 0.97x 1.03x 1.07x Intel Sapphire Rapids 0.97x 1.01x 1.08x 1.26x Intel Ice Lake 0.99x 1.02x 0.97x 1.08x Intel Skylake-X 1.04x 1.03x 1.01x NA Intel Haswell NA NA NA NA AMD Turin (Zen 5) 1.01x 1.10x 1.25x 1.31x AMD Genoa (Zen 4) 0.97x 1.04x 1.09x 1.42x AMD Milan (Zen 3) NA NA NA NA AMD Rome (Zen 2) NA NA NA NA NVIDIA Grace (V2) 1.57x 2.51x NA NA Google Axion 2 1.12x 3.17x NA NA Ampere Siryn 1.06x 1.70x NA NA ARM Neoverse-N1 1.49x 2.01x NA NA Google Axion (Maple) 1.47x 2.03x NA NA ARM Cortex-A53 1.06x 1.69x NA NA Add AVX512BW ARGBExtractAlphaRow, ARGBCopyYToAlphaRow and ARGBMirrorRow, and AVX512VBMI RGB24MirrorRow, in row_gcc.cc and as intrinsics in row_win.cc. They are x64 only. The remainder is done with a k-mask (bzhi) zero-masked load and a masked store, so they handle any width without an Any wrapper and never access memory outside the row. RGB24MirrorRow_AVX512VBMI loads and stores 3 full zmm vectors per 64 pixels using 3 vpermi2b. The one output byte that comes from the third input vector is merged with a masked vpbroadcastb. The 1 to 63 pixel tail rebases the same tables by 192 - 3 * r; indices >= 128 are taken from the third vector with vpmovb2m and a masked vpermb. Add ARGBCopyYToAlphaRow_NEON for Arm 32 bit (vld4 + vst4) and AArch64 (zip + sli), and rewrite AArch64 ARGBExtractAlphaRow_NEON with ldp + uzp2. Both prefetch source and destination, which is 2.5x faster on Neoverse-N1 and 1.2x to 1.5x faster on Cortex-A53 for 1280x720. Add WASM SIMD ARGBExtractAlphaRow and ARGBCopyYToAlphaRow, using a swizzle table for CopyYToAlpha. Rework the RVV versions to use unit stride loads with vnsrl and vzext.vf4. R=rrwinterton@gmail.com Bug: None Change-Id: I436f27d70a3db0a2b176c91eb789a9491133cfcb Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8523531 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-10-07 12:44:53 -07:00
Frank Barchard	98697ab033	[libyuv] Add WASM SIMD128 compare, rotate, scale and row functions ... d8 geomean speedup vs C compiled with -O2 -msimd128: File AMD Zen 3 ARM Neoverse-V2 row_wasm.cc 5.67x 3.24x scale_wasm.cc 5.47x 3.17x compare_wasm.cc 4.58x 5.77x rotate_wasm.cc 6.83x 6.03x Bug: None Change-Id: Ia06c805dc4029f94842c40e7301b2933beb8c1e5 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8524106 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-10-06 19:05:05 -07:00
Frank Barchard	eb8eda9c79	[libyuv] Add WASM SIMD128 and Relaxed SIMD for RAWToRGB24Row ... Implement RAWToRGB24Row_WASMSIMD and RAWToRGB24Row_Any_WASMSIMD in source/row_wasm.cc for RGB24ToRAW and RAWToRGB24 conversions: - Process 16 pixels (48 bytes) per loop iteration using three 128-bit loads, three byte-wise shuffles, and four 2-way blends. - Use wasm_i8x16_relaxed_swizzle when __wasm_relaxed_simd__ is enabled (compiling to a single vpshufb on x86 without bounds-checking overhead) and fall back to wasm_i8x16_swizzle for baseline SIMD128. - Add kCpuHasWASM (0x100) and kCpuHasWASMSIMD (0x101) CPU feature flags and enable -msimd128 in BUILD.bazel, BUILD.gn, and CMakeLists.txt for WebAssembly targets. Performance (RAWToRGB24_Opt, 1280x720, V8 d8): CPU WASM C SIMD128 Relaxed SIMD Spd Rlx Spd ------------------------------------------------------------------- AMD Zen 4 595.0 us 137.0 us 84.0 us 4.34x 7.08x AMD Zen 3 708.0 us 148.0 us 107.0 us 4.78x 6.62x AMD Zen 5 375.0 us 107.0 us 67.0 us 3.50x 5.60x ARM Neoverse-V2 638.0 us 118.0 us 116.0 us 5.41x 5.50x ARM Neoverse-N1 1046.0 us 294.0 us 294.0 us 3.56x 3.56x Intel Ice Lake 707.0 us 209.0 us 220.0 us 3.38x 3.21x Intel Haswell 1110.0 us 344.0 us 360.0 us 3.23x 3.08x Intel Emerald Rapids 519.0 us 207.0 us 208.0 us 2.51x 2.50x Intel Sapphire Rapids 551.0 us 220.0 us 255.0 us 2.50x 2.16x ARM Cortex-A53 14440.0 us 8660.0us 6920.0us 1.67x 2.09x Intel Granite Rapids 453.0 us 225.0 us 221.0 us 2.01x 2.05x Bug: 42271072 Test: libyuv_test --gunit_filter=*RAWToRGB24*:*RGB24ToRAW*:*Cpu* Change-Id: Id902668263a2ab70aa300eeeea18851490c917b5 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8515581 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-10-05 17:45:41 -07:00
Frank Barchard	2b7ecc0aad	RAWToRGB24 for NEON, x86, and LoongArch ... R=rrwinterton@gmail.com Test: libyuv_unittest --gtest_filter=*RAWToRGB24*:*RGB24ToRAW* Bug: None Change-Id: I4454f3785ba5c39dc776ba1f5151cd9d3e68055b Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8509286 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-10-05 12:09:40 -07:00
Frank Barchard	18835bfb35	ARGBToRGB24 AVX512 with tail masking ... Bug: None Change-Id: I7590c66940b5354ca0fdd8ff120392434d2785d7 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8490011 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-09-29 16:15:44 -07:00
Frank Barchard	783ea90384	[libyuv] Enable LIBYUV_ENABLE_ROWWIN and MSAN_MEMSET for MSan builds ... Enable LIBYUV_ENABLE_ROWWIN instead of LIBYUV_DISABLE_X86 under MemorySanitizer so that C++ SIMD intrinsics in row_win.cc and their corresponding _Any wrappers in row_any.cc are compiled and tested under MSan. Rename MEMSET to MSAN_MEMSET in row_any.cc and scale_any.cc, and initialize vout buffers across all ANY* macros in row_any.cc. Use calloc for align_buffer_64 and align_buffer_page_end under LIBYUV_MSAN, and zero-initialize stack pixel buffers in color_test.cc. Test: out/msan/libyuv_unittest Bug: None Change-Id: I0b9fc69af159531699bb2b2db8b48f39831289ee Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8456982 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-09-28 10:49:58 -07:00
Frank Barchard	f1c9b58986	clang format and bump version ... Bug: None Change-Id: I6a482d5c538b113549b88e820473af644d6d3704 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8467772 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-09-28 10:42:45 -07:00
Frank Barchard	2124173f34	I422ToRGB24 avx512bw tail and row coalesce ... I422ToRGB24Matrix (299x299, repeat=10000) ==================================================================== CPU Before After Improvement -------------------------------------------------------------------- Intel Skylake-X 275 ms 213 ms -22.5% (1.29x) Intel Ice Lake 225 ms 177 ms -21.3% (1.27x) Intel Sapphire Rapids 181 ms 144 ms -20.4% (1.26x) Intel Emerald Rapids 167 ms 136 ms -18.6% (1.23x) Intel Granite Rapids 170 ms 140 ms -17.6% (1.21x) AMD Zen 4 (Genoa) 192 ms 136 ms -29.2% (1.41x) AMD Zen 5 (Turin) 102 ms 68 ms -33.3% (1.50x) AMD Zen 6 (Venice) 102 ms 60 ms -41.2% (1.70x) ==================================================================== I422ToRGB24Matrix (298x299, repeat=10000) ==================================================================== CPU Before After Improvement -------------------------------------------------------------------- Intel Skylake-X 271 ms 184 ms -32.1% (1.47x) Intel Ice Lake 235 ms 152 ms -35.3% (1.55x) Intel Sapphire Rapids 199 ms 128 ms -35.7% (1.55x) Intel Emerald Rapids 170 ms 121 ms -28.8% (1.40x) Intel Granite Rapids 177 ms 125 ms -29.4% (1.42x) AMD Zen 4 (Genoa) 185 ms 119 ms -35.7% (1.55x) AMD Zen 5 (Turin) 102 ms 59 ms -42.2% (1.73x) AMD Zen 6 (Venice) 100 ms 51 ms -49.0% (1.96x) ==================================================================== Bug: None Change-Id: Id75e3d1df6d26bc28a252672816a7c438cb664a4 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8452502 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-09-24 12:07:29 -07:00
Frank Barchard	41a6e684a6	I422ToRGB24Row_AVX512VBMI handle tail with mask ... I422ToRGB24Matrix (299x299, repeat=10000) ==================================================================== CPU Before After Improvement -------------------------------------------------------------------- Intel Ice Lake 220 ms 181 ms -17.7% (1.22x) Intel Sapphire Rapids 181 ms 145 ms -19.9% (1.25x) Intel Emerald Rapids 169 ms 148 ms -12.4% (1.14x) Intel Granite Rapids 191 ms 141 ms -26.2% (1.35x) AMD Zen 4 (Genoa) 171 ms 136 ms -20.5% (1.26x) AMD Zen 5 (Turin) 94 ms 72 ms -23.4% (1.31x) AMD Zen 6 (Venice) 95 ms 66 ms -30.5% (1.44x) ==================================================================== Bug: None Change-Id: I4cd657410cc5253b5e8fd2caa78d17e2b3812393 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8445804 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-09-23 22:42:30 -07:00
Frank Barchard	a09d909f63	[libyuv] Only memset row_any temporary buffers for MSAN ... libyuv_test '--gunit_filter=*I422ToRGB24_Opt' --libyuv_width=299 --libyuv_height=299 --libyuv_repeat=10000 --libyuv_flags=-1 --libyuv_cpu_info=-1 Was I422ToRGB24_Opt (370 ms) 88.46% I422ToRGB24Row_AVX2 5.57% __llvm_libc_google3::memcpy() 4.16% I422ToRGB24Row_Any_AVX2 0.54% I422ToRGB24Matrix R=rrwinterton@gmail.com Bug: None Test: libyuv_unittest --gtest_filter=*Any* Change-Id: Ib8a16eaa7b59b71d8567f9a5f2291b31998486e9 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8447593 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-09-22 19:19:45 -07:00
Frank Barchard	f51ad56a3e	Multiply_16 port to avx512, rvv, neon 32 bit, and windows ... - Also remove redundant neon check for I8MM kernels ====================================================================== CPU Before After Speedup ---------------------------------------------------------------------- Intel Granite Rapids (AVX10.1) 251 ms 141 ms 1.78x (+43.8%) Intel Emerald Rapids (AVX512) 199 ms 108 ms 1.84x (+45.7%) Intel Sapphire Rapids (AVX512) 230 ms 112 ms 2.05x (+51.3%) AMD Turin Zen 5 (AVX512) 156 ms 99 ms 1.58x (+36.5%) AMD Venice Zen 6 (AVX512) 204 ms 106 ms 1.92x (+48.0%) Intel Skylake-X (AVX512) 320 ms 156 ms 2.05x (+51.2%) AMD Genoa Zen 4 (AVX512) 236 ms 346 ms 0.68x (-46.6%) Intel Ice Lake (AVX512) 242 ms 316 ms 0.77x (-30.6%) AMD Milan Zen 3 (AVX2) 252 ms 230 ms 1.10x (+8.7%) Intel Haswell (AVX2) 369 ms 456 ms 0.81x (-23.6%) NVIDIA Grace (SVE2) 336 ms 337 ms 1.00x (-0.3%) ARM Neoverse-N1 (NEON) 1081 ms 1105 ms 0.98x (-2.2%) ====================================================================== Bug: 537818862 Change-Id: I3ddeddb040fc52a30dabe7ae2c71d16dda021568 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8343467 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-09-01 19:08:52 -07:00
Frank Barchard	2fcc1272dd	Convert8To16 takes bits instead of scale. ... - NV16/24 FOURCC and document - Rotate take uint8* for ubsan Bug: 537818862 Change-Id: I2bc7109abaffe2464e031e33c22296c007077d82 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8339427 Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-09-01 16:07:23 -07:00
Frank Barchard	af714f0810	Convert8To16 use shift to allow scale=65536 ... - Add avx512bw, neon 32 bit and RVV versions - Adapt SSE2 to psrlw instead of pmulhuw CPU Before After Improvement --------------------------------------------------------------- Granite Rapids (AVX10) 77 ms 75 ms +2.6% (1.03x) Emerald Rapids (AVX512) 75 ms 67 ms +10.7% (1.12x) Sapphire Rapids (AVX512) 91 ms 85 ms +6.6% (1.07x) AMD Venice Zen 6 (AVX512) 30 ms 30 ms 0.0% (1.00x) AMD Turin Zen 5 (AVX512) 32 ms 32 ms 0.0% (1.00x) AMD Genoa Zen 4 (AVX512) 35 ms 36 ms -2.9% (0.97x) Skylake-X (AVX512) 119 ms 119 ms 0.0% (1.00x) Ice Lake (AVX512) 93 ms 104 ms -11.8% (0.89x) AMD Milan Zen 3 (AVX2) 43 ms 44 ms -2.3% (0.98x) Haswell (AVX2) 129 ms 138 ms -7.0% (0.93x) NVIDIA Grace (SVE2) 42 ms 43 ms -2.4% (0.98x) Neoverse-N1 (NEON) 87 ms 87 ms 0.0% (1.00x) --------------------------------------------------------------- Bug: None Change-Id: Ia8f310a70c8b1229927fcdf117fe22e131d70ea9 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8330912 Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-09-01 12:21:37 -07:00
yuanhecai	60102bd795	loongarch: Fix RGBToY SIMD ... Add missing RGBToYMatrixRow_Any_{LSX,LASX} and kAddY offset fix so RAW/RGB24 to YUV works on LoongArch. Change-Id: I4c990a3705d7f521fa6d0c9e7e01f1e36f81c379 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8292090 Reviewed-by: Wan-Teh Chang <wtc@google.com> Reviewed-by: Frank Barchard <fbarchard@google.com> Commit-Queue: Wan-Teh Chang <wtc@google.com>	2026-08-31 20:57:34 -07:00
Frank Barchard	0c781f87b8	[libyuv] Keep LIBYUV_API in basic_types.h and remove legacy types and bool macros ... Reduces include/libyuv/basic_types.h to only define LIBYUV_API. Directly includes <stdint.h> and <stddef.h> where needed in public headers. Replaces LIBYUV_BOOL, LIBYUV_FALSE, and LIBYUV_TRUE with C++ bool, true, and false in C++ code (MJpegDecoder, unit tests) and int (0, 1) for C API functions. Updates documentation and unit tests. Test: ninja -C out/Release libyuv_unittest && ./out/Release/libyuv_unittest --gtest_filter=* Change-Id: I2d58e872f0f976263bcda546dcc3b1b5f7e53d58 TAG=agy CONV=2730bc56-93ff-48ed-b615-dcde1b834394 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280662 Commit-Queue: Wan-Teh Chang <wtc@google.com> Reviewed-by: Wan-Teh Chang <wtc@google.com>	2026-08-28 08:32:19 -07:00
Frank Barchard	969c96cfb4	BlendPlaneRow_AVX512BW and NEON ... CPU / Architecture Before (C) After (NEON) Improvement ---------------------------------------------------------------------- Cortex-A53 (Pixel 2 XL Little) 3248 ms 792 ms 4.10x Cortex-A73 (Pixel 2 XL Big) 432 ms 392 ms 1.10x Cortex-A55 (Pixel 7 Little) 780 ms 563 ms 1.39x Cortex-A78 (Pixel 7 Mid) 254 ms 170 ms 1.49x Cortex-X1 (Pixel 7 Big) 158 ms 129 ms 1.22x Cortex-A510 (Galaxy S22 Little) 816 ms 441 ms 1.85x Cortex-A710 (Galaxy S22 Mid) 270 ms 129 ms 2.09x Cortex-X2 (Galaxy S22 Big) 116 ms 80 ms 1.45x Cortex-A510 (Galaxy S23 Little) 723 ms 373 ms 1.94x Cortex-A715 (Galaxy S23 Mid) 183 ms 119 ms 1.54x Cortex-X3 (Galaxy S23 Big) 122 ms 72 ms 1.69x Lumix C1 Pro (PD2509 Little) 246 ms 156 ms 1.58x Lumix C1 Premium (PD2509 Mid) 211 ms 112 ms 1.88x Lumix C1 Ultra (PD2509 Big) 82 ms 50 ms 1.64x Oryon-v3M (OnePlus Mid) 233 ms 150 ms 1.55x Oryon-v3L (OnePlus Big) 103 ms 85 ms 1.21x On Icelake Function (1280x720) C SSSE3 AVX2 AVX-512BW Gain ---------------------------------------------------------------------- BlendPlane 901 ms 165 ms 135 ms 115 ms 1.17x I420Blend 1398 ms 287 ms 198 ms 187 ms 1.06x Bug: 546077674 Change-Id: I189c44cc757c0e26838a7b6fdb2db1cbe5f96f20 TAG=agy CONV=0f82c25c-616e-4bb3-beeb-981decb0f40c Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8297264 Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-08-26 17:45:34 -07:00
Frank Barchard	3793561ddc	RAWToRGB24 avx2/avx512 use permq/shufb and avx512vbmi ... CPU / Arch Before (Kernel) After (Kernel) Delta ---------------------------------------------------------------------- Haswell (x86_64) 263 ms (SSSE3) 263 ms (AVX2) 0.0% Milan (Zen 3) 115 ms (SSSE3) 138 ms (AVX2) -20.0% SapphRap (x86_64) 173 ms (SSSE3) 178 ms (AVX512VBMI) -2.9% GNR (Granite R) 191 ms (SSSE3) 189 ms (AVX512VBMI) +1.0% Turin (Zen 5) 69 ms (SSSE3) 55 ms (AVX512VBMI) +20.3% Icelake (x86_64) 204 ms (SSSE3) 186 ms (AVX512VBMI) +8.8% Cascade Lake 237 ms (SSSE3) 229 ms (AVX512BW) +3.4% Neoverse (aarch64)* 103 ms (SVE2) 103 ms (SVE2) 0.0% Bug: None Change-Id: Ib218632e16b3308a214e9ffcbc745d6c41116c6e Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8280536 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-08-25 14:37:32 -07:00
Frank Barchard	a0fa632939	Add AVX2 row functions for RGB565, ARGB1555, and ARGB4444 ... On Icelake: Was C I420ToARGB1555_Opt (1230 ms) I420ToARGB4444_Opt (1195 ms) I420ToRGB565_Opt (1141 ms) Now AVX2 I420ToARGB1555_Opt (372 ms) I420ToARGB4444_Opt (397 ms) I420ToRGB565_Opt (347 ms) I420ToRGB24Filter_Opt (567 ms) I420ToRGB565Dither_Opt (418 ms) I420ToARGBFilter_Opt (381 ms) I420ToRGBA_Opt (304 ms) I420ToBGRA_Opt (296 ms) I420ToARGB_Opt (257 ms) I420ToABGR_Opt (254 ms) I420ToAR30_Opt (215 ms) I420ToRGB24_Opt (205 ms) I420ToRAW_Opt (193 ms) Bug: None Change-Id: I36cf95d29ba85ae63c549fef7f55cb1fe0f95d8e Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8282746 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-08-24 19:41:39 -07:00
Frank Barchard	0f36c7f808	STORERGB24 macros for avx512 ... - improves I420ToRAW performance I420ToRAW on Skylake Was AVX2 I420ToRAW_Opt (262 ms) Now AVX512 I420ToRAW_Opt (212 ms) I420ToRAW on Icelake with avx512vbmi Was I420ToRAW_Opt (200 ms) Now I420ToRAW_Opt (187 ms) Bug: None Change-Id: If5b171801ba931557d379696fc128d9b155f7c44 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8267746 Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-08-18 14:18:47 -07:00
Wan-Teh Chang	d67b51303d	Revert "Add the ARGBToRGB565Row_SSE2() function back" ... This reverts commit 87c797b3c29cc6fb233fc329a93aace2942ba2d9. Reason for revert: Only commit 8aeb3a9 needs the workaround Original change's description: > Add the ARGBToRGB565Row_SSE2() function back > > Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions > back because they hide the unaligned access in ARGBToRGB565Row_C(). > > To reproduce the unalgined access error, run the following command in > Google's internal repository: > blaze test --config=asan third_party/libyuv:libyuv_test \ > --test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned > > Bug: b:547803993 > Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219 > Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046 > Commit-Queue: Wan-Teh Chang <wtc@google.com> > Reviewed-by: Frank Barchard <fbarchard@google.com> Bug: b:547803993 Change-Id: I904126f032f4ed37c470d8972152a137ca315168 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8258919 Reviewed-by: Frank Barchard <fbarchard@google.com> Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com> Commit-Queue: Wan-Teh Chang <wtc@google.com>	2026-08-17 14:43:10 -07:00
Wan-Teh Chang	87c797b3c2	Add the ARGBToRGB565Row_SSE2() function back ... Add the ARGBToRGB565Row_SSE2() and ARGBToRGB565Row_Any_SSE2() functions back because they hide the unaligned access in ARGBToRGB565Row_C(). To reproduce the unalgined access error, run the following command in Google's internal repository: blaze test --config=asan third_party/libyuv:libyuv_test \ --test_filter=LibYUVConvertTest.I420ToARGBToRGB565_Unaligned Bug: b:547803993 Change-Id: I9cc6b643bc007c100dfc092ef3ec814428789219 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8265046 Commit-Queue: Wan-Teh Chang <wtc@google.com> Reviewed-by: Frank Barchard <fbarchard@google.com>	2026-08-17 13:09:01 -07:00
Frank Barchard	c20b35a7bc	I420ToAR30 add avx512bw support ... Benchmark for 720p 1000 images CPU Architecture AVX2 (us) AVX512 (us) Speedup Change --------------------------------------------------------------------- Intel Skylake 325 229 1.42x -29.5% Intel Cascade Lake 289 213 1.36x -26.3% Intel Granite Rapids 209 168 1.24x -19.6% AMD Zen 4 168 165 1.02x -1.8% AMD Zen 5 (Turin) 180 112 1.61x -37.8% Bug: None Change-Id: I3d07c9b36196305ba7ff63973246c080ff872367 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8251940 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: Dale Curtis <dalecurtis@chromium.org>	2026-08-13 15:40:19 -07:00
Wan-Teh Chang	79d22698bc	Add static_assert to source/row_any.cc ... Bug: chromium:543078936 Change-Id: Idd25a8cd987e5295788173dce3cc2ec8c55e9bff Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8230873 Reviewed-by: Frank Barchard <fbarchard@google.com> Commit-Queue: Wan-Teh Chang <wtc@google.com>	2026-08-11 18:28:15 -07:00
Frank Barchard	eac1ea9744	I410ToI420 use 1 step HalfWidthRow function for UV ... On Icelake Was C+AVX512 I410ToI420_Opt (593 ms) I412ToI420_Opt (581 ms) Now AVX512 I410ToI420_Opt (241 ms) I412ToI420_Opt (227 ms) 60.28% HalfWidthRow_16To8_AVX512BW 32.34% Convert16To8Row_AVX512BW Samsung S25 Oryon Was C+NEON I410ToI420_Opt (141 ms) I412ToI420_Opt (127 ms) Now NEON I410ToI420_Opt (89 ms) I412ToI420_Opt (88 ms) Vivo X300 lunix c1 Was C+NEON I410ToI420_Opt (682 ms) I412ToI420_Opt (679 ms) With SVE2 (slow) I410ToI420_Opt (622 ms) I412ToI420_Opt (584 ms) 65.59% HalfWidthRow_16To8_SVE2 26.74% Convert16To8Row_SVE2 Now NEON I410ToI420_Opt (412 ms) I412ToI420_Opt (406 ms) Bug: chromium:531172025 Change-Id: I1732157f6be660d2a01086f21daf85fe7f7d166f Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8230200 Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: Wan-Teh Chang <wtc@google.com>	2026-08-10 18:19:25 -07:00
Wan-Teh Chang	c409140ff6	Correct vin/vout buffer sizes in ANY21S and ANY13 ... Tested: libyuv_unittest --gtest_filter=LibYUVConvertTest.YUY2ToNV12Overflow libyuv_unittest --gtest_filter=LibYUVPlanarTest.SplitRGBPlaneOverflow libyuv_unittest --gtest_filter=LibYUVPlanarTest.SplitARGBPlaneOverflow Bug: chromium:543078936 Change-Id: I45cc5ec912612b09ac720e3bdefa11c8ad503098 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8219514 Reviewed-by: Frank Barchard <fbarchard@google.com> Commit-Queue: Wan-Teh Chang <wtc@google.com>	2026-08-10 14:20:54 -07:00
Wan-Teh Chang	e2a1744681	Correct vin/vout buffer sizes in ANY21CT ... Tested: libyuv_unittest --gtest_filter=LibYUVConvertTest.P010ToARGBMatrixFilterOverflow libyuv_unittest --gtest_filter=LibYUVConvertTest.P210ToARGBMatrixFilterOverflow libyuv_unittest --gtest_filter=LibYUVConvertTest.P010ToAR30MatrixFilterOverflow libyuv_unittest --gtest_filter=LibYUVConvertTest.P210ToAR30MatrixFilterOverflow Bug: chromium:543078936 Change-Id: I7001c21d3ef9f7a01bf884332ea6ccbdfd7a69d0 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8229574 Commit-Queue: Wan-Teh Chang <wtc@google.com> Reviewed-by: Frank Barchard <fbarchard@google.com>	2026-08-10 13:47:24 -07:00
Wan-Teh Chang	60c4b08950	Use standard way of applying ptrdiff_t casts to n ... Remove unnecessary ptrdiff_t casts on r. Change-Id: I36cf40ea1891a9b0e1c3b1ec2f73559040f243b6 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8229575 Reviewed-by: Frank Barchard <fbarchard@google.com> Commit-Queue: Wan-Teh Chang <wtc@google.com>	2026-08-10 10:32:01 -07:00
Frank Barchard	5dc477899b	Harden I21xToI420 against oversize heights ... I21xToI420 (reached via I210ToI420 / I212ToI420): fix integer overflows in ScalePlaneVertical_16To8() by switching to HalfRow_16To8. Add a unit test covering the I21x vertical-scale path. Co-authored-by: Timothy Nikkel <tnikkel@mozilla.com> Test: libyuv_unittest --gtest_filter=*.I21xToI420NoHeightOverflow AMD Zen4 Was LibYUVConvertTest.I210ToI420_Opt (81 ms) 30.32% Convert16To8Row_AVX2 29.81% Convert16To8Row_AVX512BW 25.72% InterpolateRow_16To8_AVX2 Now LibYUVConvertTest.I210ToI420_Opt (56 ms) 41.32% Convert16To8Row_AVX512BW 40.13% HalfRow_16To8_AVX512BW Vivo X300 (ARM Lunix C1 Ultra) Was LibYUVConvertTest.I210ToI420_Opt (89 ms) 38.95% InterpolateRow_16To8_NEON 37.25% Convert16To8Row_NEON Now LibYUVConvertTest.I210ToI420_Opt (85 ms) 47.26% Convert16To8Row_SVE2 30.39% HalfRow_16To8_SVE2 Bug: chromium:531172025 Change-Id: I23e1215030c40b8577d58617098a8aeec27dd99a Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8221883 Reviewed-by: Wan-Teh Chang <wtc@google.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-08-08 12:11:18 -07:00
Frank Barchard	7138e0a2a7	Make RGBToY a 1 pass avx512bw ... - RGBToARGB use vpermi2q+vpshufb - direct RGBToY for avx512 and rvv +----------+----------+-------------+----------+---------+-----------+ | ARGBToY | ARGBToUV | ARGBToUV444 | RGBToY | RGBToUV | RGBToUV444| +----------+----------+-------------+----------+---------+-----------+ | avx512bw | avx512bw | avx512bw | avx512bw | 2 step | 2 step | | avx2 | avx2 | avx2 | 2 step | 2 step | 2 step | | ssse3 | ssse3 | ssse3 | 2 step | 2 step | 2 step | | dot | i8mm | i8mm | dot | neon | neon | | sve2 | sve2 | - | - | - | - | | sme | sme | - | - | - | - | | rvv | rvv | rvv | rvv | rvv | rvv | +----------+----------+-------------+----------+---------+-----------+ Bug: 42280902 Change-Id: I9f83bf3039431f9fb5391bd7d9eb393e186d6bcf Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8175442 Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-08-03 17:53:19 -07:00
Frank Barchard	0148582bb8	Change all 1 use wrappers to matrix calls ... Bug: 42280902 Change-Id: Ieccb1e8eeaec3064fc5d36d2f1233fd38eb2a128 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8177647 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-07-31 17:29:30 -07:00
Frank Barchard	1a746db4ce	Remove unused wrappers for 'matrix' functions ... Bug: 42280902 Change-Id: Idbc757e10d1bcf276ce00dbdd07a39f4ad01a5af Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8177664 Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-07-30 19:03:49 -07:00
Frank Barchard	86a4d94d7a	I420ToRAW and I420ToRGB24 1 pass AVX2 and AVX512VBMI ... - Implemented `I422ToRGB24Row_AVX2` and `I422ToRGB24Row_AVX512VBMI` in: - `row_gcc.cc`: Inline assembly for GCC/Clang. - `row_win.cc`: C++ intrinsics for MSVC (also verified with Clang). Reduced width alignment requirement: changed from 32-pixel to 16-pixel alignment in `convert_argb.cc` and `row_any.cc`. This allows the AVX2 path to be used for more common video resolutions. ``` I420ToRAW vs Rust on Icelake Xeon Size I420ToRAW yuv420_to_rgb iterations ------- --------- --------------- ---------- 640x480 57 us 77.377 µs/iter 3000 1280x720 170 us 221.671 µs/iter 1000 1920x1080 396 us 494.324 µs/iter 500 3840x2160 2040 us 2357 µs/iter 200 ``` Bug: 42280902 Change-Id: I07c0505c95410ea16a6218c858844791a11ef073 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7908323 Reviewed-by: Wan-Teh Chang <wtc@google.com> Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-07-20 18:15:10 -07:00
Frank Barchard	3bdb3b94ca	I420ToRAW use 2 step AVX512 ... On Icelake Was AVX2 I420ToRAW_Opt (283 ms) 67.55% I422ToARGBRow_AVX2 26.46% ARGBToRGB24Row_AVX2 Now AVX512VBMI I420ToRAW_Opt (238 ms) 73.08% I422ToARGBRow_AVX512BW 21.59% ARGBToRGB24Row_AVX512VBMI Bug: 42280902 Change-Id: I9d4d21faed30c529a5e593819f103be115709f37 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7909924 Reviewed-by: richard winterton <rrwinterton@gmail.com> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-06-08 14:32:13 -07:00
Frank Barchard	4be798d7c5	BGRAToI420 use BgraConstants for a direct conversion using AVX512BW ... row win (msvc) Was C/SSSE3 BGRAToARGB_Opt (594 ms) BGRAToARGB_Endswap_Opt (609 ms) BGRAToI420_Opt (122 ms) Now AVX2 BGRAToARGB_Opt (100 ms) BGRAToARGB_Endswap_Opt (99 ms) BGRAToI420_Opt (115 ms) Clang/GCC AVX512BW BGRAToARGB_Opt (86 ms) BGRAToARGB_Endswap_Opt (91 ms) BGRAToI420_Opt (110 ms) Bug: 42280902 Change-Id: I52cb2b0cacea8f2f0b138ec3cc521185dbef8595 Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7905821 Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: richard winterton <rrwinterton@gmail.com>	2026-06-08 12:21:47 -07:00
Frank Barchard	e14b0e2c60	RGB565ToARGB use AVX2 instead of SSE2 ... Now AVX2/AVX512 ARGB4444ToI420_Opt (204 ms) RGB565ToI420_Opt (211 ms) ARGB1555ToI420_Opt (231 ms) RAWToI420_Opt (197 ms) RGB24ToI420_Opt (197 ms) Was SSE2/AVX2 ARGB4444ToI420_Opt (276 ms) RGB565ToI420_Opt (292 ms) ARGB1555ToI420_Opt (332 ms) RAWToI420_Opt (237 ms) RGB24ToI420_Opt (232 ms) Bug: libyuv:508639302 Change-Id: I2005189d1b6af15cb5ebef1f6d66b426fa9df8eb Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7891416 Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Commit-Queue: Frank Barchard <fbarchard@google.com>	2026-06-02 18:28:02 -07:00
Frank Barchard	e449eb2172	J400ToARGB switch from SSE2 to AVX2 ... - port for row_win - remove unused HAS_ macros Was C/SSE2 MSVC J400ToARGB_Opt (1967 ms) Clang J400ToARGB_Opt (568 ms) Now AVX2 MSVC J400ToARGB_Opt (411 ms) Clang J400ToARGB_Opt (418 ms) Test: libyuv_unittest --gtest_filter=*J400ToARGB* Bug: libyuv:508639302 Change-Id: Ifdfb026832b708b61f55477250cc5ee52449f421 TAG=agy CONV=186608fc-966a-4ea7-bf57-9fe07cc1383c Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7877368 Commit-Queue: Frank Barchard <fbarchard@google.com> Reviewed-by: Justin Green <greenjustin@google.com>	2026-05-28 21:24:32 -07:00
Wan-Teh Chang	904f562d86	Remove redundant #include <stddef.h> ... "libyuv/basic_types.h" includes <stddef.h>. So it is not necessary to include both <stddef.h> and "libyuv/basic_types.h". Change-Id: I5a461258a3c6820d1007ac635838f910237f367f Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7884381 Reviewed-by: Frank Barchard <fbarchard@google.com>	2026-05-28 17:10:22 -07:00