libyuv/unit_test
Frank Barchard a84bf45a21 [libyuv] Fix MSan uninitialized errors in AVX512 alpha and scale rows
Clang MemorySanitizer builds define LIBYUV_ENABLE_ROWWIN and compile
row_win.cc and scale_win.cc instead of inline assembly. Two AVX-512
intrinsic patterns in those files are not modeled by LLVM MSan:

1. _mm512_mask_cvtepi32_storeu_epi8 and _mm512_mask_cvtepi64_storeu_epi8
   lower to @llvm.x86.avx512.mask.pmov.db.mem.512 and
   @llvm.x86.avx512.mask.pmov.qb.mem.512, which LLVM MSan handles as
   unknown intrinsics without unpoisoning destination shadow memory.
   Replace them in ARGBExtractAlphaRow_AVX512BW, ScaleAddCols1_AVX512BW,
   and ScaleAddCols2_AVX512BW with _mm512_cvtepi32_epi8 /
   _mm512_cvtepi64_epi8 followed by _mm_mask_storeu_epi8, which lowers
   to @llvm.masked.store and still compiles to a single vpmovdb/vpmovqb
   instruction.

2. ARGBCopyYToAlphaRow_AVX512BW loaded dst and blended the alpha channel
   using _mm512_ternarylogic_epi32, which LLVM MSan does not model
   bitwise and flags when any destination byte (such as the old alpha
   channel) is uninitialized. Replace the load + vpternlogd + store in
   both row_win.cc and row_gcc.cc with byte-masked stores
   (_mm512_mask_storeu_epi8 / vmovdqu8 with mask 0x8888888888888888)
   that write only the alpha bytes without reading dst.

Expand ARGBCopyAlpha, ARGBExtractAlpha, and ARGBCopyYToAlpha unit tests
into _Any, _Unaligned, _Invert, and _Opt variants and poison destination
buffers under MSan to catch unmodeled masked stores. Document MSan
AVX-512 intrinsic guidelines in GEMINI.md.

Test: out/msan/libyuv_unittest --gtest_filter=*Alpha*:*ScalePlaneBox_Any*
Bug: 578615529
Change-Id: I54ac4a5844aca6cfe3aa3445fa04b8221c610b35
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8672001
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
2026-10-27 16:27:29 -07:00
..
testdata Deprecate MIPS and MSA support. 2025-10-16 12:20:40 -07:00
basictypes_test.cc [libyuv] Keep LIBYUV_API in basic_types.h and remove legacy types and bool macros 2026-08-28 08:32:19 -07:00
color_test.cc [libyuv] Enable LIBYUV_ENABLE_ROWWIN and MSAN_MEMSET for MSan builds 2026-09-28 10:49:58 -07:00
compare_test.cc [libyuv] Optimize SsimWindowRow with SIMD and use compare_win on x86 2026-10-20 15:00:52 -07:00
convert_argb_test.cc clang format and bump version 2026-09-28 10:42:45 -07:00
convert_test.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
cpu_test.cc [libyuv] Add Qualcomm Hexagon HVX SIMD support 2026-10-19 14:16:18 -07:00
cpu_thread_test.cc Unittests use ASSERT instead of EXPECT 2026-05-11 19:10:47 -07:00
math_test.cc [libyuv] Keep LIBYUV_API in basic_types.h and remove legacy types and bool macros 2026-08-28 08:32:19 -07:00
planar_test.cc [libyuv] Fix MSan uninitialized errors in AVX512 alpha and scale rows 2026-10-27 16:27:29 -07:00
rotate_argb_test.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_test.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
scale_argb_test.cc Avoid signed int overflow in scale functions using uint32_t 2026-10-14 23:16:54 -07:00
scale_plane_test.cc [libyuv] Fix MSan uninitialized errors in AVX512 alpha and scale rows 2026-10-27 16:27:29 -07:00
scale_rgb_test.cc Unittests use ASSERT instead of EXPECT 2026-05-11 19:10:47 -07:00
scale_test.cc convert, scale: Add P010/P210/P410 convert paths and NV16/NV24/Px10 scale 2026-09-23 20:43:22 -07:00
scale_uv_test.cc [libyuv] Multi-platform ScaleUVAddCols3 for ScaleUVBox at 3x 2026-10-19 15:56:38 -07:00
unit_test.cc [libyuv] Add Qualcomm Hexagon HVX SIMD support 2026-10-19 14:16:18 -07:00
unit_test.h [libyuv] Enable LIBYUV_ENABLE_ROWWIN and MSAN_MEMSET for MSan builds 2026-09-28 10:49:58 -07:00
video_common_test.cc convert, scale: Add P010/P210/P410 convert paths and NV16/NV24/Px10 scale 2026-09-23 20:43:22 -07:00