[libyuv] Fix potential UV coalescing overflow in NV12ToI420

Adds a safety check to prevent signed integer overflow in the UV
plane coalescing logic within NV12ToI420. This ensures that
halfwidth * halfheight does not overflow INT_MAX, matching the Y
plane coalescing check and preventing potential undefined behavior
(signed integer overflow) which could lead to negative widths being
passed to SIMD functions.

Test: libyuv_unittest --gtest_filter=*NV12Crop*
Bug: None

CONV=6401df25-4d5d-4595-a231-f72c2c8e78df
TAG=agy
R=wtc@google.com

Change-Id: I15a51609a1e000a82f4b6958b4ada444efb1f2f4
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7886824
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
This commit is contained in:
Frank Barchard 2026-05-29 19:42:33 -07:00 committed by libyuv-scoped@luci-project-accounts.iam.gserviceaccount.com
parent d2c6dd5e6a
commit 957f295ea9

View File

@ -1448,7 +1448,8 @@ int NV12ToI420(const uint8_t* src_y,
}
// Coalesce rows.
if (src_stride_uv == halfwidth * 2 && dst_stride_u == halfwidth &&
dst_stride_v == halfwidth) {
dst_stride_v == halfwidth &&
(ptrdiff_t)halfwidth * halfheight <= INT_MAX) {
halfwidth *= halfheight;
halfheight = 1;
src_stride_uv = dst_stride_u = dst_stride_v = 0;