Maximilian Hils 45258a40a0 Fix int overflow in ARGBRotate pointer math
ARGBRotate90, ARGBRotate180 and ARGBRotate270 adjusted the source and
destination base pointers with expressions like src_stride_argb *
(height - 1) and dst_stride_argb * (width - 1) using 32-bit int
multiplication. For large frames this product overflows int before it
is applied as a pointer offset, yielding an out-of-bounds pointer even
on 64-bit targets (in the 90/270 cases the bad base pointer is then
passed to ARGBTranspose).

Cast the stride to ptrdiff_t before multiplying so the arithmetic is
performed in pointer width, matching the existing correct pattern at
rotate.cc:164 and rotate_argb.cc:235.

R=fbarchard@google.com

Change-Id: Ia4fd5d813de4fb7fa72c46aee723eca65980bccc
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8070980
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Commit-Queue: Max Hils <mhils@google.com>
2026-07-17 09:19:03 -07:00
..
compare_common.cc
compare_gcc.cc ARGBToJ444 use 256 for fixed point scale UV 2025-02-27 13:04:15 -08:00
compare_neon64.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
compare_neon.cc
compare_win.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
compare.cc Don't coalesce rows if width*height would overflow 2026-05-29 11:57:47 -07:00
convert_argb.cc I420ToRAW use 2 step AVX512 2026-06-08 14:32:13 -07:00
convert_from_argb.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
convert_from.cc Fix integer overflow when flipping negative height 2026-06-03 16:17:37 -07:00
convert_jpeg.cc
convert_to_argb.cc Fix Coverity Control flow issues (DEADCODE) 2026-07-01 10:39:22 -07:00
convert_to_i420.cc Fix int negation overflow in ConvertToARGB/I420 2026-06-05 12:34:38 -07:00
convert.cc I010ToNV12: dispatch Convert16To8Row on halfwidth 2026-07-01 18:38:27 -07:00
cpu_id.cc add bmm detect and vdpphps in util/cpuid 2026-06-09 14:52:48 -07:00
mjpeg_decoder.cc Fix incorrect deletion. Should use delete[], not delete 2026-07-06 14:11:24 -07:00
mjpeg_validate.cc
planar_functions.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
rotate_any.cc Deprecate MIPS and MSA support. 2025-10-16 12:20:40 -07:00
rotate_argb.cc Fix int overflow in ARGBRotate pointer math 2026-07-17 09:19:03 -07:00
rotate_common.cc Remove redundant #include <stddef.h> 2026-05-28 17:10:22 -07:00
rotate_gcc.cc Use ptrdiff_t for buffer offsets 2026-04-28 18:21:42 -07:00
rotate_lsx.cc
rotate_neon64.cc Fix integer overflow in multiplications of stride 2026-05-28 14:12:37 -07:00
rotate_neon.cc Fix integer overflow in multiplications of stride 2026-05-28 14:12:37 -07:00
rotate_sme.cc
rotate_win.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
rotate.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
row_any.cc I420ToRAW use 2 step AVX512 2026-06-08 14:32:13 -07:00
row_common.cc I420ToRAW use 2 step AVX512 2026-06-08 14:32:13 -07:00
row_gcc.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
row_lasx.cc loongarch: fix row_lsx.cc and row_lasx.cc 2026-07-01 13:06:09 -07:00
row_lsx.cc loongarch: fix row_lsx.cc and row_lasx.cc 2026-07-01 13:06:09 -07:00
row_neon64.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
row_neon.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
row_rvv.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
row_sme.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
row_sve.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
row_win.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
scale_any.cc Deprecate MIPS and MSA support. 2025-10-16 12:20:40 -07:00
scale_argb.cc Fix Coverity Control flow issues (DEADCODE) 2026-07-01 10:39:22 -07:00
scale_common.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
scale_gcc.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
scale_lsx.cc
scale_neon64.cc
scale_neon.cc
scale_rgb.cc Fix Coverity Control flow issues (DEADCODE) 2026-07-01 10:39:22 -07:00
scale_rvv.cc Replace RAWToY/RGB24ToY with RGBToYMatrix 2026-04-21 17:11:14 -07:00
scale_sme.cc
scale_uv.cc Validate int param is not INT_MIN before negating 2026-06-04 21:55:57 -07:00
scale_win.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
scale.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
test.sh
video_common.cc