Frank Barchard 8dabf898ee Avoid signed int overflow in scale functions using uint32_t
Use uint32_t for the 16.16 fixed-point row coordinate (uy) and max_y
in the vertical loops of ScalePlane*, ScaleUV*, and the Up2 linear
helpers. With src_height up to 32768, (src_height << 16) and y += dy
overflow signed int (e.g. for dst_height == 2 the post-read step
reaches 5/4 * (src_height << 16), which exceeds INT_MAX for any
src_height >= 26215, and for kFilterNone the final increment after
the last row reaches (src_height << 16) + dy / 2). uint32_t cannot
overflow for these ranges (uy <= 0xFFFF0000) and stays in a single
32-bit register on 32-bit targets (x86, Arm, WASM), unlike int64_t.

Undo the interim +/-32767 height cap from https://crrev.com/c/8524280
so ScalePlane, ScalePlane_16, UVScale, and UVScale_16 accept
|src_height| <= 32768 again.

ScaleUVBilinearUp2 and ScaleUVBilinearUp2_16 take ptrdiff_t strides
so 2 * dst_stride does not overflow int for images over 2 GB.

ARGBScale, ARGBScaleClip, and UVScale_16 reject |src_width| > 32768
and |src_height| > 32768, and ScalePlane, ScalePlane_16, and
ScaleSlope avoid FixedDiv(32768, 1) overflow when dst_width == 1 or
dst_height == 1.

Bug: chromium:531172025
Test: libyuv_unittest --gtest_filter=*ScalePlaneOversizeHeight*:*ScalePlaneBoxMaxSrcHeight*:*ScaleLargeImageOver2GB*:*ARGBScale*
Change-Id: Ica2e05d7dfe94f81c6631c587f1ff46c2ad44735
2026-10-13 19:09:47 -07:00
..
compare_common.cc [libyuv] Keep LIBYUV_API in basic_types.h and remove legacy types and bool macros 2026-08-28 08:32:19 -07:00
compare_gcc.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
compare_neon64.cc Fix remaining inline asm input/output constraints and clobbers 2026-10-13 11:20:48 -07:00
compare_neon.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
compare_wasm.cc [libyuv] Add WASM SIMD128 compare, rotate, scale and row functions 2026-10-06 19:05:05 -07:00
compare_win.cc Compare AVX512BW and AVX2, intrinsics for Visual C and ROWWIN 2026-10-02 15:30:05 -07:00
compare.cc [libyuv] Add WASM SIMD128 compare, rotate, scale and row functions 2026-10-06 19:05:05 -07:00
convert_argb.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
convert_from_argb.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
convert_from.cc [libyuv] Add WASM SIMD128 compare, rotate, scale and row functions 2026-10-06 19:05:05 -07:00
convert_jpeg.cc [libyuv] Keep LIBYUV_API in basic_types.h and remove legacy types and bool macros 2026-08-28 08:32:19 -07:00
convert_to_argb.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
convert_to_i420.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
convert.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
cpu_id.cc [libyuv] Add WASM SIMD128 and Relaxed SIMD for RAWToRGB24Row 2026-10-05 17:45:41 -07:00
mjpeg_decoder.cc [libyuv] Fix RISC-V CMake/GN/Bazel builds and C compiler driver linking 2026-09-28 10:59:30 -07:00
mjpeg_validate.cc [libyuv] Keep LIBYUV_API in basic_types.h and remove legacy types and bool macros 2026-08-28 08:32:19 -07:00
planar_functions.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
rotate_any.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_argb.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_common.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_gcc.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_lasx.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_lsx.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_neon64.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_neon.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_rvv.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_sme.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_sve.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_wasm.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate_win.cc [libyuv] Optimize RotatePlane_16 SIMD and compact 32-bit transposes 2026-10-12 10:57:34 -07:00
rotate.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
row_any.cc [libyuv] Replace AVX-512 _Any wrappers with k-mask tail handling 2026-10-12 15:33:19 -07:00
row_common.cc clang format and bump version 2026-09-28 10:42:45 -07:00
row_gcc.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
row_lasx.cc [libyuv] Optimize ARGBRotate for Intel SSE2, AVX2, and AVX-512 2026-10-08 16:20:46 -07:00
row_lsx.cc [libyuv] Optimize ARGBRotate for Intel SSE2, AVX2, and AVX-512 2026-10-08 16:20:46 -07:00
row_neon64.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
row_neon.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
row_rvv.cc Fix remaining inline asm input/output constraints and clobbers 2026-10-13 11:20:48 -07:00
row_sme.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
row_sve.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
row_wasm.cc [libyuv] AVX512, NEON, WASM and RVV alpha copy and mirror rows 2026-10-07 12:44:53 -07:00
row_win.cc Add 25 native k-masked AVX-512 row kernels 2026-10-12 18:14:41 -07:00
scale_any.cc [libyuv] Add WASM SIMD128 compare, rotate, scale and row functions 2026-10-06 19:05:05 -07:00
scale_argb.cc Avoid signed int overflow in scale functions using uint32_t 2026-10-13 19:09:47 -07:00
scale_common.cc Avoid signed int overflow in scale functions using uint32_t 2026-10-13 19:09:47 -07:00
scale_gcc.cc Fix remaining inline asm input/output constraints and clobbers 2026-10-13 11:20:48 -07:00
scale_lsx.cc DetilePlane and unittest for NEON 2022-01-31 20:05:55 +00:00
scale_neon64.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
scale_neon.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
scale_rgb.cc Fix Coverity Control flow issues (DEADCODE) 2026-07-01 10:39:22 -07:00
scale_rvv.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
scale_sme.cc Fix missing inline asm register clobbers 2026-10-08 14:44:23 -07:00
scale_uv.cc Avoid signed int overflow in scale functions using uint32_t 2026-10-13 19:09:47 -07:00
scale_wasm.cc [libyuv] Add WASM SIMD128 compare, rotate, scale and row functions 2026-10-06 19:05:05 -07:00
scale_win.cc BGRAToI420 use BgraConstants for a direct conversion using AVX512BW 2026-06-08 12:21:47 -07:00
scale.cc Avoid signed int overflow in scale functions using uint32_t 2026-10-13 19:09:47 -07:00
test.sh Optimze ABGRToI420 for AVX2 2020-06-04 18:24:45 +00:00
video_common.cc Lint cleanup after C99 change CL 2018-01-24 19:16:03 +00:00