Frank Barchard 9e39c1f271 ubsan overflow fix for multiply by 0x01010101
This is an UBSan error reported by libjingle

[ RUN      ] WebRtcVideoFrameTest.ConvertToYUY2BufferStride
[000:000] (videoframe.cc:375): Validate frame passed. format: I420 bpp: 12 size: 1280x720 bytes: 1382400 expected: 1382400 sample[0..3]: 73, 73, 73, 73
../../chromium/src/third_party/libyuv/source/row_gcc.cc:2903:25: runtime error: signed integer overflow: 128 * 16843009 cannot be represented in type 'int'
[8/614] WebRtcVideoFrameTest.ConvertToYUY2BufferStride returned/aborted with exit code 1 (32 ms)
[9/614] WebRtcVideoFrameTest.ConvertToYUY2BufferInverted (29 ms)
Note: Google Test filter = WebRtcVideoFrameTest.ConvertToYUY2BufferInverted

The source is uint8 and the multiply is by 0x01010101 to replicate the byte to 4 bytes.
Changing the constant to 0x01010101u should avoid overflow.

R=harryjin@google.com
TBR=harryjin@google.com
BUG=libyuv:563

Review URL: https://codereview.chromium.org/1657533005 .
2016-02-01 12:29:04 -08:00
..
compare_common.cc xmmword cast for clang 2015-08-18 11:13:12 -07:00
compare_gcc.cc nolint removed 2015-08-31 10:52:13 -07:00
compare_neon64.cc xmmword cast for clang 2015-08-18 11:13:12 -07:00
compare_neon.cc xmmword cast for clang 2015-08-18 11:13:12 -07:00
compare_win.cc xmmword cast for clang 2015-08-18 11:13:12 -07:00
compare.cc xmmword cast for clang 2015-08-18 11:13:12 -07:00
convert_argb.cc rename yuv matrix constants to be more clear about what they are 2015-11-03 17:09:53 -08:00
convert_from_argb.cc refactor ARGBToI422 using ARGBToI420 internally 2016-01-12 17:05:49 -08:00
convert_from.cc Fix I420ToNV21 for wrong dst_stride_y parameter. 2016-01-14 17:38:54 -08:00
convert_jpeg.cc libyuv::MJPGToI420() and libyuv::MJPGToARGB() return failure if callback to JPeg fails. 2014-01-28 03:08:59 +00:00
convert_to_argb.cc Remove Q420 fourcc support. 2015-02-11 18:20:54 +00:00
convert_to_i420.cc Remove Q420 fourcc support. 2015-02-11 18:20:54 +00:00
convert.cc change all pix parameters to width for consistency 2015-10-07 22:30:36 -07:00
cpu_id.cc fix for gcc compile error: variable duplicate define 2015-12-08 19:03:43 -08:00
mjpeg_decoder.cc nolint removed 2015-08-31 10:52:13 -07:00
mjpeg_validate.cc validate scan EOI from end for better coverage 2015-09-14 10:58:51 -07:00
planar_functions.cc Fix memory overwrite in YUY2ToNV12 odd wdiths 2016-01-19 11:28:09 -08:00
rotate_any.cc rotate nv12 any width 2015-08-07 23:48:38 -07:00
rotate_argb.cc rotate include and proto cleanup 2015-07-22 18:09:04 -07:00
rotate_common.cc rotate include and proto cleanup 2015-07-22 18:09:04 -07:00
rotate_gcc.cc use visual c 32 bit code for clangcl 2015-08-11 10:10:45 -07:00
rotate_mips.cc rename rotate macros and functions to match 2015-07-27 17:00:41 -07:00
rotate_neon64.cc rotate include and proto cleanup 2015-07-22 18:09:04 -07:00
rotate_neon.cc remove align directives 2015-08-04 17:00:03 -07:00
rotate_win.cc use visual c 32 bit code for clangcl 2015-08-11 10:10:45 -07:00
rotate.cc Fix ifdef mismatch for mirroruv 2016-01-11 16:33:36 -08:00
row_any.cc refactor ARGBToI422 using ARGBToI420 internally 2016-01-12 17:05:49 -08:00
row_common.cc refactor ARGBToI422 using ARGBToI420 internally 2016-01-12 17:05:49 -08:00
row_gcc.cc ubsan overflow fix for multiply by 0x01010101 2016-02-01 12:29:04 -08:00
row_mips.cc remove I422ToBGRA and use I422ToRGBA internally 2015-11-02 10:24:12 -08:00
row_neon64.cc refactor ARGBToI422 using ARGBToI420 internally 2016-01-12 17:05:49 -08:00
row_neon.cc refactor ARGBToI422 using ARGBToI420 internally 2016-01-12 17:05:49 -08:00
row_win.cc refactor ARGBToI422 using ARGBToI420 internally 2016-01-12 17:05:49 -08:00
scale_any.cc Odd width variation of scale down by 2 for subsampling 2016-01-06 15:12:17 -08:00
scale_argb.cc Add rounding to InterpolateRow for improved quality and consistency. 2015-12-17 15:24:06 -08:00
scale_common.cc Odd width variation of scale down by 2 for subsampling 2016-01-06 15:12:17 -08:00
scale_gcc.cc fix for InterpolateRow_AVX2 2015-12-22 12:29:54 -08:00
scale_mips.cc remove align directives 2015-08-04 17:00:03 -07:00
scale_neon64.cc work arounds for ios 64 bit compiler where int passed into assembly needs to be explicitely cast to 'w' register. 2015-05-05 22:46:16 +00:00
scale_neon.cc remove align directives 2015-08-04 17:00:03 -07:00
scale_win.cc Remove use_sysroot=0 2016-01-11 14:57:50 -08:00
scale.cc Add rounding to InterpolateRow for improved quality and consistency. 2015-12-17 15:24:06 -08:00
video_common.cc Remove bayer format support from libyuv. This format is very rare and used on legacy hardware. Its not well optimized and has bugs related to odd widths. Removing the format will allow tests to pass under more circumstances, run faster and allow focus on higher priority quality and performance issues. 2015-02-09 19:58:19 +00:00