Frank Barchard 9e39c1f271 ubsan overflow fix for multiply by 0x01010101
This is an UBSan error reported by libjingle

[ RUN      ] WebRtcVideoFrameTest.ConvertToYUY2BufferStride
[000:000] (videoframe.cc:375): Validate frame passed. format: I420 bpp: 12 size: 1280x720 bytes: 1382400 expected: 1382400 sample[0..3]: 73, 73, 73, 73
../../chromium/src/third_party/libyuv/source/row_gcc.cc:2903:25: runtime error: signed integer overflow: 128 * 16843009 cannot be represented in type 'int'
[8/614] WebRtcVideoFrameTest.ConvertToYUY2BufferStride returned/aborted with exit code 1 (32 ms)
[9/614] WebRtcVideoFrameTest.ConvertToYUY2BufferInverted (29 ms)
Note: Google Test filter = WebRtcVideoFrameTest.ConvertToYUY2BufferInverted

The source is uint8 and the multiply is by 0x01010101 to replicate the byte to 4 bytes.
Changing the constant to 0x01010101u should avoid overflow.

R=harryjin@google.com
TBR=harryjin@google.com
BUG=libyuv:563

Review URL: https://codereview.chromium.org/1657533005 .
2016-02-01 12:29:04 -08:00
..
basic_types.h Declare parameters that are unused, since C does not let you give a type without name. 2014-01-07 05:42:27 +00:00
compare_row.h xmmword cast for clang 2015-08-18 11:13:12 -07:00
compare.h Detect Endian of ARGB image. 2014-11-05 18:46:06 +00:00
convert_argb.h Add J444ToARGB conversion function. 2015-10-06 18:46:53 -07:00
convert_from_argb.h Change Sobel to use JPeg Luma calculation instead of extracting G channel. Using luma produces a better sobel that respects all 3 channels of RGB. Historically the G channel was used to improve performance, and because the luma of I420 is a constrained range, hurting quality. Using the JPeg variation of YUV, the luma is more accurate, including cross platform, better optimized for AVX2 and odd widths, and full range. 2015-05-27 22:32:26 +00:00
convert_from.h Dither from I420 to RGB565 in 2 steps - I420ToARGB then ARGBToRGB565. 2015-03-10 01:45:04 +00:00
convert.h rename I400 to J400 and I400 reference to I400. J400 is a simple replication of values to convert to RGB, which is what the old I400 was. I400 reference is the Y part of the YUV formula, so renaming that to I400. 2015-03-17 00:01:18 +00:00
cpu_id.h remove mips dsp detect 2015-11-03 16:57:40 -08:00
mjpeg_decoder.h fixes for blank line lint warnings 2014-08-14 19:42:48 +00:00
planar_functions.h Remove duplicate prototype for MJPGToARGB 2016-01-26 17:02:45 -08:00
rotate_argb.h C header compatible 2013-03-22 08:40:40 +00:00
rotate_row.h Fix MIPS DSPR2 build failure. 2015-12-16 10:37:42 -08:00
rotate.h use LIBYUV_BOOL instead of bool 2014-01-07 03:59:31 +00:00
row.h refactor ARGBToI422 using ARGBToI420 internally 2016-01-12 17:05:49 -08:00
scale_argb.h scale with conversion using 2 steps with unittest 2015-11-13 11:25:56 -08:00
scale_row.h Odd width variation of scale down by 2 for subsampling 2016-01-06 15:12:17 -08:00
scale.h scale16 api use LIBYUV_API 2014-09-15 22:01:43 +00:00
version.h ubsan overflow fix for multiply by 0x01010101 2016-02-01 12:29:04 -08:00
video_common.h H420 functionality 2015-09-06 11:01:40 -07:00