1. Fix multi-bit TestCpuFlag(A | B) checks in AYUVToNV12, AYUVToNV21,
RAWToRGBA, NV12ToRGB24Matrix, NV21ToRGB24Matrix, MergeRGBPlane, and
MergeXR30Plane to compare == (A | B) so AVX-512 VBMI/BW kernels are
not dispatched when AVX512VBMI or AVX512VL is absent.
2. Fix 2-byte SVE2 heap buffer overflow on odd widths in
I422ToARGB1555Row_SVE_SC and I422ToARGB4444Row_SVE_SC using
trn1/trn2 + st2b under .b predicates.
3. Fix ARGBPolynomialRow_SSE2 to use packssdw before packuswb, and add
missing vzeroupper to SplitRGBRow_AVX2.
4. Fix CalcFrameSsim to return 0.0 instead of 0.0 / 0 (NaN) when
width <= 8 || height <= 8 so I420Ssim stays finite on small frames.
5. Fix negative-height vertical inversion in ConvertFromI420,
SplitARGBPlane (when dst_a is NULL), I422Rotate, I010Rotate, and
I210Rotate.
6. Fix horizontal flip (negative x/dx) in ScaleFilterCols_NEON (signed
sbfx #16, #16 offset extraction) and ScaleAddCols1_16_C (shift
x >>= 16 before indexing src_ptr).
7. Fix ConvertToARGB odd crop_x chroma alignment in NV12/NV21, src_u
offset in I420/YV12/J420/H420/U420, and FOURCC_U422 calling
U422ToARGB instead of H422ToARGB.
8. Remove dead 14-instruction block in ScaleRowUp2_Bilinear_16_SSE2.
9. Fix UVScale_16 for src_width == 1 (P010ToP410/P210ToP410), enable
and fix column order in TransposeWx4_32_SVE2, fix ByteToFloatRow
NEON Any BPP (3 -> 4), fix AB64ToARGBRow_Any_*, ScaleColsUp2_SSE2,
ScaleUVRowDown2Box_SME, and UYVYToARGBRow_SME macro guards, fix SVE2
loop tail branches (b.lt/b.ge), and remove duplicate HAS_* macros.
10. Fix ConvertFP16ToFP32Column_NEON loop condition and unsigned b.lo
underflow on widths < 8 or not multiples of 8.
11. Fix UYVYToNV12 IS_ALIGNED(awidth, ...) checks and WASM SIMD
IS_ALIGNED(width, 16) checks in I420ToRGB24Matrix /
I422ToRGB24Matrix.
12. Wire SplitUVRow_SVE2, SplitUVRow_LSX, and SplitUVRow_LASX in
NV24ToARGB.
13. Fix MJpegDecoder heap buffer overflows in LoadFrame (reallocate
databuf_[i] when scanlines_size changes), DecodeToBuffers, and
DecodeToCallback (clamp scanlines_to_copy to lines_left and align
skip to max_v_samp_factor).
14. Replace width & (vl - 1) tail calculations with post-loop remainder
across 18 SVE2 functions in row_sve.h for non-power-of-2 SVE VL,
and remove duplicate label 2: in UYVYToARGBRow_SVE_SC.
15. Handle width & 15 tail pixels in DetileToYUY2_C and validate
IS_POWEROFTWO(tile_height) in DetileSplitUVPlane and DetileToYUY2.
16. Avoid signed 32-bit overflow in DivideRow_16_C and C16TO8, fix
ClampFloatToZero_SSE2 pointer increment (lea 0x4) and loop counter
(sub $0x1), and initialize xmm3 in ARGBCopyYToAlphaRow_SSE2.
17. Check max leaf 7 subleaf and max extended CPUID leaf in GetCpuFlags
before querying subleaf 7:1 and extended leaf 0x80000021.
18. Free plane_u on error in I422ToNV21, and fix 4x/2x scratch row
over-allocation in ScaleARGBBilinearDown and ScaleUVBilinearDown.
19. Add ARGBSetRow_RVV and ARGBShuffleRow_RVV in row_rvv.cc.
20. Fix NEON mask (15 -> 7) in ScaleUVRowUp2_Linear_Any_NEON /
_16_Any_NEON, use SIMD InterpolateRow + SplitUVRow in NV16ToI420,
wire SSSE3/SSE2 in ARGBToI400Matrix / ARGBToYUY2Matrix /
ARGBToUYVYMatrix and RGB24Mirror, and split HalfFloatRow multiply
into (src * scale) * 2^-112 to avoid 50x Intel denormal stalls.
Bug: b/574714081
Test: libyuv_unittest
Change-Id: Iec3ae5c58d52bee3ef13cf9b765ac84b733aa576
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8601931
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
1275x1650 (US Letter 150 DPI, 1000 iterations) speedup:
CPU ARGBExtractAlpha ARGBCopyYToAlpha ARGBMirror RGB24Mirror
-----------------------------------------------------------------------
Intel Granite Rapids 1.00x 1.00x 1.00x 1.04x
Intel Emerald Rapids 1.01x 0.97x 1.03x 1.07x
Intel Sapphire Rapids 0.97x 1.01x 1.08x 1.26x
Intel Ice Lake 0.99x 1.02x 0.97x 1.08x
Intel Skylake-X 1.04x 1.03x 1.01x NA
Intel Haswell NA NA NA NA
AMD Turin (Zen 5) 1.01x 1.10x 1.25x 1.31x
AMD Genoa (Zen 4) 0.97x 1.04x 1.09x 1.42x
AMD Milan (Zen 3) NA NA NA NA
AMD Rome (Zen 2) NA NA NA NA
NVIDIA Grace (V2) 1.57x 2.51x NA NA
Google Axion 2 1.12x 3.17x NA NA
Ampere Siryn 1.06x 1.70x NA NA
ARM Neoverse-N1 1.49x 2.01x NA NA
Google Axion (Maple) 1.47x 2.03x NA NA
ARM Cortex-A53 1.06x 1.69x NA NA
Add AVX512BW ARGBExtractAlphaRow, ARGBCopyYToAlphaRow and
ARGBMirrorRow, and AVX512VBMI RGB24MirrorRow, in row_gcc.cc and as
intrinsics in row_win.cc. They are x64 only. The remainder is done
with a k-mask (bzhi) zero-masked load and a masked store, so they
handle any width without an Any wrapper and never access memory
outside the row.
RGB24MirrorRow_AVX512VBMI loads and stores 3 full zmm vectors per
64 pixels using 3 vpermi2b. The one output byte that comes from the
third input vector is merged with a masked vpbroadcastb. The 1 to 63
pixel tail rebases the same tables by 192 - 3 * r; indices >= 128 are
taken from the third vector with vpmovb2m and a masked vpermb.
Add ARGBCopyYToAlphaRow_NEON for Arm 32 bit (vld4 + vst4) and
AArch64 (zip + sli), and rewrite AArch64 ARGBExtractAlphaRow_NEON
with ldp + uzp2. Both prefetch source and destination, which is
2.5x faster on Neoverse-N1 and 1.2x to 1.5x faster on Cortex-A53
for 1280x720. Add WASM SIMD ARGBExtractAlphaRow and
ARGBCopyYToAlphaRow, using a swizzle table for CopyYToAlpha. Rework
the RVV versions to use unit stride loads with vnsrl and vzext.vf4.
R=rrwinterton@gmail.com
Bug: None
Change-Id: I436f27d70a3db0a2b176c91eb789a9491133cfcb
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8523531
Reviewed-by: richard winterton <rrwinterton@gmail.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
- MergeUVRow_SVE2: Interleaves U and V
- SplitUVRow_SVE2: De-interleaves UV
- InterpolateRow_SVE2: Bilinear row interpolation
- YUY2ToYRow_SVE2, YUY2ToUVRow_SVE2, UYVYToYRow_SVE2, UYVYToUVRow_SVE2
- HalfMergeUVRow_SVE2: 2:1 vertical averaging of U and V
OnePlus 15 with Qualcomm Snapdragon 8 Elite Gen 5 Oryon V3
Benchmark (1280x720 1000x) | SVE2 | NEON | Result
-------------------------------+--------+--------+------------------
TestInterpolatePlane | 119 ms | 164 ms | SVE2 is 1.38x win
TestInterpolatePlane_16 | 138 ms | 172 ms | SVE2 is 1.25x win
SplitUVPlane_Opt | 68 ms | 69 ms | SVE2 is 1.01x win
MergeUVPlane_Opt | 40 ms | 40 ms | Tied
SwapUVPlane_Opt | 72 ms | 72 ms | Tied
HalfMergeUVPlane_Opt | 50 ms | 50 ms | Tied
I420ToNV12_Opt | 25 ms | 26 ms | SVE2 is faster
I420ToNV21_Opt | 26 ms | 26 ms | Tied
YUY2ToNV12_Opt | 79 ms | 81 ms | SVE2 is faster
NV21ToNV12_Opt | 36 ms | 35 ms | Tied
Android420ToI420ToNV12_Opt | 53 ms | 53 ms | Tied
Android420ToI420ToNV21_Opt | 35 ms | 33 ms | NEON is faster
YUY2ToI420_Opt | 87 ms | 84 ms | NEON is faster
UYVYToI420_Opt | 87 ms | 83 ms | NEON is faster
YUY2ToI422_Opt | 101 ms | 97 ms | NEON is faster
UYVYToI422_Opt | 101 ms | 97 ms | NEON is faster
UYVYToNV12_Opt | 111 ms | 108 ms | NEON is faster
ARGBToNV21_Opt | 279 ms | 279 ms | Tied
ABGRToNV21_Opt | 279 ms | 280 ms | SVE2 is faster
ARGBToNV12_Opt | 285 ms | 280 ms | NEON is faster
ABGRToNV12_Opt | 284 ms | 281 ms | NEON is faster
Bug: 42280902
Change-Id: I53f633fe453f2853f7bc9fe17280e938912c4e8f
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8196068
Reviewed-by: Justin Green <greenjustin@google.com>
Commit-Queue: Frank Barchard <fbarchard@google.com>
ARGBRotate90, ARGBRotate180 and ARGBRotate270 adjusted the source and
destination base pointers with expressions like src_stride_argb *
(height - 1) and dst_stride_argb * (width - 1) using 32-bit int
multiplication. For large frames this product overflows int before it
is applied as a pointer offset, yielding an out-of-bounds pointer even
on 64-bit targets (in the 90/270 cases the bad base pointer is then
passed to ARGBTranspose).
Cast the stride to ptrdiff_t before multiplying so the arithmetic is
performed in pointer width, matching the existing correct pattern at
rotate.cc:164 and rotate_argb.cc:235.
R=fbarchard@google.com
Change-Id: Ia4fd5d813de4fb7fa72c46aee723eca65980bccc
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/8070980
Reviewed-by: Frank Barchard <fbarchard@google.com>
Reviewed-by: Wan-Teh Chang <wtc@google.com>
Commit-Queue: Max Hils <mhils@google.com>
Treat height == INT_MIN as invalid. Omit explicit height == INT_MIN
check if we disallow height < 32768.
Perform multiplications of stride in the ptrdiff_t type.
Add checks for invalid width and height to some functions.
Bug: 518806561
Change-Id: I5e39fffed7f806852a8758d4b59df919839c0a3b
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/7891415
Reviewed-by: Frank Barchard <fbarchard@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Add a streaming-SVE implementation of CopyRow using normal vector
load/store instructions.
Change-Id: Ia551413f9740a96473fa2e8a0958953be2f4b04e
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/6074374
Reviewed-by: Justin Green <greenjustin@google.com>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
* Run on SiFive internal FPGA:
MergeUVPlane_Opt(~6x vs scalar)
SplitUVPlane_Opt(~6x vs scalar)
TestCopyPlane(~8x vs scalar)
ARGBInterpolate0_Opt(~10x vs scalar)
ARGBInterpolate64_Opt(~9x vs scalar)
ARGBInterpolate168_Opt(~9x vs scalar)
ARGBInterpolate192_Opt(~8.5x vs scalar)
ARGBInterpolate255_Opt(~8x vs scalar)
Bug: libyuv:956
Change-Id: I8372341865f75f42e30371ef943d5c2e4be7b79a
Signed-off-by: Darren Hsieh <darren.hsieh@sifive.com>
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/4574186
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Optimize 20 functions in source/scale_lsx.cc file.
All test cases passed on loongarch platform.
Bug: libyuv:913
Change-Id: I85bcb3b0bfd9461bb6f93202546507352cbd624a
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/3351469
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Some processors support both MSA and MMI.
when they are enabled together, MSA will be preferd.
This patch move MSA initialization after MMI, so that
MSA can overide MMI and be setted to effective.
Change-Id: I8a52cce83ee4ec9727d47c99b287c9580329b149
Reviewed-on: https://chromium-review.googlesource.com/c/libyuv/libyuv/+/2155944
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
This was changed in 21be9122aadf7824efe3fc19b2a09ff253a688e1.
Change-Id: I6c04dc92f673557e10c231bd090ec8aa88b6bee4
Reviewed-on: https://chromium-review.googlesource.com/1146183
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Currently, libyuv supports MIPS SIMD Arch(MSA),
but libyuv does not supports MultiMedia Instruction(MMI)(such as loongson3a platform).
In order to improve performance of libyuv on loongson3a platform,
this provides optimize 98 functions with mmi.
BUG=libyuv:804
Change-Id: I8947626009efad769b3103a867363ece25d79629
Reviewed-on: https://chromium-review.googlesource.com/1122064
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>
ARGB rotation using scaling code. Previously it had forward
declarations of the low level row functions used. This CL
uses the header and hooks up Any and MSA versions of the code.
Bug: libyuv:779
Test: perf record out/Release/libyuv_unittest --gtest_filter=*ARGBRotate90_Opt --libyuv_width=640 --libyuv_height=359 --libyuv_repeat=999
Change-Id: Ifacd58b26bb17a236181a404fad589fd2543b911
Reviewed-on: https://chromium-review.googlesource.com/927530
Reviewed-by: Weiyong Yao <braveyao@chromium.org>
Bug: libyuv:765
Test: build for mips still passes
Change-Id: I99105ad3951d2210c0793e3b9241c178442fdc37
Reviewed-on: https://chromium-review.googlesource.com/826404
Reviewed-by: Weiyong Yao <braveyao@chromium.org>
Commit-Queue: Frank Barchard <fbarchard@chromium.org>
BUG=None
TEST=try bots and lint test
Change-Id: I1ab462adf2d309117862c5eb4b244a61ae202951
Reviewed-on: https://chromium-review.googlesource.com/450658
Commit-Queue: Frank Barchard <fbarchard@google.com>
Reviewed-by: Henrik Kjellander <kjellander@chromium.org>
