avcodec/osq: Fixes several undefined overflows in do_decode()

Fixes: signed integer overflow: 1239596184 + 2119376059 cannot be represented in type 'int'
Fixes: 376136844/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_OSQ_fuzzer-6581164455821312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f511b4518fa4337f603275f865eb13ac5520d0f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer
2024-12-01 03:31:56 +01:00
parent cd163136b5
commit dfeb22ffa7

View File

@ -299,7 +299,7 @@ static int do_decode(AVCodecContext *avctx, AVFrame *frame, int decorrelate, int
dst[n] += (int)(P2 + P3) / 2 + (unsigned)p;
break;
case 8:
dst[n] += (int)(P2 + P3) / 2;
dst[n] += (int)(P2 + P3) / 2 + 0U;
break;
case 9:
dst[n] += (int)(P2 * 2 + P3) / 3 + (unsigned)p;
@ -308,13 +308,13 @@ static int do_decode(AVCodecContext *avctx, AVFrame *frame, int decorrelate, int
dst[n] += (int)(P2 + P3 * 2) / 3 + (unsigned)p;
break;
case 11:
dst[n] += (int)((unsigned)dst[A] + dst[B]) / 2;
dst[n] += (int)((unsigned)dst[A] + dst[B]) / 2 + 0U;
break;
case 12:
dst[n] += (unsigned)dst[B];
break;
case 13:
dst[n] += (int)(unsigned)(dst[D] + dst[B]) / 2;
dst[n] += (int)((unsigned)dst[D] + dst[B]) / 2 + 0U;
break;
case 14:
dst[n] += (int)((unsigned)P2 + dst[A]) / 2 + (unsigned)p;