Consider the following sequence of events:
- open a codec without AV_CODEC_CAP_DELAY
- decode call fails with an error
- ff_thread_flush() is called
- drain packet is sent
Then the last step would make ff_thread_decode_frame() return an error,
because p->result can still be set to an error value. This is because
submit_packet returns immediately if AV_CODEC_CAP_DELAY is not set, and
no worker thread gets the chance to reset p->result, yet its value is
trusted by ff_thread_decode_frame().
Fix this by clearing the error fields on flush.
Fixes tsan warnings like this in fate-vp8-test-vector-007:
WARNING: ThreadSanitizer: data race (pid=65909)
Write of size 4 at 0x7d8c0000e088 by thread T1:
#0 vp8_decode_mb_row_sliced vp8.c:2519 (ffmpeg:x86_64+0x100995ede)
[..]
Previous write of size 4 at 0x7d8c0000e088 by thread T2:
#0 vp8_decode_mb_row_sliced vp8.c:2519 (ffmpeg:x86_64+0x100995ede)
Fixes tsan warnings like this in fate-vp8-test-vector-007:
WARNING: ThreadSanitizer: data race (pid=3590)
Write of size 4 at 0x7d8c0000e07c by thread T2:
#0 decode_mb_row_no_filter src/libavcodec/vp8.c:2330 (ffmpeg+0x000000ffb59e)
[..]
Previous write of size 4 at 0x7d8c0000e07c by thread T1:
#0 decode_mb_row_no_filter src/libavcodec/vp8.c:2330 (ffmpeg+0x000000ffb59e)
This removes the last use of the ff_put/add_pixels_clamped global
function pointers, and as such they are removed.
This patch has a negative effect on performance on MIPS, since there's
a SIMD-optimized put/add_pixels_clamped, but no xvid or jrev. From a
code maintenance point of view, that is probably acceptable.
Because the global function pointers are removed, this fixes the following
tsan warnings when running e.g. fate-dnxhr-parse:
WARNING: ThreadSanitizer: data race (pid=29917)
Write of size 8 at 0x0000025b12d8 by thread T2 (mutexes: write M1543):
#0 ff_idctdsp_init src/libavcodec/idctdsp.c:313 (ffmpeg+0x00000044b68e)
[..]
Previous write of size 8 at 0x0000025b12d8 by thread T1 (mutexes: write M1541):
#0 ff_idctdsp_init src/libavcodec/idctdsp.c:313 (ffmpeg+0x00000044b68e)
These use the mmx IDCT, but sse2 put/add_pixels_clamped implementations.
This way we don't need to use the ff_put/add_pixels_clamped function
pointers.
Instead, hardcode the use of the _arm implementation of add_pixels,
and use the C version for put_pixels (as no arm-optimized version
exists). Since there's separate implementations of idct{,_put,_add}
for neon, this has no practical impact on performance.
This tries to handle cases where separate invocations of decode_frame()
(each running in separate threads) write to respective fields in the
same AVFrame->data[]. Having per-field owners makes interaction between
readers (the referencing thread) and writers (the decoding thread)
slightly more optimal if both accesses are field-based, since they will
use the respective producer's thread objects (mutex/cond) instead of
sharing the thread objects of the first field's producer.
In practice, this fixes the following tsan-warning in fate-h264:
WARNING: ThreadSanitizer: data race (pid=21615)
Read of size 4 at 0x7d640000d9fc by thread T2 (mutexes: write M1006):
#0 ff_thread_report_progress pthread_frame.c:569 (ffmpeg:x86_64+0x100f7cf54)
[..]
Previous write of size 4 at 0x7d640000d9fc by main thread (mutexes: write M1004):
#0 update_context_from_user pthread_frame.c:335 (ffmpeg:x86_64+0x100f81abb)
Fixes the following tsan warning when running fate-vsynth_lena-ffvhuff:
WARNING: ThreadSanitizer: data race (pid=6484)
Write of size 8 at 0x7d64000154b8 by main thread (mutexes: write M1331):
#0 update_context_from_user src/libavcodec/pthread_frame.c:331 (ffmpeg+0x000000dca887)
[..]
Previous read of size 8 at 0x7d64000154b8 by thread T2 (mutexes: write M1334):
#0 draw_slice src/libavcodec/huffyuvdec.c:857 (ffmpeg+0x000000bcc86f)
Add a codec capable of decoding some formats of the RFC4175. For now
it's only capable of handling YCbCr-4:2:2 with 8-bit or 10-bit depth.
For 8-bit it's a simple pass-through, for 10-bit it depacks the stream
in the AV_PIX_FMT_YUV422P10 pixel format.
Signed-off-by: Damien Riegel <damien.riegel@savoirfairelinux.com>
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Fixes tsan warnings in fate-apng:
WARNING: ThreadSanitizer: data race (pid=51230)
Read of size 4 at 0x7d50000042fc by main thread (mutexes: write M1000):
#0 frame_copy_props frame.c:302 (ffmpeg:x86_64+0x1019a35d6)
[..]
Previous write of size 4 at 0x7d50000042fc by thread T1 (mutexes: write M997):
#0 decode_idat_chunk pngdec.c:708 (ffmpeg:x86_64+0x100f5562a)
Fixes a reported (but false) race condition in tsan for fate-apng:
WARNING: ThreadSanitizer: data race (pid=6274)
Read of size 4 at 0x7d680001ec78 by main thread (mutexes: write M1338):
#0 update_thread_context src/libavcodec/pngdec.c:1456 (ffmpeg+0x000000dacf0c)
[..]
Previous write of size 4 at 0x7d680001ec78 by thread T1 (mutexes: write M1335):
#0 decode_idat_chunk src/libavcodec/pngdec.c:737 (ffmpeg+0x000000dae951)
Values from subsequent values are guaranteed to be identical (since
poc and nal_unit_type are checked to be the same between slices), so
this doesn't affect output in any way, but does resolve the remaining
reported race conditions (by tsan) in fate-hevc.
In practice, this fixes tsan warnings like this:
WARNING: ThreadSanitizer: data race (pid=25334)
Read of size 4 at 0x7d9c0001adcc by main thread (mutexes: write M1386):
#0 hevc_update_thread_context src/libavcodec/hevcdec.c:3310 (ffmpeg+0x000000b41c7c)
[..]
Previous write of size 4 at 0x7d9c0001adcc by thread T1 (mutexes: write M1383):
#0 hls_slice_header src/libavcodec/hevcdec.c:596 (ffmpeg+0x000000b43a22)
Otherwise the thread may still be in the middle of decoding a previous
frame, which would effectively trigger a race condition on any field
concurrently read and written.
In practice, this fixes tsan warnings like the following:
WARNING: ThreadSanitizer: data race (pid=17380)
Write of size 4 at 0x7d64000160fc by main thread:
#0 update_context_from_user src/libavcodec/pthread_frame.c:335 (ffmpeg+0x000000dca515)
[..]
Previous read of size 4 at 0x7d64000160fc by thread T2 (mutexes: write M1821):
#0 ff_thread_report_progress src/libavcodec/pthread_frame.c:565 (ffmpeg+0x000000dcb08a)
Fixes tsan warnings in several audio codecs (flac, alac, wavpack, tta
and tak) that look like this:
WARNING: ThreadSanitizer: data race (pid=14340)
Read of size 4 at 0x7d64000169d8 by main thread (mutexes: write M1335):
#0 update_context_from_thread src/libavcodec/pthread_frame.c:284 (ffmpeg+0x000000dc795f)
[..]
Previous write of size 4 at 0x7d64000169d8 by thread T1 (mutexes: write M1333):
#0 wavpack_decode_block src/libavcodec/wavpack.c:1012 (ffmpeg+0x00000112b175)
This is how the ref list manager links bitstream IDs to H264Picture/Ref
objects, and is local to the producer thread. There is no need for the
consumer thread to know the bitstream IDs of its references in their
respective producer threads.
In practice, this fixes tsan warnings when running fate-h264:
WARNING: ThreadSanitizer: data race (pid=19295)
Read of size 4 at 0x7dbc0000e614 by main thread (mutexes: write M1914):
#0 ff_h264_ref_picture src/libavcodec/h264_picture.c:112 (ffmpeg+0x0000013b3709)
[..]
Previous write of size 4 at 0x7dbc0000e614 by thread T2 (mutexes: write M1917):
#0 build_def_list src/libavcodec/h264_refs.c:91 (ffmpeg+0x0000013b46cf)
The value must be identical between slices, since mbaff depends on
picture_structure and sps, both of which are checked to be identical
to the first slice before this point.
In practice, this silences some tsan warnings in fate-h264.
This fixes race conditions reported by tsan in fate-lagarith. The races
were because each thread's LagarithContext::avctx was set to the first
thread's AVCodecContext.
Otherwise all thread's private contexts have the avctx pointer set to
the AVCodecContext of the first thread, which means all writes to
ctx->avctx->* (in e.g. read_header) are effectively race conditions.
Fixes fate-dnxhd under tsan.
Fixes CID1401671
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
wm4
James Almer
Steve Lhomme
Shivraj Patil
Ronald S. Bultje
Michael Niedermayer
Rostislav Pehlivanov
Damien Riegel
Clément Bœsch
Matthieu Bouron
Martin Vignali
Jun Zhao
Diego Biurrun
Paul B Mahol
Dzung Hoang
Takayuki 'January June' Suwa
Mark Thompson