This filter accepts all the dnn networks which do image processing.
Currently, frame with formats rgb24 and bgr24 are supported. Other
formats such as gray and YUV will be supported next. The dnn network
can accept data in float32 or uint8 format. And the dnn network can
change frame size.
The following is a python script to halve the value of the first
channel of the pixel. It demos how to setup and execute dnn model
with python+tensorflow. It also generates .pb file which will be
used by ffmpeg.
import tensorflow as tf
import numpy as np
import imageio
in_img = imageio.imread('in.bmp')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]
filter_data = np.array([0.5, 0, 0, 0, 1., 0, 0, 0, 1.]).reshape(1,1,3,3).astype(np.float32)
filter = tf.Variable(filter_data)
x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
y = tf.nn.conv2d(x, filter, strides=[1, 1, 1, 1], padding='VALID', name='dnn_out')
sess=tf.Session()
sess.run(tf.global_variables_initializer())
output = sess.run(y, feed_dict={x: in_data})
graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'halve_first_channel.pb', as_text=False)
output = output * 255.0
output = output.astype(np.uint8)
imageio.imsave("out.bmp", np.squeeze(output))
To do the same thing with ffmpeg:
- generate halve_first_channel.pb with the above script
- generate halve_first_channel.model with tools/python/convert.py
- try with following commands
./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.model:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=native -y out.native.png
./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.pb:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=tensorflow -y out.tf.png
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
Before this commit an av_assert0 would fail if a v4l2 device did not
support a target format.
For example,
./ffmpeg -f v4l2 -codec:v h264 -i /dev/video0 -f mpegts -
would signal an abort if /dev/video0 did not support h264.
The new behaviour is to return an AVERROR(EINVAL) error code. An
av_assert0 has been added to verify this return.
Fixes#6629
Signed-off-by: Andriy Gelman <andriy.gelman@gmail.com>
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
If the alpha decoder init failed we presented the error message from the normal
decoder. This change should prevent such mistakes.
Signed-off-by: Marton Balint <cus@passwd.hu>
Signed-off-by: James Zern <jzern@google.com>
This compared to the other suggestions is cleaner and easier to understand
keeping the condition in the if() simple.
This affects alot of fate tests.
See: [FFmpeg-devel] [PATCH 05/11] avformat/nutenc: Don't pass NULL to memcmp
See: [FFmpeg-devel] [PATCH]lavf/nutenc: Do not call memcmp() with NULL argument
Fixes: Ticket 7980
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This introduces two new AVOption options for the FTP protocol,
one named ftp-user to supply the username to be used for auth,
one named ftp-password to supply the password to be used for auth.
These are useful for when an API user does not wish to deal with
URL manipulation and percent encoding.
Setting them while also having credentials in the URL will use the
credentials from the URL. The rationale for this is that credentials
embedded in the URL are probably more specific to what the user is
trying to do than anything set by some API user.
Signed-off-by: Nicolas Frattaroli <ffmpeg@fratti.ch>
Signed-off-by: Marton Balint <cus@passwd.hu>
Allows user to set maximum number of buffered packets when
probing a codec. It was a hard-coded parameter before this commit.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
1. must enable low_power mode since just VDENC can be supported by iHD
driver right now
2. Coding option1 and extra_data are not supported by MSDK
3. IVF header will be inserted in MSDK by default, but it is not needed
for FFmpeg, so disable it.
Signed-off-by: Zhong Li <zhongli_dev@126.com>
MFXVideoENCODE_Query calls CheckVideoParamQueryLike in MSDK and
will determine whether to set param.mfx.TargetKbps to the allowed
minTargetKbps according to the bitrate_limit in extco2 buffer.
Thus q->param.ExtParam must be set before MFXVideoENCODE_Query in case
minTargetKbps is written to TargetKbps by default.
1080P AVC encoding with option "-bitrate_limit 0 -b:v 100k":
Before patch:
902 kbps
After patch:
156 kbps
Signed-off-by: Linjie Fu <linjie.fu@intel.com>
This part of the code counts the number of planes returned by the v4l2
device for each queried capture/output buffer.
When testing the GPU h264 encoder on Nvidia's Jetson Nano, this caused an
infinite loop because avbuf->buf.length included some empty buffers (i.e.
where avbuf->buf.m.planes[i].length = 0), meaning that the counter was
never incremented and break was never reached.
This is fixed in the commit by using a well defined iteration range.
Signed-off-by: Aman Gupta <aman@tmm1.net>
G2M allows large images from small input and also reallocates
multiple buffers on any resolution change.
Fixes: Timeout (22sec -> 5sec)
Fixes: 18022/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_G2M_fuzzer-5089192530411520
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: store to null pointer of type 'uint32_t' (aka 'unsigned int')
Fixes: 18021/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APTX_HD_fuzzer-5761738313564160
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 2 * 1881153568 cannot be represented in type 'int'
Fixes: 17996/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AAC_FIXED_fuzzer-5687126468853760
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 1077952576 + 1355863565 cannot be represented in type 'int'
Fixes: 16196/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5679842317565952
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Jun Zhao
Guo, Yejun
Andreas Rheinhardt
leozhang
Andriy Gelman
Lou Logan
Baptiste Coudurier
Marton Balint
Michael Niedermayer
Limin Wang
Zhong Li
Nicolas Frattaroli
Linjie Fu