The gcrypt definition of `bn_new` used to use the return statement
on errors, with an AVERROR return value, regardless of the signature
of the function where the macro is used - it is called in
`dh_generate_key` and `ff_dh_init` which return pointers. As a result,
compiling with gcrypt and the ffrtmpcrypt protocol resulted in an
int-conversion warning. GCC 14 may upgrade these to errors [1].
This patch fixes the problem by changing the macro to remove `AVERROR`
and instead set `bn` to null if the allocation fails. This is the
behaviour of all the other `bn_new` implementations and so the result is
already checked at all the callsites. AFAICT, this should be the only
change needed to get ffmpeg off Fedora's naughty list of projects with
warnings which may be upgraded to errors in GCC 14 [2].
[1]: https://gcc.gnu.org/pipermail/gcc/2023-May/241264.html
[2]: https://www.mail-archive.com/devel@lists.fedoraproject.org/msg196024.html
Signed-off-by: Frank Plowman <post@frankplowman.com>
Signed-off-by: Martin Storsjö <martin@martin.st>
Instead use our own struct, which we already use when using
gcrypt and gnutls.
In OpenSSL 1.1, the DH struct has been made opaque.
Signed-off-by: Martin Storsjö <martin@martin.st>
Either disabling or init'ing secure memory is required after the use
of gcry_check_version. From a look at the functions rtmpdh uses, I
noticed none require the use of secure memory, so we disable it [1][2].
This resolves some errors returned by rtmpdh code with uninitialized
gcrypt, especifically:
Fatal: failed to create the RNG lock: Invalid argument
FATAL: failed to acquire the FSM lock in libgrypt: Invalid argument
Version "1.5.4" was arbitrarily chosen. An older version probably works
as well, but I couldn't compile older versions to test on my machine.
[1]
https://gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html
[2]
https://www.gnupg.org/documentation/manuals/gcrypt/Controlling-the-library.html
Signed-off-by: Ricardo Constantino <wiiaboo@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
mpz_import and mpz_export were added in GMP 4.1, in 2002.
This simplifies the DH code by clarifying that it only uses pure
bignum functions, no other parts of nettle/hogweed.
Signed-off-by: Martin Storsjö <martin@martin.st>
There was a misunderstanding betewen bits and bytes for the parameter
value for generating random big numbers.
Signed-off-by: Martin Storsjö <martin@martin.st>
Replace mpz_random by mpz_urandomb with a random state initialization in
order to improve the randomness.
Signed-off-by: Martin Storsjö <martin@martin.st>
This adds two protocols, but one of them is an internal implementation
detail just used as an abstraction layer/generalization in the code. The
RTMPE protocol implementation uses ffrtmpcrypt:// as an alternative to the
tcp:// protocol. This allows moving most of the lower level logic out
from the higher level generic rtmp code.
Signed-off-by: Martin Storsjö <martin@martin.st>
Frank Plowman
Thomas Volkert
Rostislav Pehlivanov
Diego Biurrun
Martin Storsjö
Anton Khirnov
Ricardo Constantino
Samuel Pitoiset