Recherche avancée

Sur d’autres sites (4682)

• Using FFMPEG for Bidirectional Voice Communication with Symmetric RTP

  3 février 2024, par Batuhan Öksüz

  I am trying to make a voice communication between two peers. The first peer is my local machine which is behind a symmetric NAT. The second peer is my server running on an AWS EC2 device which has a public IP address. I want to use FFMPEG for sending the audio stream through RTP while at the same time listening to a known port to receive the audio stream the remote peer sends to my device. In order to not deal with NAT traversal issues, I want to be able to use the same IP address and port number I use for sending on my local device for receiving. Is this plausible ? I'm starting to think that this is not possible and here is my rationale :

  


  

  • If my understanding is correct UDP doesn't allow full-duplex communication ; that is, one cannot use one IP:port pair for both sending and receiving data packets at the same time. Is that correct ?

  


  • If one wants to bind a socket that is already been bound, the function throws a bind error saying "Address already in use.". I figured there is a UDP option to allow binding to the same port or address even if they are in use. These options are namely SO_REUSEADDR and SO_REUSEPORT. However, this page states that it is only possible if the port is in TIME_WAIT state. So this also supports my suspicion.

  


  


  On the other hand, there is symmetric RTP that clearly states a device can receive RTP streams from an address and port that it simultaneously uses to transmit RTP streams from. With exact words of the RFC :

  


  


  A device supports symmetric RTP if it selects, communicates, and uses
IP addresses and port numbers such that, when receiving a
bidirectional RTP media stream on UDP port "A" and IP address "a", it
also transmits RTP media for that stream from the same source UDP
port "A" and IP address "a". That is, it uses the same UDP port to
transmit and receive one RTP stream.

  


  


  


  A device that doesn't support symmetric RTP would transmit RTP from a
different port, or from a different IP address, than the port and IP
address used to receive RTP for that bidirectional media steam.

  


  


  So this is where I get confused. Is symmetric RTP somehow works around the limitations of UDP ? How am I getting this wrong ?

  


  Now going back to FFMPEG and the use of symmetric RTP, I see there is an rtp option we can use to set it up, the so called localrtpport=n. I can find almost no explanation to what it does and how it's useful though. Can anyone help me with that ? As far as I can tell, this option tells FFMPEG to use port "n" as the outbound port when transmitting an RTP stream. So if the receiver transmitted its stream to this port then the problem of symmetric NAT requirement would be resolved. Or so I thought...

  


  To draw you a better picture, here are my FFMPEG commands (I'm trying everything in my local host in these commands) :

  


  # My Mac with en0 IP of 192.168.1.64
ffmpeg \
-hide_banner \
-re \
-fflags +genpts \
-f lavfi -i aevalsrc="sin(400*2*PI*t)" \
-ar 8000 \
-f mulaw \
-f rtp -reuse 1 "rtp://192.168.1.72:9193?reuse=1&localrtpport=16386&localrtcpport=16387" \
-protocol_whitelist udp,rtp \
-i rtp://192.168.1.64:16386 \
audio_signal_with_symmetric_rtp.mp3


  


  Here I am simply generating a fixed sound signal and outputting it in mulaw format through rtp. I am using the 'localrtport' option to set my outbound port and I am expecting to receive the remote peer's stream on the same port. This command starts running and and waits for the incoming stream. As soon as I start transmitting the stream from my Raspberry Pi which is on the same wireless network, I get the dreaded "Address already in use." error and the process terminates. Here is the command I use on the Raspberry :

  


  # My Raspberry Pi with wlan0 IP of 192.168.1.72
ffmpeg \
-hide_banner \
-re \
-f lavfi -i aevalsrc="sin(400*2*PI*t)" \
-ar 8000 \
-f mulaw \
-f rtp "rtp://192.168.1.64:16386?reuse=1&localrtpport=16386"


  


  TLDR

  


  The short form of the question comes down to this : How can I make use of symmetric RTP with FFMPEG, receive a stream from the same port I am actively transmitting another stream from ? Is what I'm asking impossible ? Should I go for an alternate route and try to set up a TURN server for my system ? Any help would be appreciated.

  


• when ffpmeg drops frames some things aren't played back in real time

  8 février 2024, par Alex028502

  I am trying to run a bunch of ffpmeg processes that act as simulators for cameras, and something funny is happening when I the processor can't keep up with the configured frame rate.

  


  I have replaced the rtsp stream with an output file, and managed to reproduce the issue, so will just show that to keep it simple.

  


  First here is a makefile that creates my source movie :

  


  clock.mp4: Makefile
    rm -f $@
    ffmpeg -f lavfi -i color=c=black:s=4096x2160:r=25 -vf \
"drawtext=fontfile=/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf:fontsize=72:fontcolor=white:x=(w-text_w)/2:y=(h-text_h)/2: \
text='%{eif\:trunc(n/25)\:d}':start_number=0:rate=25" \
-t 60 -r 25 $@


  


  that gives me a one minute long movie that prints the second number to the screen. I have tested it out and the seconds are close enough. I put a lot of pixels to make it easier to jam up my CPU.

  


  Here is the script that creates a processes similar to the one I am trying to debug (called experiment.sh)

  


  I am actually using H.264, but H.265 is easier to overwhelm the processor with

  


  #! /usr/bin/env bash

set -e

echo starting > message$1.txt

rm -f superclock$1.mp4
ffmpeg -re -stream_loop -1 -i clock.mp4 \
       -an -vcodec libx265 -preset ultrafast -sc_threshold -1 -x265-params repeat-headers=1 \
       -vf "drawtext=fontfile=/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf:fontsize=24:fontcolor=white:x=10:y=10:text='%{localtime\:%X}', \
            drawtext=fontfile=/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf:fontsize=24:fontcolor=white:x=10:y=(h-text_h-10):textfile=message$1.txt:reload=1, \
            scale=1920x1080,fps=25" \
       -b:v 3M -minrate 3M -maxrate 3M \
       -bufsize 6M -g 25 superclock$1.mp4 &
pid=$!

for x in $(seq 0 10)
do
    echo $x > message$1.txt
    sleep 10
done

kill -INT $pid || true


  


  It should

  


  

  • put the second in the middle of the screen - 'cause it gets it from the source video

  


  • put the approximate sixth of minute in the lower left corner
(only approximate because of the sleep but close enough)

  


  • put the the wall clock time in the upper left corner

  


  


  and it works

  


  make clock.mpg
./experiment.sh 0
vlc superclock0.mp4


  


  shows something like this


  


  Now here is the interesting part

  


  If I run the script in four different terminals at the same time

  


  ./experiment.sh 1
./experiment.sh 2
./experiment.sh 3
./experiment.sh 4


  


  It can't keep up with the frame rate, and I see this in the output :

  


  frame= 1515 fps= 16 q=0.0 size=     768kB time=00:00:59.96 bitrate= 104.9k


  


  I was hoping the end result would all look ok when I watch it except with fewer frames, but the timestamps of the frames would make it all work as expected

  


  However...

  


  

  • The time in the middle, that is inherited from the source video, the seconds in the middle of the screen, stays in sync with VLC's clock.

  


  • the wall clock in the upper left seems play at 150% speed

  


  • the every ten seconds incrementor in the lower left seems to increment every 7 seconds

  


  • the video is only 1:25 long even though it was recording for at least 1:40 according to sleeps

  


  • the wall clock in the upper right hand corner makes it more than 1'40" and then counter in the lower left makes it to 10.

  


  


  

  


  Here are four states to compare

  


  |                  | Start    | 30" in   | end      |
|------------------+----------+----------|----------|
| Video Time       | 00:00    | 00:30    | 01:24    |
| Wall Clock Time  | 15:05:50 | 15:06:39 | 15:07:39 |
| sixth of minute  | 0        | 4        | 10       |
| seconds counter  | 0        | 30       | 24       |


  


  

  


  So you can see the vlc clock keeps pace with the original clock from the source movie.. even when it is only able to produce frames at 2/3 of the rate. However, the it is taking 50% long to get through the whole source movie I guess ?

  


  I am having trouble coming up with a theory that can explain exactly how this happens.

  


  Does anybody know how I can "correct" this ? (make it so that the movie is played at the same rate that it is recorded)

  


  I am thinking of using a lower frame rate and size as the input video.. but it would be nice to have something that will always work as expected, just with a lower frame rate, no matter how busy the processor is.

  


• Merge commit '2bbb5abd877104fa9bc342c521bb49bc1aad50ce'

  20 mai 2017, par Clément Bœsch

  Merge commit '2bbb5abd877104fa9bc342c521bb49bc1aad50ce'
  * commit '2bbb5abd877104fa9bc342c521bb49bc1aad50ce' :
  
    build : Map -Wall compiler flag to -W3 for MSVC and -Wextra to -W4
  Merged-by : Clément Bœsch <u@pkh.me>
  

  • [DH] configure