Sur d’autres sites (8724)

• ffmpeg stream chrome kiosk mode ubuntu 16.04 server

  21 décembre 2016, par Raul

  I have a weird out-of-sync issue while using ffmpeg to stream to youtube live a chrome browser from an ub untu 16.04 server.

  Issue : output video streamed to youtube has audio/video out of sync, sometimes with as much as 3s

  Current flow :

  1) start pulseaudio - we using something like this to start it :

  pulseaudio --start -vvv --disallow-exit --log-target=syslog --high-priority --exit-idle-time=-1 --daemonize

  2) start Xvfb

  Xvfb :0 -ac -screen 0 1920x1080x24

  3) start chrome linux in kiosk mode

  google-chrome --kiosk --disable-gpu --incognito --no-first-run --disable-java --disable-plugins --disable-translate --disk-cache-size=$((1024 * 1024)) --disk-cache-dir=/tmp/chrome/ --user-data-dir=/tmp/chrome/ --force-device-scale-factor=1 --window-size=1920,1080 --window-position=0,0 LOCATION_URL

  4) start ffmpeg

  ffmpeg -y \
  
    -thread_queue_size 8192 -rtbufsize 250M -f x11grab -video_size 1920x1080 -framerate 24 -i :0 \
  
    -thread_queue_size 8192 -channel_layout stereo -f alsa -i pulse \
  
    -c:v libx264 -pix_fmt yuv420p -c:v libx264 -g 48 -crf 24 -filter:v fps=24 -preset ultrafast -tune zerolatency \
  
    -c:a aac -strict -2 -channel_layout stereo -ab 96k -ac 2 -flags +global_header \
  
    -f flv YOUTUBE_LIVE_STREAMING_RTMP

  Note : this is running on an amazon ec2 instance, meaning there is no soundcard, so alsa and pulseaudio are creating a dummy audio card. However, the latency does not come from there. Logs :

  Nov 25 06:14:22 ip-172-31-29-8 pulseaudio[26602]: [pulseaudio] protocol-native.c: Adjust latency mode enabled, configuring sink latency to half of overall latency.
  
  Nov 25 06:14:22 ip-172-31-29-8 pulseaudio[26602]: [pulseaudio] protocol-native.c: Requested latency=23.22 ms, Received latency=23.22 ms
  
  Nov 25 06:14:22 ip-172-31-29-8 pulseaudio[26602]: [pulseaudio] protocol-native.c: Final latency 69.66 ms = 23.22 ms + 2*11.61 ms + 23.22 ms

  At this point, here’s what we observed :

  1. if we start ffmpeg exactly after issuing the command to start chrome, we see the DTS errors from ffmpeg. Audio is out of sync with the video and has delay of 3-5seconds AHEAD. We also noticed the out of sync remains the same for the full duration of the stream

  2. if we start ffmpeg after around 10seconds, audio and video are almost in sync. We then manually added a -itsoffset -0.125 to the ffmpeg command and everything is perfect.

  Questions :

  1. Why would ffmpeg have so much lag if it’s started right after chrome ?
  2. Is starting the ffmpeg after 10s or X seconds the expected behavior ? That is, is this because the system needs to wait for audio/video signals to be "ready" or something ?
  3. Is there a way to 100% calculate or know when Chrome is fully ready and start ffmpeg ? We found sometimes it takes 5s, sometimes 10. Depends on the URL we load.
  4. Besides the DTS error that ffmpeg throws, is there any other way to know if audio/video is out-of-sync ? as sometimes we have a delay of between 0.5 to 1s, but ffmpeg does not report anything. And a restart is required to "re-balance" the audio/video inputs and get them back in sync.
  5. Can pulseaudio be the problem in this scenario ?

  Thank you

  UPDATE Dec 20

  We were able to do some tricks to force chrome to start the audio on page load, and that will force connect to pulseaudio. Doing that, plus adding a 3s delay for ffmpeg to start, there is no more delay when ffmpeg starts.
  However, our app is a webRTC app, and we have a STRANGER thing happening : if we start the page with no webcam/audio, once the webcam/audio is enabled, ffmpeg (while showing no errors) has a delay of 2s or so. While keep talking, in about max 30s, that delay is GONE.

  So the new questions are :

  1. Besides the DTS error that ffmpeg throws, is there any other way to know if audio/video is out-of-sync ? as sometimes we have a delay of between 0.5 to 1s, but ffmpeg does not report anything.
  2. What could cause the initial audio/video out of sync issue and then catching up ?

• FFMPEG encode audio and forced subtitles at same time ?

  8 janvier 2017, par Nick Bell

  I’m using latest static build of ffmpeg windows.

  My input file (.mkv) is :

  [video] - 1080, V_MPEG4/ISO/AVC, 14.6 Mbps, ID#0
  
  [audio] - DTS 5.1, 1510 Kbps, ID#1
  
  [subtitles] - S_TEXT/ASS Lossless English, ID#14

  My problem is this : I convert the audio, so that my target player, a XB1 console (media support faq), is able to play audio/video. However sometimes its rather difficult to hear or parts may be in foreign language, so I want to force the english subtitles into the mix at the same time I convert the audio.

  Currently for the audio, I use the following command

  ffmpeg -i input.mkv -codec copy -acodec ac3 output.mkv

  Can I somehow tie in the forced subtitles (onto the video) in order to save an extra process of taking the output.mkv and trying to force subtitles on ?

  Edit : I’ve tried using the following command to extract subtitles to be able to edit them

  ffmpeg -i Movie.mkv -map 0:s:14 subs.srt

  However i get the error : Stream map '0:s:14' matches no streams

  Edit2 : attempted to extract subtitles and succeeded with

  ffmpeg -i input.mkv -map 0:14 -c copy subtitles.ass

  but still looking to force the subtitles, nonetheless !

  Also - a little bonus to this question - can I somehow extract the .ass file and edit it to only produce subtitles for foreign parts - so english audio doesn’t have subtitles during the movie but foreign audio does have subtitles ?

  Cheers

  Edit3 :

  When I try to use both of the commands at once (my earlier mentioned audio converter & one from the ffmpeg wiki)

  ffmpeg -i input.mkv -codec copy -acodec ac3 -vf "ass=subs.ass" output.mkv

  I get the following error from ffmpeg,

  Filtergraph 'ass=subs.ass' was defined for video output stream 0:0 but codec copy was selected.
  
  Filtering and streamcopy cannot be used together.

• FFmpeg api : combine Camera Stream and Screen Capture or Video File stream to one stream (C/C++)

  31 décembre 2016, par lostin2010

  I have one big question which spent me 2 total days to solve , but fail .

  I want to combine Camera Stream with another stream (.flv,.mpg) to one stream . Just like the picture below. camera is a part of the Live , background is another stream.

  

  My camera device is

  [dshow @ 000373e0]  "TTQ HD Camera"
  
  [dshow @ 000373e0]     Alternative name "@device_pnp_\\?\usb#vid_114d&pid_8455&mi_00#6&1e9bcf33&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global"

  I decode my Camera Stream , its format is YUYV422, and decode another flv file its format is ’YUV420p’.
  I use each decoder of oneself to build its own avfilter , camera is in0, flv file is in1 . and use this filter_spec

  color=c=black@1:s=1920x1080[x0];[in0]null[ine0];[ine0]scale=w=960:h=540[inn0];[x0][inn0]overlay=1920*0/2:1080*0/2[x1];[in1]null[ine1];[ine1]scale=w=1160:h=740[inn1];[x1][inn1]overlay=1920*1/2:1080*0/2[x2];[x2]null[out]

  i build a filter_graph.then I read packet out separately and add_frame to filter.

  for (i = 0; i < video_num; i++)//i=0 camera packet , i=1 flv file packet 
  
  {
  
      while ((read_frame_done = av_read_frame(ifmt_ctx[i], &packet)) >= 0)
  
      {
  
         ret = av_buffersrc_add_frame(filter_ctx[stream_index].buffersrc_ctx[i],     frame[i]);
  
      }
  
  }

  then i get frame out into picref

  while (1) {
  
      ret = av_buffersink_get_frame_flags(filter_ctx[stream_index].buffersink_ctx, picref, 0);
  
  }

  I encode picref or display it with SDL , I find there is only the flv stream , no camera stream on showing. i don’t know why.
  but if I change the source stream from camera stream to another flv file, means two flv file as source streams, then it is correct like demo picture above. this confuses me a lot .
  who can help me, I will really thank you.