Recherche avancée

Sur d’autres sites (5656)

• FFMPEG : How to avoid audio/video desync in output of crossfaded clips when input is variable frame rate video

  25 décembre 2018, par Anders Lunde

  I’m doing screen recordings of gameplay (Dota2) using my NVIDIA graphics card GeForce experience hardware recording (NVEC Encoder). This creates a variable frame rate output video. My NVIDIA settings are 60 fps 15000 kbps. I have paid a guy to make a program that generates scripts that given start/stop timepoints can extract clips from the video and merge them with crossfade. See example code below. The script works for many input recordings but fails often : The audio and video are desynchronized (usually audio delay) in many of the clips, ca 0.5 seconds. I think it fails more when frame rate dropped more during recording. He does not know how to fix the problem, and I wonder if anyone could point out if anything could be fixed in the script (example below) ?

  Processing speed is quite important (now making a 10 min ’highlight’ video takes ca 7-10 min). Solutions increasing that amount very much more is not of too big interest, unfortunately. His approach has been to work separately with audio and video and merge in the end. He already has a program to make ffmpeg code for working with different scenarios (also adding overlays, adding music, intro/outro) so it would be preferable with some easy fixes to his code and not dramatic redesigning of the logic. But if nothing else can fix the problem, a redesign in logic is ok. Using other tools than ffmpeg is also ok, but should be automatable (scripts/cli) and not increase processing times too much.

  Running the program "mediainfo" on the input video shows that framerate dropped quite low for this input video :

  Frame rate mode : Variable

  Frame rate : 60.000 FPS

  Minimum frame rate : 3.059 FPS

  Maximum frame rate : 63.739 FPS

  Full report here : https://pastebin.com/TX061Wih

  The input video can be downloaded from dropbox here (6 GB) :
  https://www.dropbox.com/s/ftwdgapazbi62pr/fullgame.mp4?dl=0

  Here the example of a script when asked to extract two clips from input video at 9:57 (41 sec length) and 15:45 (28 sec length) and crossfade merge them with a 0.5 crossfade time. There might be some code-remnants from options that are not used in this example (overlays, music, intro/outro). Using the input video above, this creates audio/video desync.

  6 commands excecuted in sequence :

  ffmpeg.exe -loglevel warning -ss 00:09:57 -i fullgame.mp4 -t 00:00:41 -filter_complex "[0:a]afade=t=out:st=40.5:d=0.5[a1]" -map "[a1]" -y out_temp_00.mp4.wav
  
  
  
  ffmpeg.exe -loglevel warning -i fullgame.mp4 -ss 00:09:57 -t 00:00:41 -an -vcodec copy -f mpegts -avoid_negative_ts make_zero -y out_temp_00.mp4.ts
  
  
  
  ffmpeg.exe -loglevel warning -ss 00:15:45 -i fullgame.mp4 -t 00:00:28 -filter_complex "[0:a]afade=t=in:st=0:d=0.5[a1]" -map "[a1]" -y out_temp_01.mp4.wav
  
  
  
  ffmpeg.exe -loglevel warning -i fullgame.mp4 -ss 00:15:45 -t 00:00:28 -an -vcodec copy -f mpegts -avoid_negative_ts make_zero -y out_temp_01.mp4.ts
  
  
  
  ffmpeg.exe -loglevel warning -i out_temp_00.mp4.wav -i out_temp_01.mp4.wav -y -filter_complex "[0:a]adelay=0|0[a0];[1:a]adelay=40500|40500[a1];[a0][a1]amix=inputs=2:dropout_transition=68.5,atrim=duration=68.5[outa0];[outa0]loudnorm[outa]" -map "[outa]" -ar 48000 -acodec aac -strict -2 fullgame_Output.mp4.aac
  
  
  
  ffmpeg.exe -loglevel warning -i out_temp_00.mp4.ts -i out_temp_01.mp4.ts -y -i fullgame_Output.mp4.aac  -filter_complex "[0:v]trim=start=0.5,setpts=PTS-STARTPTS[0c];[1:v]trim=start=0.5,setpts=PTS-STARTPTS[1c];[0:v]trim=40.5:41,setpts=PTS-STARTPTS[fo];[1:v]trim=0:0.5[fi];[fi]format=pix_fmts=yuva420p,fade=t=in:st=0:d=0.5:alpha=1[z];[fo]format=pix_fmts=yuva420p,fade=t=out:st=0:d=0.5:alpha=1[x];[z]fifo[w];[x]fifo[q];[q][w]overlay[r];[0c][r][1c]concat=n=3[outv]" -map "[outv]" -map 2:a -shortest -acodec copy -vcodec libx264 -preset ultrafast -b 15000k -aspect 1920:1080 fullgame_Output.mp4

  P.S.

  I already asked for help at an ffmpeg chat room. One guy said he knew what the problem was, but didnt know how to fix it(?) :

  [00:10] <kepstin> oh, wait, you're using -vcodec copy
  
  [00:10] <kepstin> that explains everything.
  
  [00:10] <kepstin> when you're using -vcodec copy, the start time (set with -ss) is rounded to the nearest keyframe
  
  [00:10] <kepstin> it's not exact
  
  [00:11] <kepstin> depending on the keyframe interval, this will result in possibly quite large shifts
  
  [00:11] <kepstin> (also, your commands are applying audio filters on commands with -an, which is confusing/contradictory)
  
  [00:12] <birdboy88> so the problem is that the audio temporary clips are not being extracted from the same excat timepoints?
  
  [00:13] <kepstin> birdboy88: yeah, your audio is being re-encoded to wav so it's being cut sample-accurate, but the video's not being precisely cut.
  
  [00:16] <birdboy88> kepstin: so I need to use slow seek (?) to extract video accurately? Or somehow extract audio only where there are video keyframes?
  
  [00:17] <kepstin> birdboy88: i don't know how to extract audio starting at video keyframes with ffmpeg cli. You're already doing slow seek, which doesn't help (you should move the -ss option to before the -i option to speed it up)
  
  [00:17] <kepstin> if you want accurate video cutting when saving to a file, you have to re-encode the video
  
  [00:18] <kepstin> (doing this in a single ffmpeg command means you don't have to save to a file, so you can avoid the issue)
  
  [00:18] * kepstin is off for a bit now
  
  </kepstin></kepstin></kepstin></birdboy88></kepstin></birdboy88></kepstin></kepstin></kepstin></kepstin></kepstin></kepstin>

  EDIT :
  Everything is done with the latest ffmpeg version.

  I was unable to get Gyan’s code to work. It always loses some audio (audio is either 40.5 or 27.5, so only one audio is used). This is the only one working for me (changes were adelay=40500|40500 and amix=inputs=2[a0] ;[a0]loudnorm) :

  ffmpeg -i fullgame.mp4 -filter_complex "[0]split=2[vpre][vpost];
  
  [0]asplit=2[apre][apost];
  
  [vpre]trim=start='00:09:57':duration='00:00:41',setpts=PTS-STARTPTS[vpre-t];
  
  [apre]atrim=start='00:09:57':duration='00:00:41',asetpts=PTS-STARTPTS,afade=t=out:st=40.5:d=0.5[apre-t];
  
  [vpost]trim=start='00:15:45':duration='00:00:28',setpts=PTS-STARTPTS,format=yuva420p,fade=t=in:st=0:d=0.5:alpha=1,setpts=PTS+40.5/TB[vpost-t];
  
  [apost]atrim=start='00:15:45':duration='00:00:28',asetpts=PTS-STARTPTS,afade=t=in:st=0:d=0.5,adelay=40500|40500[apost-t];
  
  [vpre-t][vpost-t]overlay[v];
  
  [apre-t][apost-t]amix=inputs=2[a0];[a0]loudnorm[a]" -map "[v]" -map "[a]" -y -c:v libx264 -preset ultrafast -b:v 15000k -aspect 1920:1080 -c:a aac fullgame_Output.mp4

  Then I tried using a similar setup but with 3 clips, but on one machine I got error : "Error while filtering : Cannot allocate memory". And my 16 GB memory machine the processing speed is 0.02x ! Any way to avoid this ? This is the code I tried :

  ffmpeg -i fullgame.mp4 -filter_complex "[0]split=3[vpre][vpost][v3];
  
  [0]asplit=3[apre][apost][a3];
  
  [vpre]trim=start=357:duration=41,setpts=PTS-STARTPTS[vpre-t];
  
  [apre]atrim=start=357:duration=41,asetpts=PTS-STARTPTS,afade=t=out:st=40.5:d=0.5[apre-t];
  
  [vpost]trim=start=795:duration=28,setpts=PTS-STARTPTS,format=yuva420p,fade=t=in:st=0:d=0.5:alpha=1,fade=t=out:st=40.5:d=0.5:alpha=1,setpts=PTS+40.5/TB[vpost-t];
  
  [apost]atrim=start=795:duration=28,asetpts=PTS-STARTPTS,afade=t=in:st=0:d=0.5,afade=t=out:st=27.5:d=0.5,adelay=40500|40500[apost-t];
  
  [v3]trim=start=95:duration=30,setpts=PTS-STARTPTS,format=yuva420p,fade=t=in:st=0:d=0.5,setpts=PTS+41Û0.5/TB[v3-t];
  
  [a3]atrim=start=95:duration=30,asetpts=PTS-STARTPTS,afade=t=in:st=0:d=0.5,adelay=68500|68500[a3-t];
  
  [vpre-t][vpost-t]overlay[v1];
  
  [v1][v3-t]overlay[v];
  
  [apre-t][apost-t][a3-t]amix=inputs=3[a0];
  
  [a0]loudnorm[a]" -map "[v]" -map "[a]" -y -c:v libx264 -preset ultrafast -b:v 15000k -aspect 1920:1080 -c:a aac fullgame_Output.mp4

• FFMpeg - add sound to video that already contain sound

  24 mars 2016, par jacky brown

  Here is what I have : input1.avi - video that contain sounds. input2.avi - video that doesn’t contain sounds. music.mp3 - audio file.

  I want to add background music(music.mp3 file) to the video.

  C :\input1.avi -i C :\music.mp3 -shortest -c:v copy -c:a copy C :\output1.avi
  then output1.avi is the same as input1 - movie with sounds but without the background music (music.mp3)

  when I try to use the other file (video without sounds) :

  C :\input2.avi -i C :\music.mp3 -shortest -c:v copy -c:a copy C :\output2.avi
  then output2.avi is the same as input2 + it have the background music.

  I tried to execute this too :

  C:\ffmpeg\bin>ffmpeg -i C:\input.avi -i C:\music.mp3 -shortest -c:v copy -filter_ complex "[1]volume=1.5[1a];[0][1a]amerge[a]" -map 0:v -map "[a]" -ac 2 C:\output1.avi

  but got the next error messsage :

  ffmpeg version N-78949-g6f5048f Copyright (c) 2000-2016 the FFmpeg developers
  
    built with gcc 5.3.0 (GCC)
  
    configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
  
  isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
  
  le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --
  
  enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-l
  
  ibilbc --enable-libmodplug --enable-libmfx --enable-libmp3lame --enable-libopenc
  
  ore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --ena
  
  ble-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable
  
  -libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --ena
  
  ble-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx
  
  264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable
  
  -lzma --enable-decklink --enable-zlib
  
    libavutil      55. 19.100 / 55. 19.100
  
    libavcodec     57. 27.101 / 57. 27.101
  
    libavformat    57. 28.100 / 57. 28.100
  
    libavdevice    57.  0.101 / 57.  0.101
  
    libavfilter     6. 39.100 /  6. 39.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  Input #0, avi, from 'C:\output1.avi':
  
    Metadata:
  
      encoder         : Lavf57.28.100
  
    Duration: 00:02:05.76, start: 0.000000, bitrate: 450 kb/s
  
      Stream #0:0: Video: mpeg4 (Simple Profile) (XVID / 0x44495658), yuv420p, 720
  
  x480 [SAR 1:1 DAR 3:2], 440 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
  
      Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 44100 Hz, stereo, s16p, 128 k
  
  b/s
  
  [mp3 @ 00000000005abc20] Skipping 0 bytes of junk at 32370.
  
  Input #1, mp3, from 'C:\music.mp3':
  
    Metadata:
  
      title           : Broadcast News Package - News Intro
  
      artist          : After Effects News Template
  
    Duration: 00:01:57.89, start: 0.025057, bitrate: 194 kb/s
  
      Stream #1:0: Audio: mp3, 44100 Hz, stereo, s16p, 192 kb/s
  
      Metadata:
  
        encoder         : Lavc56.26
  
  [Parsed_amerge_1 @ 0000000000610200] No channel layout for input 1
  
  [Parsed_amerge_1 @ 0000000000610200] No channel layout for input 2
  
  [AVFilterGraph @ 00000000005ddfe0] The following filters could not choose their
  
  formats: Parsed_amerge_1
  
  Consider inserting the (a)format filter near their input or output.
  
  Error configuring complex filters.
  
  I/O error

  So why input1 does not contain the background music ? and how can I decrease or increase the volume of music.mp3 file ?

• ANSI FATE

  24 août 2010, par Multimedia Mike — FATE Server

  The new FATE server is shaping up well. I think most of the old configurations have been migrated to the new server. I see one new compiler for x86_64– PathScale. It’s not faring particularly well at this point.

  New Tests
  As I write this, I noticed that there are now an even 700 tests, twice as many as the last time I trumpeted such a milestone. (It should be noted that the new FATE system finally breaks down the master regression suite into individual tests.) Thankfully, it’s no longer necessary to wait for me to create or edit tests (anyone with FFmpeg privileges can do this), nor is it necessary to keep up with this blog to know exactly what tests are new. Now, you can simply inspect the file history on tests/fate.mak and tests/fate2.mak (I think these 2 files are going to merge in the near future).

  Vitor, as of r24865 : “Add FATE test for ANSI/ASCII animation and TTY demuxer.” Eh ? What’s this about ? I admit I was completely removed from FFmpeg development for much of June and July so I could have missed a lot. Fortunately, I can check the file history to see which lines were added to make this test happen. And if FATE is exercising the test, you know exactly where the samples will live. Here’s this new decoder in action on the relevant sample :

  
  
  

  The file history fingers Suxen drol/Peter Ross for this handiwork. I might have guessed– the only person who is arguably more enamored with old, weird formats than even I. Now we wait for the day that YouTube has support for this format. I’m sure there are huge archives of these animations out there (and I wager that Trixter and Jason Scott know where).

  
  
  It’s an animation — it just keeps going
  

  Meanwhile, the FATE suite now encompasses a bunch of perceptual audio formats, thanks to the 1-off testing method and a few other techniques. These formats include Bink audio, WMA Pro, WMA voice, Vorbis, ATRAC1, ATRAC3, MS-GSM, AC3, E-AC3, NellyMoser, TrueSpeech, Intel Music Coder, QDM2, RealAudio Cooker, QCELP (just going down the source control log here), and others, no doubt.

  Then there’s this curious tidbit : “Add FATE test for WMV8 DRM”. The test spec is "fate-wmv8-drm: CMD = framecrc -cryptokey 137381538c84c068111902a59c5cf6c340247c39 -i $(SAMPLES)/wmv8/wmv_drm.wmv -an". I would still like to investigate FFmpeg’s cryptographic capabilities, which I suspect are moving in a direction to function as a complete SSL stack one day.

  New Platforms
  As for new platforms, the new FATE system finally allows testing on OS/2 (remember that classic ? It was “the totally cool way to run your computer”). Thanks to Dave Yeo for taking this on.

  Further, a new MIPS-based platform recently appeared on the FATE list. This one reports itself as running on 74kf CPU. Googling for this processor quickly brings up Mans’ post about the Popcorn Hour device. So, congratulations to him for getting the mundane box to serve a higher purpose. Perhaps one day, I’ll be able to do the same for that Belco Alpha-400 netbook.