Recherche avancée

Sur d’autres sites (5401)

• FFMPEG — using 'amix' to combine short audio clip with a video results in final video's sound cutting off early

  12 avril 2022, par kilika

  I am trying to combine the following :

  


  (a) : 29s video clip that has its own audio that lasts the entire duration

  


  (b) : audio clip I want to play at the start of the video, in conjunction with original audio, and is 2 seconds long

  


  I successfully use 'amix' to obtain a video at the end with combined audio, but the problem is that the final video's audio cuts off at around 26 out of the 29 seconds of the video and goes silent.

  


  What doesn't make any sense is that the resulting video plays as it should, with the audio successfully mixed. But the output video's audio stream loses the last 3 seconds.

  


  Here's the 'amix' command I'm using (sending via subprocess) :

  


  subprocess.call(['ffmpeg','-i', input.mp4', '-i', "audioclip.mp3", '-filter_complex', 'amix', output.mp4'])


  


  I've also used versions of this command that spell out the -map "0:a" and -map "1:a", or tried using 'amix=inputs=2:duration:longest' among many other additions. All lead to the same problem : the final combined video's audio drops out with 3 seconds remaining in the video, even though the initial 'input.mp4' video has a full 29 out of 29 seconds of audio.

  


  Does anyone know why these last several seconds of audio from [a] are missing in the final video ?

  


  _________________________________________________________________

  


  edit : Below is my output when I run the amix command listed above :

  


  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'RuneBearinstakill_advanced.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.20.101
  Duration: 00:00:29.77, start: 0.000000, bitrate: 5441 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt470bg/bt470bg/smpte170m, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 5304 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : Bento4 Video Handler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : Bento4 Sound Handler
      vendor_id       : [0][0][0][0]
[mp3 @ 000001f0c8ec2040] Estimating duration from bitrate, this may be inaccurate
Input #1, mp3, from 'TTS_clip.mp3':
  Duration: 00:00:01.90, start: 0.000000, bitrate: 32 kb/s
  Stream #1:0: Audio: mp3, 24000 Hz, mono, fltp, 32 kb/s
Stream mapping:
  Stream #0:1 (aac) -> amix (graph 0)
  Stream #1:0 (mp3float) -> amix (graph 0)
  amix:default (graph 0) -> Stream #0:0 (aac)
  Stream #0:0 -> #0:1 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 000001f0c8cbe5c0] using SAR=1/1
[libx264 @ 000001f0c8cbe5c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001f0c8cbe5c0] profile High, level 4.0, 4:2:0, 8-bit
[libx264 @ 000001f0c8cbe5c0] 264 - core 164 r3094 bfc87b7 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=24 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'RuneBearinstakill_advancedwithtts.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.20.101
  Stream #0:0: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc59.25.100 aac
  Stream #0:1(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt470bg/bt470bg/smpte170m, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 15360 tbn (default)
    Metadata:
      handler_name    : Bento4 Video Handler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc59.25.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=  893 fps=110 q=-1.0 Lsize=   18717kB time=00:00:29.66 bitrate=5168.5kbits/s speed=3.66x    
video:18256kB audio:433kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.150179%
[aac @ 000001f0c8f9ebc0] Qavg: 921.259
[libx264 @ 000001f0c8cbe5c0] frame I:4     Avg QP:21.33  size: 71366
[libx264 @ 000001f0c8cbe5c0] frame P:633   Avg QP:23.32  size: 23837
[libx264 @ 000001f0c8cbe5c0] frame B:256   Avg QP:25.22  size: 12968
[libx264 @ 000001f0c8cbe5c0] consecutive B-frames: 57.2% 10.3% 10.1% 22.4%
[libx264 @ 000001f0c8cbe5c0] mb I  I16..4: 17.9% 71.4% 10.8%
[libx264 @ 000001f0c8cbe5c0] mb P  I16..4:  6.9% 17.6%  0.8%  P16..4: 43.1%  6.5%  1.5%  0.0%  0.0%    skip:23.6%
[libx264 @ 000001f0c8cbe5c0] mb B  I16..4:  1.5%  4.2%  0.3%  B16..8: 39.7%  4.6%  0.5%  direct: 1.6%  skip:47.6%  L0:55.9% L1:41.8% BI: 2.3%
[libx264 @ 000001f0c8cbe5c0] 8x8 transform intra:69.5% inter:87.3%
[libx264 @ 000001f0c8cbe5c0] coded y,uvDC,uvAC intra: 35.6% 26.8% 0.8% inter: 13.4% 10.8% 0.0%
[libx264 @ 000001f0c8cbe5c0] i16 v,h,dc,p: 21% 37% 12% 30%
[libx264 @ 000001f0c8cbe5c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 26% 21%  4%  5%  5%  6%  4%  5%
[libx264 @ 000001f0c8cbe5c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 28% 15%  5%  7%  7%  7%  5%  4%
[libx264 @ 000001f0c8cbe5c0] i8c dc,h,v,p: 67% 18% 14%  1%
[libx264 @ 000001f0c8cbe5c0] Weighted P-Frames: Y:0.2% UV:0.0%
[libx264 @ 000001f0c8cbe5c0] ref P L0: 72.3% 15.4%  8.7%  3.6%  0.0%
[libx264 @ 000001f0c8cbe5c0] ref B L0: 88.9%  9.5%  1.6%
[libx264 @ 000001f0c8cbe5c0] ref B L1: 97.7%  2.3%
[libx264 @ 000001f0c8cbe5c0] kb/s:5024.13


  


  And here is the output when I check the stream durations for the input video and the output video, showing how the output video's audio stream is somehow reduced by several seconds after the amix :

  


  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'RuneBearinstakill_advanced.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.20.101
  Duration: 00:00:29.77, start: 0.000000, bitrate: 5403 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt470bg/bt470bg/smpte170m, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 5266 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : Bento4 Video Handler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : Bento4 Sound Handler
      vendor_id       : [0][0][0][0]
[STREAM]
duration=29.766667
[/STREAM]
[STREAM]
duration=29.738000
[/STREAM]

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'RuneBearinstakill_advancedwithtts.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.20.101
  Duration: 00:00:29.77, start: 0.000000, bitrate: 5098 kb/s
  Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt470bg/bt470bg/smpte170m, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 4971 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : Bento4 Video Handler
      vendor_id       : [0][0][0][0]
[STREAM]
duration=27.477000
[/STREAM]
[STREAM]
duration=29.766667


  


• RoQ on Dreamcast

  18 mars 2011, par Multimedia Mike — Sega Dreamcast

  I have been working on that challenge to play back video on the Sega Dreamcast. To review, I asserted that the RoQ format would be a good fit for the Sega Dreamcast hardware. The goal was to play 640x480 video at 30 frames/second. Short version : I have determined that it is possible to decode such video in real time. However, I ran into certain data rate caveats.

  First off : Have you ever wondered if the Dreamcast can read an 80mm optical disc ? It can ! I discovered this when I only had 60 MB of RoQ samples to burn on a disc and a spindle full of these 210MB-capacity 80mm CD-Rs that I never have occasion to use.

  
  
  

  New RoQ Library
  There are open source RoQ decoders out there but I decided to write a new one. A few reasons : 1) RoQ is so simple that I didn’t think it would take too long ; 2) it would be nice to have a RoQ library that is license-compatible (BSD-like) with the rest of the KallistiOS distribution ; 3) the idroq.tar.gz distribution, while license-compatible, has enough issues that I didn’t want to correct it.

  Thankfully, I was correct about the task not being too difficult : I put together a new RoQ decoder in short order. I’m a bit embarrassed to admit that the part I had the most trouble with was properly converting YUV -> RGB.
  
  About the approach I took : While the original idroq.tar.gz decoder maintains YUV 4:2:0 codebooks (which led to chroma bugs during motion compensation) and FFmpeg’s decoder maintains YUV 4:4:4 codebooks, this decoder is built to convert the YUV 4:2:0 vectors into RGB565 vectors during the vector unpacking phase. Thus, the entire frame is rendered in RGB565 — no lengthy YUV -> RGB conversion after decoding — and all pixels are shuffled around as 16-bit units (minor speedup vs. shuffling everything as bytes).

  I also entertained the idea of maintaining YUYV codebooks (since the DC supports that colorspace as a texture format). But I scrapped that idea when I remembered it would lead to the same chroma bleeding problem seen in the original idroq.tar.gz decoder.

  Onto The Dreamcast
  I developed the library on a Linux computer, allowing it to output a series of PNM files for visual verification and debugging. Dropping it into a basic DC/KOS-compatible program was trivial and the first order of business was profiling.

  At first, I profiled the entire decode operation : open file, then read and decode each chunk while tossing away the results. I was roundly disappointed to see that, e.g., an 8.5-second RoQ sample needed a little more than 20 seconds to complete. Not real time. I performed a series of optimizations on the decoding library that netted notable performance gains when profiling on Linux. When I brought these same optimizations over to the DC, decoding time didn’t improve at all. This was my first suspicion that perhaps my assumptions regarding the DC’s optical drive’s data rate were not correct.

  Dreamcast Data Rate Profiling
  Let’s start with some definitions : In terms of data rate, an ’X’, i.e., 1X is the minimum data rate needed to read CD quality audio from a disc. At that speed, a drive should be able to stream 75 sectors each second. When reading mode 1/form 1 CD-ROM data, each sector has 2048 bytes (2 kbytes), so a single-speed data rate should achieve 150 kbytes/sec.

  The Dreamcast is supposed to possess a 12X optical drive. This would imply a maximum data rate of 150 kbytes/sec * 12 = 1800 kbytes/sec.

  Rigging up a trivial experiment using the RoQ samples burned on a few different CD-R discs, the best data rate I can see is about 500-525 kbytes/sec, or around 3.5X.

  Where’s the discrepancy ? My first theory has to do with the fact that not all optical media is created equal. This is why optical drives often advertise a slew of numbers which refer to the best theoretical speed for reading a CD vs. writing a CD-R vs. writing a CD-RW, etc. Perhaps the DC drive can’t read CD-Rs very quickly. To test this theory, I tried streaming a large file from a conventionally mastered CD-ROM. This worked well for the closest CD-ROM I had on hand : I was able to stream data at a rate that works out to about 6.5X.

  I smell a science project for another evening : Profiling read speeds from a mastered CD-ROM, burned CD-R, and also a mastered GD-ROM, on each of the 3 Dreamcast consoles I possess (I’ve heard that there’s variance between optical drives depending on manufacturing run).

  The Good News
  I added a little finer-grained code to profile just the video decoding functions. The good news is that the decoder meets my real time goals : That 8.5-second RoQ sample encoded at 640x480x30fps makes its way through the video decoding functions on the DC in a little less than 5 seconds. If the optical drive can supply the data fast enough, the video decoder can take care of the rest.

  The RoQ encoder included with FFmpeg does not honor any bitrate parameters. Instead, I encoded the same file at 320x240. It reportedly decoded in real time and can be streamed in real time as well.

  I say "reportedly" because I’m simply working from textual output at this point ; the next phase is to hook the decoder up to the display hardware.

• ffmpeg image concat file with different file extensions

  4 juillet 2018, par Rich

  I’ve been working with ffmpeg and I have got it to work perfectly quite a few times, but as of recent I am running into an issue where all the images from my text file arent used.

  My text file looks like -

  ffconcat version 1.0
  
  file 'gallery-house-77-west-55th-street-01.jpg'
  
  duration 4.44
  
  file 'img03.jpg'
  
  duration 4.44
  
  file 'inside.png'
  
  duration 4.44
  
  file 'placeholder.png'

  (amount of files, file names and extensions will always vary)

  And I’ve tried several combinations but as of now my ffmpeg function looks like —

  ffmpeg -i audio.mp3 -safe 0 -f concat -i paths.txt -c:a copy -c:v libx264 \
  
  -vf "scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:(ow-iw)/2:(oh-ih)/2,fps=25,format=yuv420p" -vsync vfr -movflags +faststart -y output.mp4 2>&1');

  When trying to run it, the video seems to convert but has various weird behaviors - sometimes only the middle file(image) in the list is shown or the first 2 only are shown/rendered.

  The audio is playing correctly though.

  I’ve tried setting the framerate for the input and I also tried setting it for the output but it leads to more strange behavior.

  This is what is echoed when running my above ffmpeg command -

   libavutil      56. 15.100 / 56. 15.100
  
    libavcodec     58. 19.100 / 58. 19.100
  
    libavformat    58. 13.100 / 58. 13.100
  
    libavdevice    58.  4.100 / 58.  4.100
  
    libavfilter     7. 18.100 /  7. 18.100
  
    libswscale      5.  2.100 /  5.  2.100
  
    libswresample   3.  2.100 /  3.  2.100
  
    libpostproc    55.  2.100 / 55.  2.100
  
  [mp3 @ 0xb46780] Estimating duration from bitrate, this may be inaccurate
  
  Input #0, mp3, from 'audio.mp3':
  
    Metadata:
  
      encoder         : Lavf57.71.100
  
    Duration: 00:00:19.98, start: 0.000000, bitrate: 48 kb/s
  
      Stream #0:0: Audio: mp3, 22050 Hz, mono, fltp, 48 kb/s
  
  Input #1, concat, from 'paths.txt':
  
    Duration: N/A, start: 0.000000, bitrate: N/A
  
      Stream #1:0: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 754x424 [SAR 72:72 DAR 377:212], 25 tbr, 25 tbn, 25 tbc
  
  Stream mapping:
  
    Stream #1:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
  
    Stream #0:0 -> #0:1 (copy)
  
  Press [q] to stop, [?] for help
  
  [swscaler @ 0xba25c0] deprecated pixel format used, make sure you did set range correctly
  
  [libx264 @ 0xb710c0] using SAR=3393/3392
  
  [libx264 @ 0xb710c0] using cpu capabilities: none!
  
  [libx264 @ 0xb710c0] profile High, level 3.1
  
  [libx264 @ 0xb710c0] 264 - core 155 r2901 7d0ff22 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=22 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  
  Output #0, mp4, to 'output.mp4':
  
    Metadata:
  
      encoder         : Lavf58.13.100
  
      Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 3393:3392 DAR 377:212], q=-1--1, 25 fps, 12800 tbn, 25 tbc
  
      Metadata:
  
        encoder         : Lavc58.19.100 libx264
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  
      Stream #0:1: Audio: mp3 (mp4a / 0x6134706D), 22050 Hz, mono, fltp, 48 kb/s
  
  [swscaler @ 0xb93140] deprecated pixel format used, make sure you did set range correctly
  
  [swscaler @ 0xb93140] Warning: data is not aligned! This can lead to a speed loss
  
  [mjpeg @ 0xb6e200] mjpeg: unsupported coding type (cd)
  
  [mjpeg @ 0xb6e200] mjpeg: unsupported coding type (c8)
  
  [mjpeg @ 0xb6e200] Found EOI before any SOF, ignoring
  
  [mjpeg @ 0xb6e200] mjpeg: unsupported coding type (c7)
  
  Error while decoding stream #1:0: Invalid data found when processing input
  
  [mjpeg @ 0xb6e200] invalid id 94
  
  Error while decoding stream #1:0: Invalid data found when processing input
  
  [mp4 @ 0xb6f8c0] Starting second pass: moving the moov atom to the beginning of the file
  
  frame=    1 fps=0.0 q=28.0 Lsize=     179kB time=00:00:19.95 bitrate=  73.6kbits/s speed=44.1x    
  
  video:58kB audio:117kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.446521%
  
  [libx264 @ 0xb710c0] frame I:1     Avg QP:28.86  size: 58683
  
  [libx264 @ 0xb710c0] mb I  I16..4: 21.2% 59.8% 19.0%
  
  [libx264 @ 0xb710c0] 8x8 transform intra:59.8%
  
  [libx264 @ 0xb710c0] coded y,uvDC,uvAC intra: 50.5% 65.2% 31.6%
  
  [libx264 @ 0xb710c0] i16 v,h,dc,p: 10% 60%  3% 28%
  
  [libx264 @ 0xb710c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 27% 26%  4%  4%  4%  6%  4%  7%
  
  [libx264 @ 0xb710c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 33% 10%  5%  6%  5%  8%  5%  7%
  
  [libx264 @ 0xb710c0] i8c dc,h,v,p: 47% 34% 11%  8%
  
  [libx264 @ 0xb710c0] kb/s:11736.60

  Any ideas whats causing the image sequence to fail ?

  EDIT

  After some more debugging I think it may be related to the fact that all the files dont have the same extension. How can I make it so that it doesn’t have to be that way or is that already a default setting and the issue is else where ?

  EDIT 2

  I can get it to work when all the image extensions are the same but when it gets to png or any other image extension it renders the movie but the movie stops (incomplete) when it gets to that frame. This happens when I -
  1. place all the jpgs before the png
  2. run the following command ----

  ffmpeg -safe 0 -f concat -i paths.txt -c:v libx264 \
  
  -vf "scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:(ow-iw)/2:(oh-ih)/2" -pix_fmt yuv420p -movflags +faststart -y output.mp4 2>&1

  So my last guess may be correct and it’s because I am mixing multiple image extensions in my ffmpeg command.

  How can I correct it so that it allows a mix of any image files ?