Recherche avancée

Sur d’autres sites (6403)

• Overlaying a transparent webm on an MP4 freezes at the start sometimes

  23 janvier 2021, par joshlat94

  I'm currently using the command
 ffmpeg -i mainvideo.mp4 -vcodec libvpx -i myoverlay.webm -filter_complex "overlay = 5:5" done.mp4
To overlay a transparent webm on top of my main mp4. But for some reason the output video is frozen for the first 1 second but the audio is completely fine.

  


  I've tried this command with other videos and it works sometimes but most of the time the first few frames seem frozen

  


  Edit.
I have two main mp4 files, broken_source.mp4 and working_source.mp4. Putting the overlay on broken_source results in the output having the first 1 second of video frozen (audio completely fine) but putting the same overlay on working_source.mp4 is perfect and has no issues

  


  Logs for putting the overlay on working_source :

  


  ffmpeg -i working_source.mp4 -vcodec libvpx -i chat.webm -filter_complex "overlay = 0:0" working_complete.mp4
ffmpeg version 4.3.1-2021-01-01-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'working_source.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.51.101
  Duration: 00:00:31.02, start: 0.000000, bitrate: 6176 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 6035 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[libvpx @ 0000016fb5074a80] v1.9.0-128-g3a38edea2
    Last message repeated 1 times
Input #1, matroska,webm, from 'chat.webm':
  Metadata:
    ENCODER         : Lavf58.51.101
  Duration: 00:00:31.00, start: 0.000000, bitrate: 449 kb/s
    Stream #1:0: Video: vp8, yuva420p(tv, progressive), 500x300, SAR 1:1 DAR 5:3, 60 fps, 60 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      alpha_mode      : 1
      ENCODER         : Lavc58.101.101 libvpx
      DURATION        : 00:00:31.000000000
[libvpx @ 0000016fb50761c0] v1.9.0-128-g3a38edea2
Stream mapping:
  Stream #0:0 (h264) -> overlay:main (graph 0)
  Stream #1:0 (libvpx) -> overlay:overlay (graph 0)
  overlay (graph 0) -> Stream #0:0 (libx264)
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libvpx @ 0000016fb50761c0] v1.9.0-128-g3a38edea2
[libx264 @ 0000016fb5072040] using SAR=1/1
[libx264 @ 0000016fb5072040] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000016fb5072040] profile High, level 4.2, 4:2:0, 8-bit
[libx264 @ 0000016fb5072040] 264 - core 161 r3027 4121277 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'working_complete.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 60 fps, 15360 tbn, 60 tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.91.100 aac
frame= 1861 fps= 44 q=-1.0 Lsize=   20013kB time=00:00:31.01 bitrate=5285.3kbits/s speed=0.729x
video:19477kB audio:486kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.244893%
[libx264 @ 0000016fb5072040] frame I:8     Avg QP:19.04  size:240534
[libx264 @ 0000016fb5072040] frame P:538   Avg QP:23.76  size: 23836
[libx264 @ 0000016fb5072040] frame B:1315  Avg QP:31.66  size:  3951
[libx264 @ 0000016fb5072040] consecutive B-frames:  3.0%  3.5% 14.8% 78.7%
[libx264 @ 0000016fb5072040] mb I  I16..4:  6.4% 47.4% 46.2%
[libx264 @ 0000016fb5072040] mb P  I16..4:  1.2%  4.2%  1.9%  P16..4: 21.4%  5.3%  2.9%  0.0%  0.0%    skip:63.1%
[libx264 @ 0000016fb5072040] mb B  I16..4:  0.1%  0.2%  0.1%  B16..8: 12.6%  1.5%  0.5%  direct: 0.4%  skip:84.6%  L0:35.9% L1:60.0% BI: 4.1%
[libx264 @ 0000016fb5072040] 8x8 transform intra:55.5% inter:60.0%
[libx264 @ 0000016fb5072040] coded y,uvDC,uvAC intra: 63.5% 77.4% 40.4% inter: 3.7% 4.6% 0.7%
[libx264 @ 0000016fb5072040] i16 v,h,dc,p: 19% 37% 15% 29%
[libx264 @ 0000016fb5072040] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 18% 23%  7%  7%  7%  7%  6%  8%
[libx264 @ 0000016fb5072040] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 22% 17%  7%  8%  7%  7%  6%  6%
[libx264 @ 0000016fb5072040] i8c dc,h,v,p: 45% 27% 18% 11%
[libx264 @ 0000016fb5072040] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000016fb5072040] ref P L0: 74.5% 13.9%  9.1%  2.5%
[libx264 @ 0000016fb5072040] ref B L0: 91.9%  7.4%  0.7%
[libx264 @ 0000016fb5072040] ref B L1: 96.6%  3.4%
[libx264 @ 0000016fb5072040] kb/s:5144.05
[aac @ 0000016fb5026a80] Qavg: 193.230 


  


  Logs for putting the overlay on broken_source :

  


  ffmpeg -i broken_source.mp4 -vcodec libvpx -i chat.webm -filter_complex "overlay = 0:0" broken_complete.mp4
ffmpeg version 4.3.1-2021-01-01-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'broken_source.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.26.101
  Duration: 00:00:45.02, start: 0.000000, bitrate: 5962 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 5952 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[libvpx @ 0000021071f90280] v1.9.0-128-g3a38edea2
    Last message repeated 1 times
Input #1, matroska,webm, from 'chat.webm':
  Metadata:
    ENCODER         : Lavf58.51.101
  Duration: 00:00:31.00, start: 0.000000, bitrate: 449 kb/s
    Stream #1:0: Video: vp8, yuva420p(tv, progressive), 500x300, SAR 1:1 DAR 5:3, 60 fps, 60 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      alpha_mode      : 1
      ENCODER         : Lavc58.101.101 libvpx
      DURATION        : 00:00:31.000000000
[libvpx @ 0000021072020dc0] v1.9.0-128-g3a38edea2
  Stream #0:0 (h264) -> overlay:main (graph 0)
  Stream #1:0 (libvpx) -> overlay:overlay (graph 0)
  overlay (graph 0) -> Stream #0:0 (libx264)
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libvpx @ 0000021072020dc0] v1.9.0-128-g3a38edea2
[libx264 @ 0000021072022980] using SAR=1/1
[libx264 @ 0000021072022980] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000021072022980] profile High, level 4.2, 4:2:0, 8-bit
[libx264 @ 0000021072022980] 264 - core 161 r3027 4121277 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'broken_complete.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 60 fps, 15360 tbn, 60 tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.91.100 aac
frame= 2701 fps= 47 q=-1.0 Lsize=   23020kB time=00:00:45.01 bitrate=4189.5kbits/s dup=60 drop=0 speed=0.789x
video:22242kB audio:708kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.309473%
[libx264 @ 0000021072022980] frame I:11    Avg QP:18.82  size:210832
[libx264 @ 0000021072022980] frame P:742   Avg QP:23.14  size: 18106
[libx264 @ 0000021072022980] frame B:1948  Avg QP:31.16  size:  3604
[libx264 @ 0000021072022980] consecutive B-frames:  1.7%  4.2%  6.2% 87.8%
[libx264 @ 0000021072022980] mb I  I16..4: 10.0% 46.2% 43.7%
[libx264 @ 0000021072022980] mb P  I16..4:  1.6%  4.5%  1.5%  P16..4: 16.5%  4.2%  2.4%  0.0%  0.0%    skip:69.3%
[libx264 @ 0000021072022980] mb B  I16..4:  0.1%  0.3%  0.1%  B16..8: 11.3%  1.4%  0.4%  direct: 0.4%  skip:86.0%  L0:36.5% L1:59.3% BI: 4.3%
[libx264 @ 0000021072022980] 8x8 transform intra:57.4% inter:64.8%
[libx264 @ 0000021072022980] coded y,uvDC,uvAC intra: 55.4% 42.1% 14.8% inter: 3.3% 2.1% 0.3%
[libx264 @ 0000021072022980] i16 v,h,dc,p: 21% 37% 15% 27%
[libx264 @ 0000021072022980] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 18% 24%  6%  7%  6%  7%  6%  7%
[libx264 @ 0000021072022980] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 22% 16%  7%  7%  7%  7%  6%  6%
[libx264 @ 0000021072022980] i8c dc,h,v,p: 58% 22% 15%  5%
[libx264 @ 0000021072022980] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000021072022980] ref P L0: 68.9% 16.6% 10.8%  3.6%
[libx264 @ 0000021072022980] ref B L0: 91.6%  7.2%  1.2%
[libx264 @ 0000021072022980] ref B L1: 95.9%  4.1%
[libx264 @ 0000021072022980] kb/s:4047.35
[aac @ 0000021072024840] Qavg: 189.405 


  


• ffmpeg conversion from h264 for webserver

  11 juin 2021, par Xav

  I'm working on a system to record videos through RPi (picamera, h264 format), export (rsync) 10s segments of videos to a distant webserver, and trying to concatenate those segment into a single video file to read on the webserver.

  


  Simplified code to record on RPi :

  


  import os
import datetime as dt
import picamera

camera = picamera.PiCamera()
camera.resolution = (1920, 1080)
camera.framerate = 25
duration = 10 #sec

[...]

camera.start_recording(filename, format='h264')
while True:
    filename = get_timestamped_filename()
    camera.split_recording(filename)
    camera.wait_recording(duration)

camera.stop_recording()


  


  For the conversion, I tried many options/searches without success. I can get the video (.ts, .mp4, .ogv, .webm) working on majority of web browser but never on ios/android (chrome/safari).
As example of test & fail with ffmpeg : https://gist.github.com/jaydenseric/220c785d6289bcfd7366

  


  When I don't try to concatenate but only convert those video :

  


  ffmpeg -i input.h264 -profile:v baseline -level 3.0 output.mp4


  


  I have the following output :

  


  ffmpeg version 4.1.6-1~deb10u1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, h264, from 'input.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 720x480, 25 fps, 25 tbr, 1200k tbn, 50 tbc
File 'output.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x556885757980] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x556885757980] profile Constrained Baseline, level 3.0
[libx264 @ 0x556885757980] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=0 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'output.mp4':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 720x480, q=-1--1, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame=  195 fps=0.0 q=-1.0 Lsize=     229kB time=00:00:07.76 bitrate= 241.3kbits/s speed=20.4x    
video:227kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.687068%
[libx264 @ 0x556885757980] frame I:1     Avg QP:23.86  size: 24802
[libx264 @ 0x556885757980] frame P:194   Avg QP:22.46  size:  1067
[libx264 @ 0x556885757980] mb I  I16..4: 26.4%  0.0% 73.6%
[libx264 @ 0x556885757980] mb P  I16..4:  1.1%  0.0%  0.4%  P16..4: 20.3%  1.5%  0.6%  0.0%  0.0%    skip:76.1%
[libx264 @ 0x556885757980] coded y,uvDC,uvAC intra: 34.9% 54.4% 12.3% inter: 1.8% 12.4% 0.0%
[libx264 @ 0x556885757980] i16 v,h,dc,p: 45% 14% 17% 23%
[libx264 @ 0x556885757980] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 35% 31% 11%  4%  4%  4%  4%  4%  4%
[libx264 @ 0x556885757980] i8c dc,h,v,p: 51% 17% 28%  4%
[libx264 @ 0x556885757980] ref P L0: 80.0% 10.6%  9.4%
[libx264 @ 0x556885757980] kb/s:237.75


  


  But still, impossible to play on mobile web (iOS, Android)

  


  Is there anything I'm doing wrong, either in the capturing process or conversion ?
Do you have any recommendations ?

  


• How to fix laggy ffmpeg screen and audio capture ?

  26 juillet 2022, par Wh0r00t

  I am using ffmpeg to capture the screen along with audio.

  


  The ffmpeg command that i tried is

  


  ffmpeg -y \
    -f x11grab \
    -framerate 60 \
    -s 1366x768 \
    -i :0.0 \
    -f alsa -i default -ac 2 \
    -r 30 \
    -c:v h264 -crf 0 -preset ultrafast -c:a vorbis -strict experimental  \
    "$HOME/Videos/$fname-$(date '+%y%m%d-%H%M-%S').mkv"


  


  The stdout of the ffmpeg https://pastebin.com/Qmi5TMKv

  


  ffmpeg version n5.0.1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
[x11grab @ 0x561faf77eb00] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, x11grab, from ':0.0':
  Duration: N/A, start: 1658814267.169414, bitrate: 2014248 kb/s
  Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1366x768, 2014248 kb/s, 60 fps, 1000k tbr, 1000k tbn
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, alsa, from 'default':
  Duration: N/A, start: 1658814267.230653, bitrate: 1536 kb/s
  Stream #1:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  Stream #1:0 -> #0:1 (pcm_s16le (native) -> vorbis (native))
Press [q] to stop, [?] for help
[libx264 @ 0x561faf7d4300] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX XOP FMA3 BMI1
[libx264 @ 0x561faf7d4300] profile High 4:4:4 Predictive, level 3.2, 4:4:4, 8-bit
[libx264 @ 0x561faf7d4300] 264 - core 164 r3081 19856cc - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=0 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=cqp mbtree=0 qp=0
[alsa @ 0x561faf78a940] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
Output #0, matroska, to '/home/earth/Videos/-220726-1114-27.mkv':
  Metadata:
    encoder         : Lavf59.16.100
  Stream #0:0: Video: h264 (H264 / 0x34363248), yuv444p(tv, progressive), 1366x768, q=2-31, 30 fps, 1k tbn
    Metadata:
      encoder         : Lavc59.18.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: vorbis (oV[0][0] / 0x566F), 48000 Hz, stereo, fltp
    Metadata:
      encoder         : Lavc59.18.100 vorbis
[vorbis @ 0x561faf7d5500] Queue input is backward in time0 bitrate=N/A speed=   0x
frame=  153 fps= 31 q=-1.0 Lsize=    2295kB time=00:00:05.06 bitrate=3709.5kbits/s dup=0 drop=150 speed=1.01x
video:2282kB audio:7kB subtitle:0kB other streams:0kB global headers:3kB muxing overhead: 0.281689%
[libx264 @ 0x561faf7d4300] frame I:1     Avg QP: 0.00  size:381729
[libx264 @ 0x561faf7d4300] frame P:152   Avg QP: 0.00  size: 12857
[libx264 @ 0x561faf7d4300] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x561faf7d4300] mb P  I16..4: 56.3%  0.0%  0.0%  P16..4:  0.1%  0.0%  0.0%  0.0%  0.0%    skip:43.6%
[libx264 @ 0x561faf7d4300] coded y,u,v intra: 1.6% 1.6% 1.6% inter: 0.2% 0.2% 0.2%
[libx264 @ 0x561faf7d4300] i16 v,h,dc,p: 99%  1%  0%  0%
[libx264 @ 0x561faf7d4300] kb/s:3664.27
Exiting normally, received signal 15.


  


  I am using the preset ultrafast because I read that it helps not to compress the video too much.
The output of the recorded test file using ffmpeg is as below.

  


   (+) Video --vid=1 (h264 1366x768 30.000fps)
 (+) Audio --aid=1 (vorbis 2ch 48000Hz)
AO: [pulse] 48000Hz stereo 2ch float
VO: [gpu] 1366x768 yuv444p
AV: 00:00:03 / 00:00:19 (17%) A-V:  0.000
[mkv] Discarding potentially broken or useless index.
AV: 00:00:14 / 00:00:19 (73%) A-V:  0.000

Exiting... (Quit)


  


  The recording works but there is a audio lag. If I record the same using simplescreenrecorder with the same settings like,

  


  audio backend - alsa

  


  source - default

  


  audio codec - vorbis

  


  video codec - h.264

  


  container - matroska

  


  preset - superfast

  


  The simplescreenrecorder log https://pastebin.com/83hMMRQF

  


  [PageRecord::StartPage] Starting page ...
[PageRecord::StartPage] Started page.
[PageRecord::StartOutput] Starting output ...
[PageRecord::StartOutput] Output file: /home/earth/Videos/simplescreenrecorder-2022-07-26_11.18.13.mkv
[Muxer::Init] Using format matroska (Matroska).
[Muxer::AddStream] Using codec libx264 (libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10).
[VideoEncoder::PrepareStream] Using pixel format nv12.
[libx264 @ 0x563436cbfd40] using SAR=1/1
[libx264 @ 0x563436cbfd40] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX XOP FMA3 BMI1
[libx264 @ 0x563436cbfd40] profile High, level 3.2, 4:2:0, 8-bit
[libx264 @ 0x563436cbfd40] 264 - core 164 r3081 19856cc - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00
[Muxer::AddStream] Using codec libvorbis (libvorbis).
[BaseEncoder::EncoderThread] Encoder thread started.
[AudioEncoder::PrepareStream] Using sample format f32p.
[BaseEncoder::EncoderThread] Encoder thread started.
[Muxer::MuxerThread] Muxer thread started.
[PageRecord::StartOutput] Started output.
[Synchronizer::SynchronizerThread] Synchronizer thread started.
[PageRecord::StartInput] Starting input ...
[X11Input::Init] Using X11 shared memory.
[X11Input::Init] Detecting screen configuration ...
[X11Input::Init] Screen 0: x1 = 0, y1 = 0, x2 = 1366, y2 = 768
[X11Input::InputThread] Input thread started.
[ALSAInput::InputThread] Using sample format s16.
[PageRecord::StartInput] Started input.
[ALSAInput::InputThread] Input thread started.
[FastResampler::Resample] Resample ratio is 1.0000 (was 0.0000).
[PageRecord::StopOutput] Stopping output ...
[PageRecord::StopOutput] Stopped output.
[PageRecord::StopInput] Stopping input ...
[X11Input::~X11Input] Stopping input thread ...
[X11Input::InputThread] Input thread stopped.
[ALSAInput::~ALSAInput] Stopping input thread ...
[ALSAInput::InputThread] Input thread stopped.
[PageRecord::StopInput] Stopped input.


  


  It works perfectly without any lag whatsoever. The output of the recorded test file using simplescreenrecorder is as below.

  


   (+) Video --vid=1 (h264 1366x768)
 (+) Audio --aid=1 (vorbis 2ch 48000Hz)
AO: [pulse] 48000Hz stereo 2ch float
VO: [gpu] 1366x768 yuv420p
AV: 00:00:01 / 00:00:17 (7%) A-V:  0.000
[mkv] Discarding potentially broken or useless index.
AV: 00:00:08 / 00:00:17 (47%) A-V:  0.000

Exiting... (Quit)


  


  The only difference that I saw between these two recordings is VO: [gpu] 1366x768 yuv444p
VO: [gpu] 1366x768 yuv420p for ffmpeg and simplescreenrecorder receptively.
I do not know if this matters but is there something that I could tweak to make ffmpeg to capture the screen and audio without any lag.
Like answered here https://unix.stackexchange.com/questions/675436/ffmpeg-recording-slows-down-when-audio-inputs-are-added
I do open pavucontrol but its not much of a help.

  


  The reason that I going with ffmpeg is because I can kill the process using pid at a particular time using cronjobs.
These are my system information, in case if it helps

  


  System:
  Host: taco Kernel: 5.18.12-arch1-1 arch: x86_64 bits: 64 Desktop: dwm
    v: 6.2 Distro: Arch Linux
Machine:
  Type: Desktop Mobo: Acer model: A75F2-M v: P21-A1 serial: N/A BIOS: Acer
    v: P21-A1 date: 02/07/2014
CPU:
  Info: quad core model: AMD A8-5500B APU with Radeon HD Graphics bits: 64
    type: MT MCP cache: L2: 4 MiB
  Speed (MHz): avg: 1400 min/max: 1400/3200 cores: 1: 1400 2: 1400 3: 1400
    4: 1400
Graphics:
  Device-1: AMD Trinity [Radeon HD 7560D] driver: radeon v: kernel
  Display: server: X.Org v: 21.1.4 driver: X: loaded: modesetting
    gpu: radeon resolution: 1366x768~60Hz
  OpenGL: renderer: AMD ARUBA (DRM 2.50.0 / 5.18.12-arch1-1 LLVM 14.0.6)
    v: 4.3 Mesa 22.1.3
Audio:
  Device-1: AMD FCH Azalia driver: snd_hda_intel
  Sound Server-1: ALSA v: k5.18.12-arch1-1 running: yes
  Sound Server-2: PulseAudio v: 16.1 running: yes
  Sound Server-3: PipeWire v: 0.3.56 running: yes


  


  Any help is much appreciated.