Recherche avancée

Sur d’autres sites (4235)

• ffmpeg zero latency webcam streaming

  9 août 2022, par SimpleSimon

  I'm trying to use ffmpeg to stream a webcam with as close to zero latency as possible.

  


  My test environment is streaming from localhost to localhost, on a macOS machine.

  


  I've tried every parameter switch that I can understand - and many suggestions found both here and elsewhere. But, the latency is persistently just about one second. And to be honest, none of the options that I've tried appear to make any difference at all.

  


  I've read that it should be possible to get the latency down to a couple of hundred milliseconds - or even as low as a single video frame (in these ideal test conditions). Can anyone help me with the magic ffmpeg settings ?

  


  Here's my base ffmpeg call. Note that I've deliberately not included here any of the exotic options that might help me. I want to start fresh with this question :

  


  ffmpeg -f avfoundation -i "1:none" -vcodec libx264 -preset ultrafast -tune zerolatency -x264-params "" -f mpegts udp://127.0.0.1:9090

  


  I monitor the stream on the same machine using ffplay, with this :

  


  ffplay udp://127.0.0.1:9090

  


  My belief is that ffmpeg (and X264) are somehow buffering the input stream from the webcam during the encoding process. I know why this can be necessary. But, for the purposes of this requirement, I'm prepared to sacrifice everything in the pursuit of low latency. Quality and stability are less important.

  


  I acknowledge that there might be latency in my ffplay monitoring. I'm not so sure though - because I have tested with other monitors, with pretty much the same results. However, I'd be happy to receive advice regarding low latency monitoring.

  


  I'm assuming I'm looking for options that will encode with all i-frames, no lookahead, and no buffering. Am I right ? And if I am, how do I achieve that ?

  


  Thanks !

  


  ---

  


  Output from ffmpeg request, as per Gyan :

  


  ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[avfoundation @ 0x7ffbc2008a00] Selected pixel format (yuv420p) is not supported by the input device.
[avfoundation @ 0x7ffbc2008a00] Supported pixel formats:
[avfoundation @ 0x7ffbc2008a00]   uyvy422
[avfoundation @ 0x7ffbc2008a00]   yuyv422
[avfoundation @ 0x7ffbc2008a00]   nv12
[avfoundation @ 0x7ffbc2008a00]   0rgb
[avfoundation @ 0x7ffbc2008a00]   bgr0
[avfoundation @ 0x7ffbc2008a00] Overriding selected pixel format to use uyvy422 instead.
Input #0, avfoundation, from '1:none':
  Duration: N/A, start: 51270.672067, bitrate: N/A
    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1280x720, 17 tbr, 1000k tbn, 1000k tbc
At least one output file must be specified```


---------------------

Second output, as per second request from Gyan:

ffmpeg -f avfoundation -analyzeduration 200k -probesize 6M -pixel_format uyvy422 -i "1:none" -vcodec libx264 -preset ultrafast -tune zerolatency -x264-params "" -f mpegts udp://127.0.0.1:9090
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, avfoundation, from '1:none':
  Duration: N/A, start: 319.565533, bitrate: N/A
    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1280x720, 15 fps, 15 tbr, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x7feda5093200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7feda5093200] profile High 4:2:2, level 3.1, 4:2:2 8-bit
Output #0, mpegts, to 'udp://127.0.0.1:9090':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (libx264), yuv422p, 1280x720, q=-1--1, 15 fps, 90k tbn, 15 tbc
    Metadata:
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame= 1209 fps= 14 q=22.0 Lsize=   65771kB time=00:01:24.13 bitrate=6404.1kbits/s speed=   1x    
video:60873kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 8.047672%
[libx264 @ 0x7feda5093200] frame I:5     Avg QP:17.80  size:142658
[libx264 @ 0x7feda5093200] frame P:1204  Avg QP:19.99  size: 51180
[libx264 @ 0x7feda5093200] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x7feda5093200] mb P  I16..4:  5.9%  0.0%  0.0%  P16..4: 71.8%  0.0%  0.0%  0.0%  0.0%    skip:22.3%
[libx264 @ 0x7feda5093200] coded y,uvDC,uvAC intra: 17.4% 87.8% 49.8% inter: 29.4% 74.5% 38.3%
[libx264 @ 0x7feda5093200] i16 v,h,dc,p: 57% 20% 11% 12%
[libx264 @ 0x7feda5093200] i8c dc,h,v,p: 43% 18% 27% 12%
[libx264 @ 0x7feda5093200] kb/s:6186.95
Exiting normally, received signal 2.
slartibartfast:~ simon$ ffmpeg -f avfoundation -analyzeduration 200k -probesize 6M -pixel_format uyvy422 -i "1:none" 
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, avfoundation, from '1:none':
  Duration: N/A, start: 421.525767, bitrate: N/A
    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1280x720, 14.25 tbr, 1000k tbn, 1000k tbc
At least one output file must be specified


  


• ffmpeg filter_complex concat never completes

  10 juillet 2020, par seawolf

  I am trying to concatenate mp4 files into a single file. I am attempting to use the concat filter directly for reasons independent from this question (so -i list.txt is not a valid solution). All video files in question are between 4 and 20 minutes in length.

  


  What I am executing :

  


  % ffmpeg -i f01.mp4 -i f02.mp4 -filter_complex "[0:v][0:a][1:v][1:a]concat=n=2:v=1:a=1[v][a]" -map "[v]" -map "[a]" output.mp4


  


  This is my understanding of the form of this command from references such as FFmpeg Filters Documentation : concat and Concatenate Videos Together Using FFMPEG !.

  


  What happens :

  


  ffmpeg gives a lot of output as it checks the metadata for each stream and then begins processing. After a short duration (several seconds to a minute or so, seems to be prortional to the duration of the first video) I start seeing messages like this :

  


  More than 1000 frames duplicated
More than 10000 frames duplicated     512kB time=00:00:00.12 bitrate=32771.0kbits/s dup=33365 drop=0 speed=0.00449x
More than 100000 frames duplicated   1280kB time=00:00:00.17 bitrate=61442.1kbits/s dup=66730 drop=0 speed=0.00272x


  


  ... and then the process never completes. If I leave my computer running for 24 hours, ffmpeg is still using max available CPU (200-300%). The output file is 48 bytes in length.

  


  Note : the inputs are only a few minutes each, so individually re-encoding the inputs would take only a few minutes each.

  


  Note : if I change the command to use f01.mp4 for both source 0 and source 1, the command completes as expected in under 5 minutes :

  


  % # example that works:
% # note, however, that this just repeats the same source twice
% ffmpeg -i f01.mp4 -i f01.mp4 -filter_complex "[0:v][0:a][1:v][1:a]concat=n=2:v=1:a=1[v][a]" -map "[v]" -map "[a]" output.mp4


  


  Full output :

  


  Here is the complete output of a run in case it's helpful :

  


  % ffmpeg -i f01.mp4 -i f02.mp4 -filter_complex "[0:v][0:a][1:v][1:a]concat=n=2:v=1:a=1[v][a]" -map "[v]" -map "[a]" output.mp4
ffmpeg version 4.3 Copyright (c) 2000-2020 the FFmpeg developers
  built with Apple clang version 11.0.3 (clang-1103.0.32.62)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'f01.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 2020-06-04T21:34:26.000000Z
    encoder         : HandBrake 1.3.2 2020050300
  Duration: 00:04:14.66, start: 0.000000, bitrate: 525 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, smpte170m/smpte170m/bt709), 708x478 [SAR 8:9 DAR 944:717], 366 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2020-06-04T21:34:26.000000Z
      handler_name    : VideoHandler
    Stream #0:1(jpn): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 149 kb/s (default)
    Metadata:
      creation_time   : 2020-06-04T21:34:26.000000Z
      handler_name    : Stereo
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'f02.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 2020-06-04T21:38:15.000000Z
    encoder         : HandBrake 1.3.2 2020050300
  Duration: 00:06:30.95, start: 0.000000, bitrate: 1328 kb/s
    Stream #1:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, smpte170m/smpte170m/bt709), 708x478 [SAR 8:9 DAR 944:717], 1179 kb/s, 29.97 fps, 30 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2020-06-04T21:38:15.000000Z
      handler_name    : VideoHandler
    Stream #1:1(jpn): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 140 kb/s (default)
    Metadata:
      creation_time   : 2020-06-04T21:38:15.000000Z
      handler_name    : Stereo
File 'output.mp4' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 (h264) -> concat:in0:v0
  Stream #0:1 (aac) -> concat:in0:a0
  Stream #1:0 (h264) -> concat:in1:v0
  Stream #1:1 (aac) -> concat:in1:a0
  concat:out:v0 -> Stream #0:0 (libx264)
  concat:out:a0 -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
[mp4 @ 0x7ff130014000] Frame rate very high for a muxer not efficiently supporting it.
Please consider specifying a lower framerate, a different muxer or -vsync 2
[libx264 @ 0x7ff130021200] using SAR=8/9
[libx264 @ 0x7ff130021200] MB rate (1350000000) > level limit (16711680)
[libx264 @ 0x7ff130021200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7ff130021200] profile High, level 6.2, 4:2:0, 8-bit
[libx264 @ 0x7ff130021200] 264 - core 160 r3011 cde9a93 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'output.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p(progressive), 708x478 [SAR 8:9 DAR 944:717], q=-1--1, 1000k tbn, 1000k tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      encoder         : Lavc58.91.100 aac
More than 1000 frames duplicated
More than 10000 frames duplicated     512kB time=00:00:00.12 bitrate=32771.0kbits/s dup=33365 drop=0 speed=0.00449x
More than 100000 frames duplicated   1280kB time=00:00:00.17 bitrate=61442.1kbits/s dup=66730 drop=0 speed=0.00272x
frame=667333 fps=1079 q=33.0 size=   14848kB time=00:00:00.76 bitrate=158379.2kbits/s dup=667312 drop=0 speed=0.00124x


  


• How to set audio format from an ffmpeg filter ?

  4 août 2020, par Vadim Kantorov

  Is it possible to set the audio format just with an ffmpeg filter ?

  


  My usecase is programmatic usage, so if it's possible to do with filters, that would simplify everything.

  


  # create sample s16 audio
ffmpeg -f lavfi -i "sine=frequency=1000:duration=5" -c:a pcm_s16le -ar 8000 test.wav

# works
ffmpeg -i test.wav -y -af 'aresample=osf=flt,aformat=sample_fmts=flt' -f f32le test_f32.raw

# fails
ffmpeg -i test.wav -y -af 'aresample=osf=flt,aformat=sample_fmts=flt' test_f32.raw
# [NULL @ 0x56042863d980] Unable to find a suitable output format for 'test_f32.raw'