Recherche avancée

Sur d’autres sites (8467)

• ffmpeg streaming fails to stream over internet to twitch.tv

  15 avril 2021, par josh joyer

  I did already streaming to twitch.tv with command :

  


  ffmpeg -stream_loop -1 -i 9stream.wav 
-f dshow -i audio="mic"
 -f dshow -i audio="realTek" 
-filter_complex "[0:a]volume=2[a0];[1:a]volume=1.5[a1];[2:a]volume=1.5[a2];[a0][a1][a2]amix=inputs=3"
 -f dshow -i video="USB2.0 PC CAMERA" 
-ac 1 -ar 11025 -acodec libmp3lame -c:v libx264 -b:v 100k -f flv -s 80x120 
rtmp://live.twitch.tv/app/live_streamingKey


  


  It was most advanced command that I used to stream online.

  


  (I do not know how to make enter in here so I put double enter)

  


  9stream.wav was played in loop as background music

  


  microphone was added

  


  stereoMix named realTek was the playback of system sounds

  


  volume was adjusted and all sounds mixed into one stream

  


  camera view was added

  


  THEN network flow was reduced by sending only one channel with low frequency of 11025 with lowest

  


  possible data size made by mp3 encoder and libx264 was used to encode video in png files.

  


  It was working fine SO I decided to make final version

  


  (this one worked with all sounds(background music,microphone,system sounds) and camera)

  


  Final version was about adding screen view and logo.

  


  I succeded writing everything to disc with command :

  


  ffmpeg 
-stream_loop -1 -i 9stream.wav 
-f dshow -i audio="mic" 
-f dshow -i audio="stereoMixRealtek" 
-i camera.png 
-f gdigrab -framerate 1 -i desktop 
-f dshow -framerate 15 -i video="USB2.0 PC CAMERA" 
-filter_complex "[0:a]volume=2[a0];[1:a]volume=1.5[a1];[2:a]volume=1.5[a2];
[a0][a1][a2]amix=inputs=3[aMix];
[4:v]scale=200:-1[v4];[5:v]scale=50:-1[v5];
[v4][v5]overlay=(W-w)-5:(H-h)-5[vScreenCam];
[vScreenCam][3:v]overlay=5:5[v]" 
-map "[v]" -map "[aMix]" -ac 1 -ar 11025 -c:a libmp3lame -r 1 -c:v libx264 output.mkv


  


  That was

  


  background music

  


  microphone

  


  system sounds

  


  logo picture

  


  screen view

  


  camera

  


  adjusting sound volume

  


  mixing sounds

  


  reducing size of screen view and camera view

  


  overlaying reduced camera view over reduced screen view

  


  adding logo

  


  choosing final view, final mixed sounds,

  


  reducing data size to one channel, reducing sample frequency,

  


  choosing mp3 codec to reduce final data size,

  


  choosing minimal framerate of one per second to reduce data size

  


  choosing libx264 codec for video.

  


  THEN I tried to use final command for network streaming with slight modification :

  


  ffmpeg 
-stream_loop -1 -i 9stream.wav 
-f dshow -i audio="mic" 
-f dshow -i audio="stereo mix" 
-i camera.png 
-f gdigrab -framerate 1 -i desktop 
-f dshow -framerate 15 -i video="USB2.0 PC CAMERA" 
-filter_complex "[0:a]volume=2[a0];[1:a]volume=1.5[a1];[2:a]volume=1.5[a2];
[a0][a1][a2]amix=inputs=3[aMix];
[4:v]scale=200:-1[v4];[5:v]scale=50:-1[v5];
[v4][v5]overlay=(W-w)-5:(H-h)-5[vScreenCam];[vScreenCam][3:v]overlay=5:5[v]" 
-map "[v]" -map "[aMix]" 
-ac 1 -ar 11025 -c:a libmp3lame -r 1 -c:v libx264 -b:v 100k -b:a 10k -f flv rtmp://live.twitch.tv/app/live_streamingKey


  


  I added parameter
-b:v 100k to reduce video flow
-b:a 10k to reduce sound flow
-f flv to be good for twitch.tv otherwise it would not accept stream

  


  BUT ffmpeg is always stopping sending data with message like this :

  


  testosteron_@testosteron MINGW64 ~/Desktop/2021b/magisterka/ScreenRecorderXi/ScreenRecorderXi/bin
$ cmd
Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. Wszelkie prawa zastrze▒one.

C:\Users\testosteron_\Desktop\2021b\magisterka\ScreenRecorderXi\ScreenRecorderXi\bin>ffmpeg -stream_loop -1 -i 9stream.wav -f dshow -i audio="@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{5B4DB0B5-B645-4AFA-930D-4710AAF753DB}" -f dshow -i audio="@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{ADECEC1D-C3CC-4BAE-8516-752251B8B63F}" -i camera.png -f gdigrab -framerate 1 -i desktop -f dshow -framerate 15 -i video="USB2.0 PC CAMERA" -filter_complex "[0:a]volume=2[a0];[1:a]volume=1.5[a1];[2:a]volume=1.5[a2];[a0][a1][a2]amix=inputs=3[aMix];[4:v]scale=200:-1[v4];[5:v]scale=50:-1[v5];[v4][v5]overlay=(W-w)-5:(H-h)-5[vScreenCam];[vScreenCam][3:v]overlay=5:5[v]" -map "[v]" -map "[aMix]" -ac 1 -ar 11025 -c:a libmp3lame -r 1 -c:v libx264 -b:v 100k -b:a 10k -f flv rtmp://live.twitch.tv/app/live_674912043_oAwGnACTndHyeZnlA6scLegm8gaxwf
ffmpeg -stream_loop -1 -i 9stream.wav -f dshow -i audio="@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{5B4DB0B5-B645-4AFA-930D-4710AAF753DB}" -f dshow -i audio="@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{ADECEC1D-C3CC-4BAE-8516-752251B8B63F}" -i camera.png -f gdigrab -framerate 1 -i desktop -f dshow -framerate 15 -i video="USB2.0 PC CAMERA" -filter_complex "[0:a]volume=2[a0];[1:a]volume=1.5[a1];[2:a]volume=1.5[a2];[a0][a1][a2]amix=inputs=3[aMix];[4:v]scale=200:-1[v4];[5:v]scale=50:-1[v5];[v4][v5]overlay=(W-w)-5:(H-h)-5[vScreenCam];[vScreenCam][3:v]overlay=5:5[v]" -map "[v]" -map "[aMix]" -ac 1 -ar 11025 -c:a libmp3lame -r 1 -c:v libx264 -b:v 100k -b:a 10k -f flv rtmp://live.twitch.tv/app/live_674912043_oAwGnACTndHyeZnlA6scLegm8gaxwf
ffmpeg version git-2020-08-02-b48397e Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.1 (GCC) 20200726
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libgsm --enable-librav1e --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 57.100 / 56. 57.100
  libavcodec     58. 99.100 / 58. 99.100
  libavformat    58. 49.100 / 58. 49.100
  libavdevice    58. 11.101 / 58. 11.101
  libavfilter     7. 87.100 /  7. 87.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, wav, from '9stream.wav':
  Metadata:
    encoder         : Lavf58.49.100
  Duration: 00:00:13.48, bitrate: 1411 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, dshow, from 'audio=@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{5B4DB0B5-B645-4AFA-930D-4710AAF753DB}':
  Duration: N/A, start: 209609.948000, bitrate: 1411 kb/s
    Stream #1:0: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s
Guessed Channel Layout for Input Stream #2.0 : stereo
Input #2, dshow, from 'audio=@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{ADECEC1D-C3CC-4BAE-8516-752251B8B63F}':
  Duration: N/A, start: 209610.502000, bitrate: 1411 kb/s
    Stream #2:0: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s
Input #3, png_pipe, from 'camera.png':
  Duration: N/A, bitrate: N/A
    Stream #3:0: Video: png, rgba(pc), 32x32 [SAR 3779:3779 DAR 1:1], 25 tbr, 25 tbn, 25 tbc
[gdigrab @ 0000009a3f019700] Capturing whole desktop as 1280x1024x32 at (0,0)
[gdigrab @ 0000009a3f019700] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #4, gdigrab, from 'desktop':
  Duration: N/A, start: 1618506176.140738, bitrate: 41943 kb/s
    Stream #4:0: Video: bmp, bgra, 1280x1024, 41943 kb/s, 1 fps, 1000k tbr, 1000k tbn, 1000k tbc
Input #5, dshow, from 'video=USB2.0 PC CAMERA':
  Duration: N/A, start: 209613.583000, bitrate: N/A
    Stream #5:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 640x480, 15 fps, 15 tbr, 10000k tbn, 10000k tbc
[dshow @ 0000009a3f034900] real-time buffer [USB2.0 PC CAMERA] [video input] too full or near too full (101% of size: 3041280 [rtbufsize parameter])! frame dropped!
    Last message repeated 9 times
Stream mapping:
  Stream #0:0 (pcm_s16le) -> volume
  Stream #1:0 (pcm_s16le) -> volume
  Stream #2:0 (pcm_s16le) -> volume
  Stream #3:0 (png) -> overlay:overlay
  Stream #4:0 (bmp) -> scale
  Stream #5:0 (rawvideo) -> scale
  overlay -> Stream #0:0 (libx264)
  amix -> Stream #0:1 (libmp3lame)
Press [q] to stop, [?] for help
[dshow @ 0000009a3efd5b80] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
[dshow @ 0000009a406fb280] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
[libx264 @ 0000009a4082ddc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
[libx264 @ 0000009a4082ddc0] profile High, level 1.1, 4:2:0, 8-bit
[libx264 @ 0000009a4082ddc0] 264 - core 161 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=5 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=1 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=100 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, flv, to 'rtmp://live.twitch.tv/app/live_streamingKey':
  Metadata:
    encoder         : Lavf58.49.100
    Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv420p(progressive), 200x160, q=-1--1, 100 kb/s, 1 fps, 1k tbn, 1 tbc (default)
    Metadata:
      encoder         : Lavc58.99.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/100000 buffer size: 0 vbv_delay: N/A
    Stream #0:1: Audio: mp3 (libmp3lame) ([2][0][0][0] / 0x0002), 11025 Hz, mono, fltp, 10 kb/s (default)
    Metadata:
      encoder         : Lavc58.99.100 libmp3lame
frame=    1 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   frame=    1 fps=1.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   frame=    1 fps=0.7 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   frame=    3 fps=1.5 q=0.0 size=       0kB time=00:00:03.08 bitrate=   1.0kbits/sframe=    4 fps=1.6 q=0.0 size=       0kB time=00:00:03.66 bitrate=   0.8kbits/sframe=    4 fps=1.3 q=0.0 size=       0kB time=00:00:03.66 bitrate=   0.8kbits/sframe=    5 fps=1.4 q=0.0 size=       0kB time=00:00:04.65 bitrate=   0.7kbits/sframe=    5 fps=1.2 q=0.0 size=       0kB time=00:00:04.65 bitrate=   0.7kbits/sframe=    6 fps=1.3 q=0.0 size=       0kB time=00:00:05.64 bitrate=   0.5kbits/sframe=    6 fps=1.2 q=0.0 size=       0kB time=00:00:05.64 bitrate=   0.5kbits/sframe=    7 fps=1.3 q=0.0 size=       0kB time=00:00:06.64 bitrate=   0.5kbits/sframe=    7 fps=1.2 q=0.0 size=       0kB time=00:00:06.64 bitrate=   0.5kbits/sframe=    8 fps=1.2 q=0.0 size=       0kB time=00:00:07.58 bitrate=   0.4kbits/sframe=    8 fps=1.1 q=0.0 size=       0kB time=00:00:07.58 bitrate=   0.4kbits/sframe=    9 fps=1.2 q=0.0 size=       0kB time=00:00:08.57 bitrate=   0.4kbits/sframe=    9 fps=1.1 q=0.0 size=       0kB time=00:00:08.57 bitrate=   0.4kbits/sframe=   10 fps=1.2 q=0.0 size=       0kB time=00:00:09.56 bitrate=   0.3kbits/sframe=   10 fps=1.1 q=0.0 size=       0kB time=00:00:09.56 bitrate=   0.3kbits/sframe=   11 fps=1.1 q=0.0 size=       1kB time=00:00:10.55 bitrate=   0.9kbits/sframe=   11 fps=1.1 q=0.0 size=       1kB time=00:00:10.55 bitrate=   0.9kbits/sframe=   12 fps=1.1 q=0.0 size=       2kB time=00:00:11.55 bitrate=   1.7kbits/sframe=   12 fps=1.1 q=0.0 size=       2kB time=00:00:11.55 bitrate=   1.7kbits/sframe=   13 fps=1.1 q=0.0 size=       4kB time=00:00:12.59 bitrate=   2.5kbits/sframe=   13 fps=1.1 q=0.0 size=       4kB time=00:00:12.59 bitrate=   2.5kbits/sframe=   14 fps=1.1 q=0.0 size=       5kB time=00:00:13.58 bitrate=   3.0kbits/sframe=   14 fps=1.1 q=0.0 size=       5kB time=00:00:13.58 bitrate=   3.0kbits/sframe=   15 fps=1.1 q=0.0 size=       6kB time=00:00:14.58 bitrate=   3.5kbits/sframe=   15 fps=1.1 q=0.0 size=       6kB time=00:00:14.58 bitrate=   3.5kbits/sframe=   16 fps=1.1 q=0.0 size=       8kB time=00:00:15.57 bitrate=   4.0kbits/sframe=   16 fps=1.1 q=0.0 size=       8kB time=00:00:15.57 bitrate=   4.0kbits/sframe=   17 fps=1.1 q=0.0 size=       9kB time=00:00:16.56 bitrate=   4.4kbits/sframe=   17 fps=1.1 q=0.0 size=       9kB time=00:00:16.56 bitrate=   4.4kbits/sframe=   18 fps=1.1 q=0.0 size=      10kB time=00:00:17.55 bitrate=   4.7kbits/sframe=   18 fps=1.0 q=0.0 size=      10kB time=00:00:17.55 bitrate=   4.7kbits/sframe=   19 fps=1.1 q=0.0 size=      11kB time=00:00:18.55 bitrate=   5.0kbits/sframe=   19 fps=1.0 q=0.0 size=      11kB time=00:00:18.55 bitrate=   5.0kbits/sframe=   20 fps=1.1 q=0.0 size=      13kB time=00:00:19.54 bitrate=   5.3kbits/sframe=   20 fps=1.0 q=0.0 size=      13kB time=00:00:19.54 bitrate=   5.3kbits/sframe=   21 fps=1.1 q=0.0 size=      14kB time=00:00:20.58 bitrate=   5.6kbits/sframe=   21 fps=1.0 q=0.0 size=      14kB time=00:00:20.58 bitrate=   5.6kbits/sframe=   22 fps=1.1 q=0.0 size=      15kB time=00:00:21.58 bitrate=   5.8kbits/sframe=   22 fps=1.0 q=0.0 size=      15kB time=00:00:21.58 bitrate=   5.8kbits/sframe=   23 fps=1.1 q=0.0 size=      17kB time=00:00:22.57 bitrate=   6.0kbits/sframe=   23 fps=1.0 q=0.0 size=      17kB time=00:00:22.57 bitrate=   6.0kbits/sframe=   24 fps=1.1 q=0.0 size=      18kB time=00:00:23.56 bitrate=   6.2kbits/sframe=   24 fps=1.0 q=0.0 size=      18kB time=00:00:23.56 bitrate=   6.2kbits/sframe=   25 fps=1.1 q=0.0 size=      19kB time=00:00:24.56 bitrate=   6.4kbits/sframe=   25 fps=1.0 q=0.0 size=      19kB time=00:00:24.56 bitrate=   6.4kbits/sframe=   26 fps=1.1 q=0.0 size=      20kB time=00:00:25.55 bitrate=   6.5kbits/sframe=   26 fps=1.0 q=0.0 size=      20kB time=00:00:25.55 bitrate=   6.5kbits/sframe=   27 fps=1.0 q=0.0 size=      22kB time=00:00:26.54 bitrate=   6.7kbits/sframe=   27 fps=1.0 q=0.0 size=      22kB time=00:00:26.54 bitrate=   6.7kbits/sframe=   28 fps=1.0 q=0.0 size=      23kB time=00:00:27.58 bitrate=   6.8kbits/sframe=   28 fps=1.0 q=0.0 size=      23kB time=00:00:27.58 bitrate=   6.8kbits/sframe=   29 fps=1.0 q=0.0 size=      24kB time=00:00:28.58 bitrate=   6.9kbits/sframe=   30 fps=1.1 q=0.0 size=      25kB time=00:00:29.00 bitrate=   7.0kbits/sframe=   30 fps=1.0 q=0.0 size=      25kB time=00:00:29.57 bitrate=   7.0kbits/sframe=   30 fps=1.0 q=0.0 size=      25kB time=00:00:29.57 bitrate=   7.0kbits/sframe=   31 fps=1.0 q=0.0 size=      27kB time=00:00:30.56 bitrate=   7.2kbits/sframe=   32 fps=1.1 q=0.0 size=      27kB time=00:00:30.56 bitrate=   7.2kbits/sframe=   32 fps=1.0 q=0.0 size=      28kB time=00:00:31.56 bitrate=   7.3kbits/sframe=   33 fps=1.1 q=0.0 size=      29kB time=00:00:32.55 bitrate=   7.4kbits/sframe=   33 fps=1.0 q=0.0 size=      29kB time=00:00:32.55 bitrate=   7.4kbits/sframe=   33 fps=1.0 q=0.0 size=      29kB time=00:00:32.55 bitrate=   7.4kbits/sframe=   34 fps=1.0 q=0.0 size=      31kB time=00:00:33.54 bitrate=   7.4kbits/sframe=   35 fps=1.1 q=0.0 size=      31kB time=00:00:33.96 bitrate=   7.5kbits/sframe=   35 fps=1.0 q=0.0 size=      32kB time=00:00:34.53 bitrate=   7.5kbits/sframe=   36 fps=1.0 q=0.0 size=      33kB time=00:00:35.58 bitrate=   7.6kbits/sframe=   36 fps=1.0 q=0.0 size=      33kB time=00:00:35.58 bitrate=   7.6kbits/sframe=   36 fps=1.0 q=0.0 size=      33kB time=00:00:35.58 bitrate=   7.6kbits/sframe=   37 fps=1.0 q=0.0 size=      34kB time=00:00:36.57 bitrate=   7.7kbits/sframe=   38 fps=1.0 q=0.0 size=      36kB time=00:00:37.56 bitrate=   7.8kbits/sframe=   38 fps=1.0 q=0.0 size=      36kB time=00:00:37.56 bitrate=   7.8kbits/sframe=   39 fps=1.0 q=0.0 size=      37kB time=00:00:38.56 bitrate=   7.8kbits/sframe=   39 fps=1.0 q=0.0 size=      37kB time=00:00:38.56 bitrate=   7.8kbits/sframe=   40 fps=1.0 q=0.0 size=      38kB time=00:00:39.55 bitrate=   7.9kbits/sframe=   40 fps=1.0 q=0.0 size=      38kB time=00:00:39.55 bitrate=   7.9kbits/sframe=   41 fps=1.0 q=0.0 size=      39kB time=00:00:40.54 bitrate=   8.0kbits/sframe=   41 fps=1.0 q=0.0 size=      39kB time=00:00:40.54 bitrate=   8.0kbits/sframe=   42 fps=1.0 q=0.0 size=      41kB time=00:00:41.59 bitrate=   8.0kbits/sframe=   42 fps=1.0 q=0.0 size=      41kB time=00:00:41.59 bitrate=   8.0kbits/sframe=   43 fps=1.0 q=0.0 size=      42kB time=00:00:42.58 bitrate=   8.1kbits/sframe=   43 fps=1.0 q=0.0 size=      42kB time=00:00:42.58 bitrate=   8.1kbits/sframe=   44 fps=1.0 q=0.0 size=      43kB time=00:00:43.57 bitrate=   8.1kbits/sframe=   44 fps=1.0 q=0.0 size=      43kB time=00:00:43.57 bitrate=   8.1kbits/sframe=   45 fps=1.0 q=0.0 size=      45kB time=00:00:44.56 bitrate=   8.2kbits/sframe=   45 fps=1.0 q=0.0 size=      45kB time=00:00:44.56 bitrate=   8.2kbits/sframe=   46 fps=1.0 q=0.0 size=      46kB time=00:00:45.56 bitrate=   8.2kbits/sframe=   46 fps=1.0 q=0.0 size=      46kB time=00:00:45.56 bitrate=   8.2kbits/sframe=   47 fps=1.0 q=0.0 size=      47kB time=00:00:46.55 bitrate=   8.3kbits/sframe=   47 fps=1.0 q=0.0 size=      47kB time=00:00:46.55 bitrate=   8.3kbits/sframe=   48 fps=1.0 q=0.0 size=      48kB time=00:00:47.54 bitrate=   8.3kbits/sframe=   48 fps=1.0 q=0.0 size=      48kB time=00:00:47.54 bitrate=   8.3kbits/sframe=   49 fps=1.0 q=0.0 size=      50kB time=00:00:48.59 bitrate=   8.4kbits/sframe=   49 fps=1.0 q=0.0 size=      50kB time=00:00:48.59 bitrate=   8.4kbits/s[flv @ 0000009a40865940] Packets poorly interleaved, failed to avoid negative timestamp -3900 in stream 0.
Try -max_interleave_delta 0 as a possible workaround.
[flv @ 0000009a40865940] Packets are not in the proper order with respect to DTS
av_interleaved_write_frame(): Invalid argument
[flv @ 0000009a40865940] Failed to update header with correct duration.
[flv @ 0000009a40865940] Failed to update header with correct filesize.
frame=   50 fps=1.0 q=6.0 Lsize=      63kB time=00:00:49.11 bitrate=  10.5kbits/s speed=   1x
video:27kB audio:48kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0000009a4082ddc0] frame I:1     Avg QP: 0.56  size: 27197
[libx264 @ 0000009a4082ddc0] frame P:15    Avg QP: 0.76  size:  2567
[libx264 @ 0000009a4082ddc0] frame B:34    Avg QP: 3.98  size:  1481
[libx264 @ 0000009a4082ddc0] consecutive B-frames:  8.0%  0.0% 12.0% 80.0%
[libx264 @ 0000009a4082ddc0] mb I  I16..4: 13.1% 13.8% 73.1%
[libx264 @ 0000009a4082ddc0] mb P  I16..4:  0.0%  0.1%  0.8%  P16..4: 17.5%  5.9%  4.2%  0.0%  0.0%    skip:71.5%
[libx264 @ 0000009a4082ddc0] mb B  I16..4:  0.0%  0.0%  0.3%  B16..8: 12.1%  4.2%  2.4%  direct: 6.3%  skip:74.7%  L0:42.9% L1:41.8% BI:15.4%
[libx264 @ 0000009a4082ddc0] final ratefactor: -7.50
[libx264 @ 0000009a4082ddc0] 8x8 transform intra:12.3% inter:14.5%
[libx264 @ 0000009a4082ddc0] coded y,uvDC,uvAC intra: 95.2% 96.9% 96.9% inter: 16.0% 14.9% 14.8%
[libx264 @ 0000009a4082ddc0] i16 v,h,dc,p: 26% 32% 32% 11%
[libx264 @ 0000009a4082ddc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  8% 40% 14%  8%  1%  2%  1%  1% 25%
[libx264 @ 0000009a4082ddc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 45%  7%  4%  5%  3%  7%  3%  9%
[libx264 @ 0000009a4082ddc0] i8c dc,h,v,p: 36% 40% 18%  6%
[libx264 @ 0000009a4082ddc0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000009a4082ddc0] ref P L0: 65.2%  2.2% 19.9% 12.7%
[libx264 @ 0000009a4082ddc0] ref B L0: 71.8% 23.0%  5.2%
[libx264 @ 0000009a4082ddc0] ref B L1: 88.2% 11.8%
[libx264 @ 0000009a4082ddc0] kb/s:17.86
Conversion failed!


  


  Main message from above was :

  


  [flv @ 0000009a40865940] Packets poorly interleaved, failed to avoid negative timestamp -3900 in stream 0.


  


  It was problem to stream 0 so it was mixed sounds stream BUT earlier it was fine with mixing

  


  and sending mix over internet BUT after I added screen view and scaling it failed to work.

  


  What is problem ?

  


  How to fix it ?

  


  Since I was able to do this to stream to disc I would assume that

  


  computer processing power is enough. Since I was able to stream over internet mixed sounds I

  


  would assume that it is not problem here. So the problem must be with sending

  


  screen view. BUT I put framerate 1 per second and downsized its resolution. I compressed

  


  sounds as much as I could. I added -b:a and -b:v commands to reduce network flow.

  


  WHAT ELSE COULD I DO TO FIX IT ?

  


• FFmpeg saturates memory + CPU usage drops to 0% during very basic conversion of PNG files to MP4 video

  7 août 2022, par mattze_frisch

  I have this Python function that runs ffmpeg with minimal options from the Windows command line :

  


  def run_ffmpeg(frames_path, ffmpeg_path=notebook_directory):
    '''
    This function runs ffmpeg.exe to convert PNG image files into a MP4 video.
    
    Parameters
    ----------
    frames_path : string
        Absolute path to the PNG files
    ffmpeg_path : string
        Absolute path to the FFmpeg executable (ffmpeg.exe)
    '''
    
    from subprocess import check_call
    
    
    check_call(
        [
            os.path.join(ffmpeg_path, 'ffmpeg'),
            '-y',    # Overwrite output files without asking
            '-report',    # Write logfile to current working directory
            '-framerate', '60',    # Input frame rate
            '-i', os.path.join(frames_path, 'frame%05d.png'),    # Path to input frames
            os.path.join(frames_path, 'video.mp4')    # Path to store output video
        ]
    )


  


  When running it from a Jupyter notebook over 2500 PNG files (RGBA, ca. 600-700 kB each, 9000 x 13934 pixels), CPU usage briefly peaks to 100% before dropping to 0%, while memory usage quickly saturates to 100% and stays there, slowing the system down almost to a freeze, so I need to terminate ffmpeg from the task manager :

  


  Screenshot

  


  The generated video file has a size of only 48 bytes and contains just a black frame when viewed in the VLC player.

  


  This is the ffmpeg log output :

  


  ffmpeg started on 2022-08-05 at 17:17:55
Report written to "ffmpeg-20220805-171755.log"
Log level: 48
Command line:
"C:\\Users\\Username\\Desktop\\folder\\ffmpeg" -y -report -framerate 60 -i "C:\\Users\\Username\\Desktop\\e\\frame%05d.png" "C:\\Users\\Username\\Desktop\\e\\video.mp4"
ffmpeg version 2022-07-14-git-882aac99d2-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --ena  libavutil      57. 29.100 / 57. 29.100
  libavcodec     59. 38.100 / 59. 38.100
  libavformat    59. 28.100 / 59. 28.100
  libavdevice    59.  8.100 / 59.  8.100
  libavfilter     8. 45.100 /  8. 45.100
  libswscale      6.  8.100 /  6.  8.100
  libswresample   4.  8.100 /  4.  8.100
  libpostproc    56.  7.100 / 56.  7.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-framerate' ... matched as AVOption 'framerate' with argument '60'.
Reading option '-i' ... matched as input url with argument 'C:\Users\Username\Desktop\e\frame%05d.png'.
Reading option 'C:\Users\Username\Desktop\e\video.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url C:\Users\Username\Desktop\e\frame%05d.png.
Successfully parsed a group of options.
Opening an input file: C:\Users\Username\Desktop\e\frame%05d.png.
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00000.png' for reading
[file @ 0000000000425680] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042d800] Statistics: 668318 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00001.png' for reading
[file @ 000000000042dac0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042d6c0] Statistics: 668371 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00002.png' for reading
[file @ 000000000042d6c0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042dac0] Statistics: 669177 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00003.png' for reading
[file @ 000000000042dac0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000437a40] Statistics: 684594 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00004.png' for reading
[file @ 0000000000437a40] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000437c00] Statistics: 703014 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00005.png' for reading
[file @ 0000000000437c00] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000437d00] Statistics: 721604 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00006.png' for reading
[file @ 0000000000437cc0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000437f40] Statistics: 739761 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00007.png' for reading
[file @ 0000000000437f40] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000438040] Statistics: 757327 bytes read, 0 seeks
[image2 @ 000000000041ff80] Probe buffer size limit of 5000000 bytes reached
Input #0, image2, from 'C:\Users\Username\Desktop\e\frame%05d.png':
  Duration: 00:00:41.67, start: 0.000000, bitrate: N/A
  Stream #0:0, 8, 1/60: Video: png, rgba(pc), 9000x13934 [SAR 29528:29528 DAR 4500:6967], 60 fps, 60 tbr, 60 tbn
Successfully opened the file.
Parsing a group of options: output url C:\Users\Username\Desktop\e\video.mp4.
Successfully parsed a group of options.
Opening an output file: C:\Users\Username\Desktop\e\video.mp4.
[file @ 000000002081e3c0] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
detected 12 logical cores
Stream mapping:
  Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00008.png' for reading
[file @ 00000000024ad980] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 00000000004379c0] Statistics: 767857 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00009.png' for reading
[file @ 000000000042d600] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 00000000004379c0] Statistics: 774848 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00010.png' for reading
[file @ 00000000004379c0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042da00] Statistics: 787178 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00011.png' for reading
[file @ 00000000004379c0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042da00] Statistics: 797084 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00012.png' for reading
[file @ 0000000000437a80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042da00] Statistics: 802870 bytes read, 0 seeks
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'video_size' to value '9000x13934'
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'pix_fmt' to value '26'
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'time_base' to value '1/60'
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'pixel_aspect' to value '29528/29528'
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'frame_rate' to value '60/1'
[graph 0 input from stream 0:0 @ 00000000208bf800] w:9000 h:13934 pixfmt:rgba tb:1/60 fr:60/1 sar:29528/29528
[format @ 00000000025ef840] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16|nv21|yuv420p10le|yuv422p10le|yuv444p10le|nv20le|gray|gray10le'
[auto_scale_0 @ 00000000025efe40] w:iw h:ih flags:'' interl:0
[format @ 00000000025ef840] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 000000000042da00] query_formats: 4 queried, 3 merged, 1 already done, 0 delayed
[auto_scale_0 @ 00000000025efe40] picking yuv444p out of 13 ref:rgba alpha:1
[auto_scale_0 @ 00000000025efe40] w:9000 h:13934 fmt:rgba sar:29528/29528 -> w:9000 h:13934 fmt:yuv444p sar:1/1 flags:0x0
[auto_scale_0 @ 00000000025efe40] w:9000 h:13934 fmt:rgba sar:29528/29528 -> w:9000 h:13934 fmt:yuv444p sar:1/1 flags:0x0
[auto_scale_0 @ 00000000025efe40] w:9000 h:13934 fmt:rgba sar:29528/29528 -> w:9000 h:13934 fmt:yuv444p sar:1/1 flags:0x0
[auto_scale_0 @ 00000000025efe40] w:9000 h:13934 fmt:rgba sar:29528/29528 -> w:9000 h:13934 fmt:yuv444p sar:1/1 flags:0x0
[libx264 @ 000000002081d280] using mv_range_thread = 376
[libx264 @ 000000002081d280] using SAR=1/1
[libx264 @ 000000002081d280] frame MB size (563x871) > level limit (139264)
[libx264 @ 000000002081d280] DPB size (4 frames, 1961492 mbs) > level limit (1 frames, 696320 mbs)
[libx264 @ 000000002081d280] MB rate (29422380) > level limit (16711680)
[libx264 @ 000000002081d280] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 000000002081d280] profile High 4:4:4 Predictive, level 6.2, 4:4:4, 8-bit
[libx264 @ 000000002081d280] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'C:\Users\Username\Desktop\e\video.mp4':
  Metadata:
    encoder         : Lavf59.28.100
  Stream #0:0, 0, 1/15360: Video: h264 (avc1 / 0x31637661), yuv444p(tv, progressive), 9000x13934 [SAR 1:1 DAR 4500:6967], q=2-31, 60 fps, 15360 tbn
    Metadata:
      encoder         : Lavc59.38.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Clipping frame in rate conversion by 0.000008
frame=    1 fps=0.8 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00013.png' for reading
[file @ 000000000a6a2180] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 810395 bytes read, 0 seeks
frame=    2 fps=0.8 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00014.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 818213 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00015.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 817936 bytes read, 0 seeks
frame=    4 fps=1.2 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00016.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 817014 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00017.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 828088 bytes read, 0 seeks
frame=    6 fps=1.5 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00018.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 831007 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00019.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 845203 bytes read, 0 seeks
frame=    8 fps=1.7 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00020.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 851548 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00021.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 847629 bytes read, 0 seeks
frame=   10 fps=1.8 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00022.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 860169 bytes read, 0 seeks
frame=   11 fps=1.4 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00023.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 857243 bytes read, 0 seeks
frame=   12 fps=1.2 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00024.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 835155 bytes read, 0 seeks


  


  What is the problem ?

  


  By the way, the color model of the image files was confirmed by doing

  


  from PIL import Image


img = Image.open('C:\\Users\\EPI-SMLM\\Desktop\\e\\frame00000.png')
img.mode
-------------------------------------------------------------------
C:\Program Files\Python38\lib\site-packages\PIL\Image.py:3035: DecompressionBombWarning: Image size (125406000 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
  warnings.warn(

'RGBA'


  


  The "decompression bomb warning" appears to be a false alarm/bug.

  


  UPDATE : I can confirm that this happens even when there are only 50 image files, i.e. 50 x 700 kB = 35 MB in total size. ffmpeg still gobbles up all available memory (almost 60 GB of private bytes !!!).

  


  And it also happens if ffmpeg is run from the command line.

  


  This must be a bug !

  


• Libav (ffmpeg) copying decoded video timestamps to encoder

  31 octobre 2016, par Jason C

  I am writing an application that decodes a single video stream from an input file (any codec, any container), does a bunch of image processing, and encodes the results to an output file (single video stream, Quicktime RLE, MOV). I am using ffmpeg’s libav 3.1.5 (Windows build for now, but the application will be cross-platform).

  There is a 1:1 correspondence between input and output frames and I want the frame timing in the output to be identical to the input. I am having a really, really hard time accomplishing this. So my general question is : How do I reliably (as in, in all cases of inputs) set the output frame timing identical to the input ?

  It took me a very long time to slog through the API and get to the point I am at now. I put together a minimal test program to work with :

  #include <cstdio>
  
  
  
  extern "C" {
  
  #include <libavcodec></libavcodec>avcodec.h>
  
  #include <libavformat></libavformat>avformat.h>
  
  #include <libavutil></libavutil>avutil.h>
  
  #include <libavutil></libavutil>imgutils.h>
  
  #include <libswscale></libswscale>swscale.h>
  
  }
  
  
  
  using namespace std;
  
  
  
  
  
  struct DecoderStuff {
  
      AVFormatContext *formatx;
  
      int nstream;
  
      AVCodec *codec;
  
      AVStream *stream;
  
      AVCodecContext *codecx;
  
      AVFrame *rawframe;
  
      AVFrame *rgbframe;
  
      SwsContext *swsx;
  
  };
  
  
  
  
  
  struct EncoderStuff {
  
      AVFormatContext *formatx;
  
      AVCodec *codec;
  
      AVStream *stream;
  
      AVCodecContext *codecx;
  
  };
  
  
  
  
  
  template <typename t="t">
  
  static void dump_timebase (const char *what, const T *o) {
  
      if (o)
  
          printf("%s timebase: %d/%d\n", what, o->time_base.num, o->time_base.den);
  
      else
  
          printf("%s timebase: null object\n", what);
  
  }
  
  
  
  
  
  // reads next frame into d.rawframe and d.rgbframe. returns false on error/eof.
  
  static bool read_frame (DecoderStuff &amp;d) {
  
  
  
      AVPacket packet;
  
      int err = 0, haveframe = 0;
  
  
  
      // read
  
      while (!haveframe &amp;&amp; err >= 0 &amp;&amp; ((err = av_read_frame(d.formatx, &amp;packet)) >= 0)) {
  
         if (packet.stream_index == d.nstream) {
  
             err = avcodec_decode_video2(d.codecx, d.rawframe, &amp;haveframe, &amp;packet);
  
         }
  
         av_packet_unref(&amp;packet);
  
      }
  
  
  
      // error output
  
      if (!haveframe &amp;&amp; err != AVERROR_EOF) {
  
          char buf[500];
  
          av_strerror(err, buf, sizeof(buf) - 1);
  
          buf[499] = 0;
  
          printf("read_frame: %s\n", buf);
  
      }
  
  
  
      // convert to rgb
  
      if (haveframe) {
  
          sws_scale(d.swsx, d.rawframe->data, d.rawframe->linesize, 0, d.rawframe->height,
  
                    d.rgbframe->data, d.rgbframe->linesize);
  
      }
  
  
  
      return haveframe;
  
  
  
  }
  
  
  
  
  
  // writes an output frame, returns false on error.
  
  static bool write_frame (EncoderStuff &amp;e, AVFrame *inframe) {
  
  
  
      // see note in so post about outframe here
  
      AVFrame *outframe = av_frame_alloc();
  
      outframe->format = inframe->format;
  
      outframe->width = inframe->width;
  
      outframe->height = inframe->height;
  
      av_image_alloc(outframe->data, outframe->linesize, outframe->width, outframe->height,
  
                     AV_PIX_FMT_RGB24, 1);
  
      //av_frame_copy(outframe, inframe);
  
      static int count = 0;
  
      for (int n = 0; n &lt; outframe->width * outframe->height; ++ n) {
  
          outframe->data[0][n*3+0] = ((n+count) % 100) ? 0 : 255;
  
          outframe->data[0][n*3+1] = ((n+count) % 100) ? 0 : 255;
  
          outframe->data[0][n*3+2] = ((n+count) % 100) ? 0 : 255;
  
      }
  
      ++ count;
  
  
  
      AVPacket packet;
  
      av_init_packet(&amp;packet);
  
      packet.size = 0;
  
      packet.data = NULL;
  
  
  
      int err, havepacket = 0;
  
      if ((err = avcodec_encode_video2(e.codecx, &amp;packet, outframe, &amp;havepacket)) >= 0 &amp;&amp; havepacket) {
  
          packet.stream_index = e.stream->index;
  
          err = av_interleaved_write_frame(e.formatx, &amp;packet);
  
      }
  
  
  
      if (err &lt; 0) {
  
          char buf[500];
  
          av_strerror(err, buf, sizeof(buf) - 1);
  
          buf[499] = 0;
  
          printf("write_frame: %s\n", buf);
  
      }
  
  
  
      av_packet_unref(&amp;packet);
  
      av_freep(&amp;outframe->data[0]);
  
      av_frame_free(&amp;outframe);
  
  
  
      return err >= 0;
  
  
  
  }
  
  
  
  
  
  int main (int argc, char *argv[]) {
  
  
  
      const char *infile = "wildlife.wmv";
  
      const char *outfile = "test.mov";
  
      DecoderStuff d = {};
  
      EncoderStuff e = {};
  
  
  
      av_register_all();
  
  
  
      // decoder
  
      avformat_open_input(&amp;d.formatx, infile, NULL, NULL);
  
      avformat_find_stream_info(d.formatx, NULL);
  
      d.nstream = av_find_best_stream(d.formatx, AVMEDIA_TYPE_VIDEO, -1, -1, &amp;d.codec, 0);
  
      d.stream = d.formatx->streams[d.nstream];
  
      d.codecx = avcodec_alloc_context3(d.codec);
  
      avcodec_parameters_to_context(d.codecx, d.stream->codecpar);
  
      avcodec_open2(d.codecx, NULL, NULL);
  
      d.rawframe = av_frame_alloc();
  
      d.rgbframe = av_frame_alloc();
  
      d.rgbframe->format = AV_PIX_FMT_RGB24;
  
      d.rgbframe->width = d.codecx->width;
  
      d.rgbframe->height = d.codecx->height;
  
      av_frame_get_buffer(d.rgbframe, 1);
  
      d.swsx = sws_getContext(d.codecx->width, d.codecx->height, d.codecx->pix_fmt,
  
                              d.codecx->width, d.codecx->height, AV_PIX_FMT_RGB24,
  
                              SWS_POINT, NULL, NULL, NULL);
  
      //av_dump_format(d.formatx, 0, infile, 0);
  
      dump_timebase("in stream", d.stream);
  
      dump_timebase("in stream:codec", d.stream->codec); // note: deprecated
  
      dump_timebase("in codec", d.codecx);
  
  
  
      // encoder
  
      avformat_alloc_output_context2(&amp;e.formatx, NULL, NULL, outfile);
  
      e.codec = avcodec_find_encoder(AV_CODEC_ID_QTRLE);
  
      e.stream = avformat_new_stream(e.formatx, e.codec);
  
      e.codecx = avcodec_alloc_context3(e.codec);
  
      e.codecx->bit_rate = 4000000; // arbitrary for qtrle
  
      e.codecx->width = d.codecx->width;
  
      e.codecx->height = d.codecx->height;
  
      e.codecx->gop_size = 30; // 99% sure this is arbitrary for qtrle
  
      e.codecx->pix_fmt = AV_PIX_FMT_RGB24;
  
      e.codecx->time_base = d.stream->time_base; // ???
  
      e.codecx->flags |= (e.formatx->flags &amp; AVFMT_GLOBALHEADER) ? AV_CODEC_FLAG_GLOBAL_HEADER : 0;
  
      avcodec_open2(e.codecx, NULL, NULL);
  
      avcodec_parameters_from_context(e.stream->codecpar, e.codecx); 
  
      //av_dump_format(e.formatx, 0, outfile, 1);
  
      dump_timebase("out stream", e.stream);
  
      dump_timebase("out stream:codec", e.stream->codec); // note: deprecated
  
      dump_timebase("out codec", e.codecx);
  
  
  
      // open file and write header
  
      avio_open(&amp;e.formatx->pb, outfile, AVIO_FLAG_WRITE); 
  
      avformat_write_header(e.formatx, NULL);
  
  
  
      // frames
  
      while (read_frame(d) &amp;&amp; write_frame(e, d.rgbframe))
  
          ;
  
  
  
      // write trailer and close file
  
      av_write_trailer(e.formatx);
  
      avio_closep(&amp;e.formatx->pb); 
  
  
  
  }
  
  </typename></cstdio>

  A few notes about that :

  • Since all of my attempts at frame timing so far have failed, I’ve removed almost all timing-related stuff from this code to start with a clean slate.
  • Almost all error checking and cleanup omitted for brevity.
  • The reason I allocate a new output frame with a new buffer in write_frame, rather than using inframe directly, is because this is more representative of what my real application is doing. My real app also uses RGB24 internally, hence the conversions here.
  • The reason I generate a weird pattern in outframe, rather than using e.g. av_copy_frame, is because I just wanted a test pattern that compressed well with Quicktime RLE (my test input ends up generating a 1.7GB output file otherwise).
  • The input video I am using, "wildlife.wmv", can be found here. I’ve hard-coded the filenames.
  • I am aware that avcodec_decode_video2 and avcodec_encode_video2 are deprecated, but don’t care. They work fine, I’ve already struggled too much getting my head around the latest version of the API, ffmpeg changes their API with nearly every release, and I really don’t feel like dealing with avcodec_send_* and avcodec_receive_* right now.
  • I think I’m supposed to be finishing off by passing a NULL frame to avcodec_encode_video2 to flush some buffers or something but I’m a bit confused about that. Unless somebody feels like explaining that let’s ignore it for now, it’s a separate question. The docs are as vague about this point as they are about everything else.
  • My test input file’s frame rate is 29.97.

  ---

  Now, as for my current attempts. The following timing related fields are present in the above code, with details/confusion in bold. There’s a lot of them, because the API is mind-bogglingly convoluted :

  • main: d.stream->time_base : Input video stream time base. For my test input file this is 1/1000.
  • main: d.stream->codec->time_base : Not sure what this is (I never could make sense of why AVStream has an AVCodecContext field when you always use your own new context anyways) and also the codec field is deprecated. For my test input file this is 1/1000.
  • main: d.codecx->time_base : Input codec context time-base. For my test input file this is 0/1. Am I supposed to set it ?
  • main: e.stream->time_base : Time base of the output stream I create. What do I set this to ?
  • main: e.stream->codec->time_base : Time base of the deprecated and mysterious codec field of the output stream I create. Do I set this to anything ?
  • main: e.codecx->time_base : Time base of the encoder context I create. What do I set this to ?
  • read_frame: packet.dts : Decoding timestamp of packet read.
  • read_frame: packet.pts : Presentation timestamp of packet read.
  • read_frame: packet.duration : Duration of packet read.
  • read_frame: d.rawframe->pts : Presentation timestamp of raw frame decoded. This is always 0. Why isn’t it read by the decoder...?
  • read_frame: d.rgbframe->pts / write_frame: inframe->pts : Presentation timestamp of decoded frame converted to RGB. Not set to anything currently.
  • read_frame: d.rawframe->pkt_* : Fields copied from packet, discovered after reading this post. They are set correctly but I don’t know if they are useful.
  • write_frame: outframe->pts : Presentation timestamp of frame being encoded. Should I set this to something ?
  • write_frame: outframe->pkt_* : Timing fields from a packet. Should I set these ? They seem to be ignored by the encoder.
  • write_frame: packet.dts : Decoding timestamp of packet being encoded. What do I set it to ?
  • write_frame: packet.pts : Presentation timestamp of packet being encoded. What do I set it to ?
  • write_frame: packet.duration : Duration of packet being encoded. What do I set it to ?

  I have tried the following, with the described results. Note that inframe is d.rgbframe :

  1.  
     • Init e.stream->time_base = d.stream->time_base
     • Init e.codecx->time_base = d.codecx->time_base
     • Set d.rgbframe->pts = packet.dts in read_frame
     • Set outframe->pts = inframe->pts in write_frame
     • Result : Warning that encoder time base is not set (since d.codecx->time_base was 0/1), seg fault.
  2.  
     • Init e.stream->time_base = d.stream->time_base
     • Init e.codecx->time_base = d.stream->time_base
     • Set d.rgbframe->pts = packet.dts in read_frame
     • Set outframe->pts = inframe->pts in write_frame
     • Result : No warnings, but VLC reports frame rate as 480.048 (no idea where this number came from) and file plays too fast. Also the encoder sets all the timing fields in packet to 0, which was not what I expected. (Edit : Turns out this is because av_interleaved_write_frame, unlike av_write_frame, takes ownership of the packet and swaps it with a blank one, and I was printing the values after that call. So they are not ignored.)
  3.  
     • Init e.stream->time_base = d.stream->time_base
     • Init e.codecx->time_base = d.stream->time_base
     • Set d.rgbframe->pts = packet.dts in read_frame
     • Set any of pts/dts/duration in packet in write_frame to anything.
     • Result : Warnings about packet timestamps not set. Encoder seems to reset all packet timing fields to 0, so none of this has any effect.
  4.  
     • Init e.stream->time_base = d.stream->time_base
     • Init e.codecx->time_base = d.stream->time_base
     • I found these fields, pkt_pts, pkt_dts, and pkt_duration in AVFrame after reading this post, so I tried copying those all the way through to outframe.
     • Result : Really had my hopes up, but ended up with same results as attempt 3 (packet timestamp not set warning, incorrect results).

  I tried various other hand-wavey permutations of the above and nothing worked. What I want to do is create an output file that plays back with the same timing and frame rate as the input (29.97 constant frame rate in this case).

  So how do I do this ? Of the zillions of timing related fields here, what do I do to make the output be the same as the input ? And how do I do it in such a way that handles arbitrary video input formats that may store their time stamps and time bases in different places ? I need this to always work.

  ---

  For reference, here is a table of all the packet and frame timestamps read from the video stream of my test input file, to give a sense of what my test file looks like. None of the input packet pts’ are set, same with frame pts, and for some reason the duration of the first 108 frames is 0. VLC plays the file fine and reports the frame rate as 29.9700089 :

  • Table is here since it was too large for this post.