Sur d’autres sites (5716)

• Laravel Process() with FFMPEG times out for any video that is even kind of large

  16 août 2024, par Hunter

  I am creating an application that lets users upload video files, and once they do it will convert them all to .mp4's (it does some other things, but since this first part of the code is breaking before it gets a chance to reach the other stuff, and the process for each is pretty similar, lets just discuss this issue).

  



  Once a user submits, it gets sent to GameController@store. Below is a code sample (the one that keeps getting sent back to me as problematic) from the controller :

  



  $process = new Process(env('APP_FFMPEG', 'ffmpeg') . ' -i ' . $fileURL . ' -f mp4 -vcodec libx264 -preset medium -profile:v main -c:a aac -movflags +faststart ' . $newFileName . ' -hide_banner');
              $process->setTimeout(360);
              $process->setIdleTimeout(60);
              $process->run();
              if (!$process->isSuccessful()) {
                throw new ProcessFailedException($process);
              }


  



  I am using the Symfony Process() class to run the FFMpeg conversion. The weird thing is, when I run this with a 1 or 2 second video, it goes without a hitch. This, to me, rules out issues like path problems, permission problems, etc. Multiple file types work with this, as long as they are very short. As soon as I put in a video that is even around 1min, it returns with this :

  



  The process "/usr/local/bin/ffmpeg -i http://localhost/storage/team_1_folder/game_26/team_1_game_26_vid_1.mp4 -f mp4 -vcodec libx264 -preset medium -profile:v main -c:a aac -movflags +faststart storage/team_1_folder/game_26/convertedVideo_1.mp4 -hide_banner" exceeded the timeout of 360 seconds.


  



  Now, notice that it says 'timeout of 360 seconds'. That is because I have also edited ALL of the possible timeout constraints, as well as the file size constraints that I can think of. Here is what I have done :
  

  



  -Changed php.ini (in my MAMP installation) values :
  
—max_execution_time = 300 (default was 30)
  
—memory_limit = 1280M (default was 128)
  
—post_max_size = 800M
  
—upload_max_filesize = 3200M
  
-max_input_time=300
  

  



  -In the GameController (code above)
  
—$process->setTimeout(360) ;
  
—$process->setTimeout(360) ;
  

  



  After all that, with a big file, the thing will just stall for the 6 minutes that I have set on the timeout thing, and then send an error. Is FFmpeg just stalling, or is this a process issue ? Who knows. Errors surrounding these things are just beautifully vague.

  



  I have tagged Laravel, PHP, and FFmpeg in hopes that someone will recognize this issue. I know that Process() is sending me the error, but since it is sending me a timeout, I don't know what is at fault.

  



  I'm not ruling anything out, so hit me with ideas, even crazy ones !

  



  Edit
Here is the output of the -report. I cut some of it off because the end part gets repetitive and goes for literally 1000 lines :

  



  ffmpeg started on 2019-06-24 at 18:50:23
Report written to "ffmpeg-20190624-185023.log"
Command line:
/usr/local/bin/ffmpeg -i http://localhost/storage/team_1_folder/game_31/team_1_game_31_vid_1.mp4 -f mp4 -vcodec libx264 -preset medium -profile:v main -c:a aac -movflags +faststart storage/team_1_folder/game_31/convertedVideo_1.mp4 -report
ffmpeg version 4.1.3 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.3_1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Splitting the commandline.
Reading option '-i' ... matched as input url with argument 'http://localhost/storage/team_1_folder/game_31/team_1_game_31_vid_1.mp4'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'mp4'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'libx264'.
Reading option '-preset' ... matched as AVOption 'preset' with argument 'medium'.
Reading option '-profile:v' ... matched as option 'profile' (set profile) with argument 'main'.
Reading option '-c:a' ... matched as option 'c' (codec name) with argument 'aac'.
Reading option '-movflags' ... matched as AVOption 'movflags' with argument '+faststart'.
Reading option 'storage/team_1_folder/game_31/convertedVideo_1.mp4' ... matched as output url.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url http://localhost/storage/team_1_folder/game_31/team_1_game_31_vid_1.mp4.
Successfully parsed a group of options.
Opening an input file: http://localhost/storage/team_1_folder/game_31/team_1_game_31_vid_1.mp4.
[NULL @ 0x7f9ad480ae00] Opening 'http://localhost/storage/team_1_folder/game_31/team_1_game_31_vid_1.mp4' for reading
[http @ 0x7f9ad3c6b540] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0x7f9ad3e00400] Original list of addresses:
[tcp @ 0x7f9ad3e00400] Address ::1 port 80
[tcp @ 0x7f9ad3e00400] Address 127.0.0.1 port 80
[tcp @ 0x7f9ad3e00400] Interleaved list of addresses:
[tcp @ 0x7f9ad3e00400] Address ::1 port 80
[tcp @ 0x7f9ad3e00400] Address 127.0.0.1 port 80
[tcp @ 0x7f9ad3e00400] Starting connection attempt to ::1 port 80
[tcp @ 0x7f9ad3e00400] Successfully connected to ::1 port 80
[http @ 0x7f9ad3c6b540] request: GET /storage/team_1_folder/game_31/team_1_game_31_vid_1.mp4 HTTP/1.1
User-Agent: Lavf/58.20.100
Accept: */*
Range: bytes=0-
Connection: close
Host: localhost
Icy-MetaData: 1


[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] ISO: File Type Major Brand: isom
[tcp @ 0x7f9ad3c6b840] Original list of addresses:
[tcp @ 0x7f9ad3c6b840] Address ::1 port 80
[tcp @ 0x7f9ad3c6b840] Address 127.0.0.1 port 80
[tcp @ 0x7f9ad3c6b840] Interleaved list of addresses:
[tcp @ 0x7f9ad3c6b840] Address ::1 port 80
[tcp @ 0x7f9ad3c6b840] Address 127.0.0.1 port 80
[tcp @ 0x7f9ad3c6b840] Starting connection attempt to ::1 port 80
[tcp @ 0x7f9ad3c6b840] Successfully connected to ::1 port 80
[http @ 0x7f9ad3c6b540] request: GET /storage/team_1_folder/game_31/team_1_game_31_vid_1.mp4 HTTP/1.1
User-Agent: Lavf/58.20.100
Accept: */*
Range: bytes=170169585-
Connection: close
Host: localhost
Icy-MetaData: 1


[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] Processing st: 0, edit list 0 - media time: 0, duration: 10584600
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] stts: 0 ctts: 0, ctts_index: 0, ctts_count: 10575
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] stts: 1 ctts: 4002, ctts_index: 1, ctts_count: 10575
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] stts: 1000 ctts: 0, ctts_index: 2, ctts_count: 10575
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] stts: 2001 ctts: 0, ctts_index: 3, ctts_count: 10575
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] stts: 3002 ctts: 0, ctts_index: 4, ctts_count: 10575
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] stts: 4003 ctts: 4004, ctts_index: 5, ctts_count: 10575
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] stts: 5004 ctts: 0, ctts_index: 6, ctts_count: 10575


  



  Edit 2
I also found this towards the bottom :

  



  [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] stts: 10582571 ctts: 0, ctts_index: 10573, ctts_count: 10575
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] stts: 10583572 ctts: 1001, ctts_index: 10574, ctts_count: 10575
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] Setting codecpar->delay to 1 for stream st: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] Processing st: 1, edit list 0 - media time: 0, duration: 15563816
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] Before avformat_find_stream_info() pos: 170509020 bytes read:372203 seeks:1 nb_streams:2
[h264 @ 0x7f9ad4811200] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x7f9ad4811200] nal_unit_type: 8(PPS), nal_ref_idc: 3
[tcp @ 0x7f9ad3c72d40] Original list of addresses:
[tcp @ 0x7f9ad3c72d40] Address ::1 port 80
[tcp @ 0x7f9ad3c72d40] Address 127.0.0.1 port 80
[tcp @ 0x7f9ad3c72d40] Interleaved list of addresses:
[tcp @ 0x7f9ad3c72d40] Address ::1 port 80
[tcp @ 0x7f9ad3c72d40] Address 127.0.0.1 port 80
[tcp @ 0x7f9ad3c72d40] Starting connection attempt to ::1 port 80
[tcp @ 0x7f9ad3c72d40] Successfully connected to ::1 port 80
[http @ 0x7f9ad3c6b540] request: GET /storage/team_1_folder/game_31/team_1_game_31_vid_1.mp4 HTTP/1.1
User-Agent: Lavf/58.20.100
Accept: */*
Range: bytes=48-
Connection: close
Host: localhost
Icy-MetaData: 1


[h264 @ 0x7f9ad4811200] nal_unit_type: 5(IDR), nal_ref_idc: 1
[h264 @ 0x7f9ad4811200] Format yuv420p chosen by get_format().
[h264 @ 0x7f9ad4811200] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 0x7f9ad4811200] no picture 
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f9ad480ae00] After avformat_find_stream_info() pos: 308454 bytes read:680609 seeks:2 frames:2
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'http://localhost/storage/team_1_folder/game_31/team_1_game_31_vid_1.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
  Duration: 00:05:52.92, start: 0.000000, bitrate: 3865 kb/s
    Stream #0:0(und), 1, 1/30000: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 3730 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und), 1, 1/44100: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Successfully opened the file.
Parsing a group of options: output url storage/team_1_folder/game_31/convertedVideo_1.mp4.
Applying option f (force format) with argument mp4.
Applying option vcodec (force video codec ('copy' to copy stream)) with argument libx264.
Applying option profile:v (set profile) with argument main.
Applying option c:a (codec name) with argument aac.
Successfully parsed a group of options.
Opening an output file: storage/team_1_folder/game_31/convertedVideo_1.mp4.
[file @ 0x7f9ad3f02740] Setting default whitelist 'file,crypto'
Successfully opened the file.
detected 4 logical cores
[h264 @ 0x7f9ad5813a00] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x7f9ad5813a00] nal_unit_type: 8(PPS), nal_ref_idc: 3
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[graph_1_in_0_1 @ 0x7f9ad3dbcc00] Setting 'time_base' to value '1/44100'
[graph_1_in_0_1 @ 0x7f9ad3dbcc00] Setting 'sample_rate' to value '44100'
[graph_1_in_0_1 @ 0x7f9ad3dbcc00] Setting 'sample_fmt' to value 'fltp'
[graph_1_in_0_1 @ 0x7f9ad3dbcc00] Setting 'channel_layout' to value '0x3'
[graph_1_in_0_1 @ 0x7f9ad3dbcc00] tb:1/44100 samplefmt:fltp samplerate:44100 chlayout:0x3
[format_out_0_1 @ 0x7f9ad3c71040] Setting 'sample_fmts' to value 'fltp'
[format_out_0_1 @ 0x7f9ad3c71040] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
[AVFilterGraph @ 0x7f9ad3e00440] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x7f9ad5813a00] nal_unit_type: 5(IDR), nal_ref_idc: 1
[h264 @ 0x7f9ad5813a00] Format yuv420p chosen by get_format().
[h264 @ 0x7f9ad5813a00] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 0x7f9ad5813a00] no picture 
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x7f9ad5804600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x7f9ad5804c00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x7f9ad5805200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x7f9ad5805800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x7f9ad5813a00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[graph 0 input from stream 0:0 @ 0x7f9ad3dbfb80] Setting 'video_size' to value '1920x1080'
[graph 0 input from stream 0:0 @ 0x7f9ad3dbfb80] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x7f9ad3dbfb80] Setting 'time_base' to value '1/30000'
[graph 0 input from stream 0:0 @ 0x7f9ad3dbfb80] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x7f9ad3dbfb80] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x7f9ad3dbfb80] Setting 'frame_rate' to value '30000/1001'
[graph 0 input from stream 0:0 @ 0x7f9ad3dbfb80] w:1920 h:1080 pixfmt:yuv420p tb:1/30000 fr:30000/1001 sar:1/1 sws_param:flags=2
[format @ 0x7f9ad3dc0100] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16|nv21|yuv420p10le|yuv422p10le|yuv444p10le|nv20le'
[AVFilterGraph @ 0x7f9ad3c6e880] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[libx264 @ 0x7f9ad5811200] using mv_range_thread = 88
[libx264 @ 0x7f9ad5811200] using SAR=1/1
[libx264 @ 0x7f9ad5811200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7f9ad5811200] profile Main, level 4.0
[libx264 @ 0x7f9ad5811200] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'storage/team_1_folder/game_31/convertedVideo_1.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.20.100
    Stream #0:0(und), 0, 1/30000: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 29.97 fps, 30k tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(und), 0, 1/44100: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.35.100 aac
Clipping frame in rate conversion by 0.000008
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x7f9ad5804600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
Clipping frame in rate conversion by 0.000999
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x7f9ad5804c00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
Clipping frame in rate conversion by 0.000999
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5805200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5805800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5813a00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5804600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5804c00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5805200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5805800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5813a00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5804600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5804c00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5805200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5805800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5813a00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5804600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5804c00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999
[h264 @ 0x7f9ad5805200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000999


  


• ffmpeg -Video live feed

  22 juin 2019, par Rajeev

  I am using AMCREST security camera at my home. My objective is to get a live feed one of IP camera attached to my NVR to a webportal using rtsp ://My environment is Raspberry pi.

  I am able to successfully start the ffserver but conversion is failing when I am trying pass the input video and stream it to video.ffm

  I have tried various combination of parameters in the command but the below one seems to be very close where I got only one error ( av_interleaved_write_frame() : Connection reset by peer)

  $ffmpeg  -thread_queue_size 800  -i "rtsp://home:Home1234@192.168.1.32:554/cam/realmonitor?channel=4&subtype=0" -f lavfi -i aevalsrc=0  http://127.0.0.1:8090/video.ffm 

  ******* ffmpeg server configuration file content : etc/livestream.conf

  #Default port
  
  HTTPPort 8090
  
  HTTPBindAddress 0.0.0.0
  
  MaxHTTPConnections 2000
  
  MaxClients 1000
  
  MaxBandwidth 100000
  
  CustomLog -
  
  
  
  #############################################################
  
  
  
  <feed>
  
    File /tmp/video.ffm                    # this creates a temp video.ffm file where streams are read/write
  
    FileMaxSize 0.5G
  
    ACL allow localhost
  
    ACL allow 127.0.0.1
  
    ACL allow 192.168.0.0 192.168.255.255
  
  </feed>
  
  
  
  <stream stream="stream">
  
  # streaming for webm file
  
  # run : ffserver -f /etc/ffserver.conf
  
  # run : ffmpeg -i videoname.mp4 http://localhost:8090/video.ffm
  
  # error : encoder setup failed
  
    Feed video.ffm
  
    Format webm
  
  
  
  # Audio settings
  
    AudioCodec vorbis
  
    AudioBitRate 64                             # Audio bitrate
  
  
  
  # Video settings
  
    VideoCodec libvpx
  
    VideoSize 720x486                           # Video resolution
  
    VideoFrameRate 30                           # Video FPS
  
    AVOptionVideo flags +global_header          # Parameters passed to encoder
  
  
  
    AVOptionVideo cpu-used 0
  
    AVOptionVideo qmin 10                       # lower the better, min 0
  
    AVOptionVideo qmax 42                       # higher outputs bad quality, max 63
  
    AVOptionVideo quality good
  
    AVOptionAudio flags +global_header
  
    PreRoll 15
  
    StartSendOnKey
  
    VideoBitRate 400                            # Video bitrate
  
  </stream>
  
  
  
  
  
  
  
  ###########################################################################
  
  
  
  # Audio only
  
  # run ffmpeg -i audio.mp3 http://localhost:8090/audio.ffm
  
  # run http://localhost:8090/audio in vlc or browser
  
  
  
  <feed>
  
    File /tmp/audio.ffm
  
    FileMaxSize 1G
  
    ACL allow localhost
  
    ACL allow 127.0.0.1
  
    ACL allow 192.168.0.0 192.168.255.255
  
  </feed>
  
  
  
  <stream audio="audio">
  
    Feed audio.ffm
  
    Format mp2                                     #audio format
  
    AudioCodec libmp3lame                          #audio codec
  
    AudioBitRate 64                                #audio bitrate
  
    AudioChannels 1                                #audio channel, 1 for mono and 2 for stereo
  
    AudioSampleRate 44100
  
    NoVideo                                        #discard video
  
  </stream>
  
  
  
  ####################################################################
  
  #view status of ffserver
  
  <stream>
  
    Format status
  
    ACL allow localhost
  
    ACL allow 192.168.0.0 192.168.255.255
  
  </stream>
  
  
  
  
  
  
  
  # Redirect index.html to the appropriate site
  
  
  
  <redirect>
  
    URL http://www.ffmpeg.org/
  
  </redirect>

  *****output of ffmpeg server running successfully on separate console***

  $ ffserver -f /etc/livestream.conf

  ffserver version 3.2.14-1 deb9u1+rpt1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
  configuration : —prefix=/usr —extra-version=’1 deb9u1+rpt1’ —toolchain=hardened —libdir=/usr/lib/arm-linux-gnueabihf —incdir=/usr/include/arm-linux-gnueabihf —enable-gpl —disable-stripping —enable-avresample —enable-avisynth —enable-gnutls —enable-ladspa —enable-libass —enable-libbluray —enable-libbs2b —enable-libcaca —enable-libcdio —enable-libebur128 —enable-libflite —enable-libfontconfig —enable-libfreetype —enable-libfribidi —enable-libgme —enable-libgsm —enable-libmp3lame —enable-libopenjpeg —enable-libopenmpt —enable-libopus —enable-libpulse —enable-librubberband —enable-libshine —enable-libsnappy —enable-libsoxr —enable-libspeex —enable-libssh —enable-libtheora —enable-libtwolame —enable-libvorbis —enable-libvpx —enable-libwavpack —enable-libwebp —enable-libx265 —enable-libxvid —enable-libzmq —enable-libzvbi —enable-omx —enable-omx-rpi —enable-mmal —enable-openal —enable-opengl —enable-sdl2 —enable-libdc1394 —enable-libiec61883 —arch=armhf —enable-chromaprint —enable-frei0r —enable-libopencv —enable-libx264 —enable-shared
  libavutil 55. 34.101 / 55. 34.101
  libavcodec 57. 64.101 / 57. 64.101
  libavformat 57. 56.101 / 57. 56.101
  libavdevice 57. 1.100 / 57. 1.100
  libavfilter 6. 65.100 / 6. 65.100
  libavresample 3. 1. 0 / 3. 1. 0
  libswscale 4. 2.100 / 4. 2.100
  libswresample 2. 3.100 / 2. 3.100
  libpostproc 54. 1.100 / 54. 1.100
  /etc/livestream.conf:45 : Setting default value for audio sample rate = 22050. Use NoDefaults to disable it.
  /etc/livestream.conf:45 : Setting default value for audio channel count = 1. Use NoDefaults to disable it.
  /etc/livestream.conf:45 : Setting default value for video bit rate tolerance = 100000. Use NoDefaults to disable it.
  /etc/livestream.conf:45 : Setting default value for video rate control equation = tex^qComp. Use NoDefaults to disable it.
  /etc/livestream.conf:45 : Setting default value for video max rate = 13749264. Use NoDefaults to disable it.
  /etc/livestream.conf:45 : Setting default value for video buffer size = 800000. Use NoDefaults to disable it.
  Fri Jun 21 19:43:59 2019 FFserver started.

  ---

  2nd Console

  $$ffmpeg  -thread_queue_size 800  -i "rtsp://home:Home1234@192.168.1.32:554/cam/realmonitor?channel=4&subtype=0" -f lavfi -i aevalsrc=0  http://127.0.0.1:8090/video.ffm 

  *************************Output**********

  Input #0, rtsp, from 'rtsp://home:Home1234@192.168.1.32:554/cam/realmonitor?channel=4&subtype=0':
  
    Metadata:
  
      title           : Media Server
  
    Duration: N/A, start: 0.290000, bitrate: N/A
  
      Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 2304x1296 [SAR 1:1 DAR 16:9], 20 fps, 250 tbr, 90k tbn, 40 tbc
  
  Input #1, lavfi, from 'aevalsrc=0':
  
    Duration: N/A, start: 0.000000, bitrate: 2822 kb/s
  
      Stream #1:0: Audio: pcm_f64le, 44100 Hz, mono, dbl, 2822 kb/s
  
  [swscaler @ 0x256dd80] deprecated pixel format used, make sure you did set range correctly
  
  [libvpx @ 0x2564190] v1.6.1
  
  Output #0, ffm, to 'http://127.0.0.1:8090/video.ffm':
  
    Metadata:
  
      title           : Media Server
  
      creation_time   : now
  
      encoder         : Lavf57.56.101
  
      Stream #0:0: Audio: vorbis (libvorbis), 22050 Hz, mono, fltp, 64 kb/s
  
      Metadata:
  
        encoder         : Lavc57.64.101 libvorbis
  
      Stream #0:1: Video: vp8 (libvpx), yuv420p, 720x486 [SAR 6:5 DAR 16:9], q=10-42, 400 kb/s, 20 fps, 1000k tbn, 30 tbc
  
      Metadata:
  
        encoder         : Lavc57.64.101 libvpx
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 800000 vbv_delay: -1
  
  Stream mapping:
  
    Stream #1:0 -> #0:0 (pcm_f64le (native) -> vorbis (libvorbis))
  
    Stream #0:0 -> #0:1 (h264 (native) -> vp8 (libvpx))
  
  Press [q] to stop, [?] for help
  
  frame=    2 fps=1.2 q=0.0 size=       8kB time=00:00:00.03 bitrate=1966.0kbits/s dup=1 drop=0 speed=0.0av_interleaved_write_frame(): Connection reset by peer
  
  Error writing trailer of http://127.0.0.1:8090/video.ffm: Connection reset by peer frame=    2 fps=1.1 q=0.0 Lsize=      40kB time=00:00:00.03 bitrate=9830.2kbits/s dup=1 drop=0 speed=0.0189x    
  
  video:29kB audio:0kB subtitle:0kB other streams:0kB global headers:4kB muxing overhead: 36.183796%
  
  Conversion failed

  My expectation is that ffmpeg will start writing data to video.ffm located in /tmp directory so that I can read the data from our browser or vlc media player by just entering the following link
  http://localhost:8090/stream

  ****** Update after 2 hr******

  I made slight change to the command parameter and my out changed as well.What it looks like the temporary video file that got generated in /tmp folder is not getting consumes by video.ffm ( I may be wrong in my analysis)

  ffmpeg  -thread_queue_size 1200  -i "rtsp://home:Home1234@192.168.1.32:554/cam/realmonitor?channel=4&subtype=0" -f lavfi -i aevalsrc=0 -override_ffserver  http://127.0.0.1:8090/video.ffm

  ***************** Output ****

  Past duration 0.660332 too large     376kB time=00:00:01.03 bitrate=2978.9kbits/s dup=3 drop=5 speed=0.171x    
  
  Past duration 0.637352 too large     840kB time=00:00:03.72 bitrate=1847.5kbits/s dup=14 drop=5 speed=0.213x    
  
  Past duration 0.678307 too large
  
  Past duration 0.713280 too large    1180kB time=00:00:05.52 bitrate=1749.0kbits/s dup=21 drop=5 speed=0.218x    
  
  Past duration 0.901085 too large    1372kB time=00:00:06.72 bitrate=1670.4kbits/s dup=26 drop=5 speed=0.22x     
  
  Past duration 0.948051 too large    2456kB time=00:00:12.97 bitrate=1551.1kbits/s dup=51 drop=5 speed=0.226x    
  
  [rtsp @ 0x1fd7670] Thread message queue blocking; consider raising the thread_queue_size option (current value: 1200)
  
  Past duration 0.713280 too large    3336kB time=00:00:17.64 bitrate=1549.0kbits/s dup=70 drop=5 speed=0.228x    
  
  [rtsp @ 0x1fd7670] max delay reached. need to consume packetbitrate=1537.0kbits/s dup=76 drop=5 speed=0.229x    
  
  [rtsp @ 0x1fd7670] RTP: missed 30 packets
  
  [rtsp @ 0x1fd7670] max delay reached. need to consume packetbitrate=1533.3kbits/s dup=78 drop=5 speed=0.228x    
  
  [rtsp @ 0x1fd7670] RTP: missed 134 packets
  
  [rtsp @ 0x1fd7670] max delay reached. need to consume packetbitrate=1539.5kbits/s dup=82 drop=5 speed=0.229x    
  
  [rtsp @ 0x1fd7670] RTP: missed 111 packets
  
  [rtsp @ 0x1fd7670] max delay reached. need to consume packetbitrate=1526.3kbits/s dup=84 drop=5 speed=0.229x    
  
  [rtsp @ 0x1fd7670] RTP: missed 19 packets
  
  [rtsp @ 0x1fd7670] max delay reached. need to consume packetbitrate=1521.7kbits/s dup=92 drop=5 speed=0.23x     
  
  [rtsp @ 0x1fd7670] RTP: missed 626 packets
  
  Past duration 0.651329 too large    4408kB time=00:00:23.64 bitrate=1527.0kbits/s dup=94 drop=5 speed=0.23x    
  
  [rtsp @ 0x1fd7670] max delay reached. need to consume packetbitrate=1516.7kbits/s dup=94 drop=5 speed=0.23x    
  
  [rtsp @ 0x1fd7670] RTP: missed 134 packets
  
  [rtsp @ 0x1fd7670] max delay reached. need to consume packetbitrate=1521.3kbits/s dup=98 drop=5 speed=0.231x    
  
  [rtsp @ 0x1fd7670] RTP: missed 123 packets
  
  Past duration 0.633354 too large    4608kB time=00:00:24.98 bitrate=1511.0kbits/s dup=99 drop=5 speed=0.23x     
  
  [rtsp @ 0x1fd7670] max delay reached. need to consume packetbitrate=1523.4kbits/s dup=99 drop=5 speed=0.231x    

• Exceeded GA’s 10M hits data limit, now what ?

  21 juin 2019, par Joselyn Khor
  Exceeded GA’s 10M hits data limit, now what ? Matomo has the answers

  “Your data volume (1XXM hits) exceeds the limit of 10M hits per month as outlined in our Terms of Service. If you continue to exceed the limit, we will stop processing new data on XXX 21, 2019. Learn more about possible solutions.”

  Yikes. Alarm bells were ringing when a Google Analytics free user came to us faced with this notice. Let’s call him ‘Mark’. Mark had reached the limits on the data he could collect through Google Analytics and was shocked by the limited options available to fix the problem, without blowing the budget. The thoughts racing through his head were :

  • “What happens to all my data ?”
  • “What if Google starts charging USD150K now ?”

  Then he came across Matomo and decided to get in touch with our support team …

  “Can you fix this issue ?” he asked us.

  “Absolutely !” we said.

  We’ll get back to helping Mark in a minute. For now let’s go over why this was such a dilemma for him.

  In order to resolve this data limits issue, one of the solutions was for him to upgrade to Google Analytics 360, which meant shelling out USD150,000 per year for their 1 billion hits per month option. Going from free to USD150,000 was too much of a stretch for a growing company.

  “Your data volume (1XXM hits) exceeds the limit of 10M hits per month …”, what did this message mean ?

  With the free version, Mark could collect up to 10 million “hits” per month, per account. Going over meant Google Analytics could stop collecting any more data for free as outlined in their Terms.

  Google Analytics’ Terms of Service (2018, sec. 2) states, “Subject to Section 15, the Service is provided without charge to You for up to 10 million Hits per month per account.”[1]

  In general, what’s a "hit" ?

  Data being sent to Google Analytics. It can be a transaction, event, social interaction or pageview - these all produce what Google calls a “hit”.

  

  And their Analytics Help Data Limits (n.d.) support page makes clear that : “If a property sends more hits per month to Analytics than allowed by the Analytics Terms of Service, there is no assurance that the excess hits will be processed. If the property’s hit volume exceeds this limit, a warning may be displayed in the user interface and you may be prevented from accessing reports.”[2]

  

  Possible solutions

  So the possible solutions given by Google Analytics’ Data Limits support page were (also shown in image below) :

  • To pay USD150K to upgrade to Google Analytics 360
  • To send fewer hits by setting up sampling
  • Or choose the slightly less relevant option to upgrade mobile app tracking to Google Analytics for Firebase.

  Without the means to pay, the free version was fast becoming inaccessible for Mark as he was facing a future where he risked no longer having access to up-to-date data used in his business’ reporting.

  Mark was facing a problem that potentially didn’t have a cost-effective solution.

  

  So what can you really do about it ?

  This is where we can help provide some assistance. If you’re reading this article, we’ll assume you can relate to Mark and share with you the advice on options we gave him.

  Options :

  One option posed by Google is for you to send fewer hits by auditing your data collection processes

  If you really don’t have the budget, you’ll need to reassess your data collection priorities and go over your strategies to see what is necessary to track, and what isn’t.

  • Make sure you know what you’re tracking and why. Look at what websites are being tracked by Google and into what properties.
  • Go through what data you’re tracking and decide what is or isn’t of value.
  • Set up data sampling, this however, will lead to inaccurate data.

  From here you can start to course correct. If you’ve found data you’re not using for analysis, get rid of these events/pageviews in your Google Analytics.

  But the limitations here are that eventually, you’re going to run out of irrelevant metrics and everything you’re tracking will be essential. So you’ll hit another brick wall and return to the same situation.

  Option 2 – Ignore and continue using the free version of Google Analytics

  With this option, you’ll have to bear the business risks involved by basing decisions off of analytics reports that may or may not be updated. In this case, you may still get contacted about exceeding the limits. As the free service is provided for only up to 10 million hits, once you’ve gone over them, you’re violating what’s stipulated in the Terms of Service. 

  There’s also the warning that “… you may be prevented from accessing reports” (Data limits, n.d.). So while we may not know for certain what Google Analytics will do, in this case it may be better to be safe rather than sorry by acting quickly to resolve it. 

  Option 3 –The Matomo solution. Upgrade to a web analytics platform that can handle your demanding data requirements

  Save money while continuing to gain valuable insights by moving over to Matomo Analytics (recommended)

  This is where you can save up to USD130,000 a year. As well as that, the transition from Google Analytics to the Matomo Cloud is a seamless experience as setup and maintenance is taken care of by our experts.

  For example, you can get up to 15M pageviews for USD1,612.50/month (or USD19,350/year) on the Essentials plan.

  Or even 25M pageviews for USD2400/month (or USD24000/year) on the Business plan – which offers additional web analytics and conversion optimization resources.

  Matomo Cloud is a great option if you’re looking for a secure, cost-effective and powerful analytics solution. You also get what Google Analytics could never offer you : full control and ownership of your own data and privacy. 

  No need to worry about losing your Google Analytics data because …

  Now you can import your historic Google Analytics data directly into your Matomo with the Google Analytics Importer tool. Simply follow the step-by-step guide to get started for free.

  Along with savings you can get :

  • A solution for the data limits issue forever. You choose the right plan to suit your data needs and adapt as you continue growing
  • 100% accurate data (no data sampling)
  • 100% data ownership of all your information without signing away your data to a third party
  • Powerful web analytics and conversion optimization features
  • Matomo Tag Manager
  • Easy setup
  • Support from Matomo’s specialists

  Learn more about Matomo Cloud pricing.

  

  Or go for Matomo On-Premise

  If you have the in-house infrastructure to support self-hosting Matomo on your own servers then there’s also the option of Matomo On-Premise. Here you’ll get full security knowing the data is on your own servers. 

  Setup will also require technical knowledge. There will also be costs associated with acquiring your own servers, and keeping up with regular maintenance and updates. With On-Premise you get maximum flexibility, with no data limits whatsoever. But if you’re coming over from Google Analytics and don’t have the infrastructure and team to host On-Premise, the Matomo Cloud could be right for you.

  Learn more about Matomo On-Premise.

  Where do you go from here ?

  Getting 10 millions hits per month is no small feat, it’s actually pretty fantastic. But if it means having to shell out USD150,000 just to be able to continue with Google Analytics, we feel your problem could be fixed with Matomo Cloud. You could then put the rest of the money you save to better use.

  If you choose Matomo, you now have the option to : 

  • Raise your data limits for a fraction of Google Analytics 360’s price
  • Get a comprehensive range of analytics features for the most impactful insights to ensure your website continues excelling
  • Get data that’s not sampled – meaning 100% accuracy in your reports
  • Migrate your data easily with the help of Matomo’s support team

  We’ll have you covered. 

  By sharing with you the options and advice we gave to Mark, we hope you’ll be able to find a solution that makes your life easier and solves the issue of data restrictions forever.

  The team at Matomo is here to help you every step of the way to ensure a stress-free transition from Google Analytics if that is what works best for you.

  For next steps, why not check out our pricing page to see what could suit your needs !

  References :

  [1] Terms of Service. (2018, July 24). In Google Analytics Terms of Service. Retrieved June 12, 2019, from https://www.google.com/analytics/terms/us.html

  [2] Data limits. (n.d.). In Analytics Help Data limits. Retrieved June 12, 2019, from https://support.google.com/analytics/answer/1070983?hl=en