Sur d’autres sites (5992)

• DASH package (mpd) I made freezes after raising the onratechange event

  7 mars 2016, par Nick Birke

  I am trying to created an adaptive stream from an mp4 file. I have a decent amount of experience at doing streaming, and have successfully implemented smooth streaming via Silverlight. We want a solution for everything that works in HTML5. I am pursuing DASH. So here are the steps I take...

  1. Resample the mp4 to different sizes using ffmpeg.
  2. Ensure the files are properly fragmented using Bento4s mp4Fragment utility.
  3. Create the DASH mpd file and associated segments using Bento4s mp4dash utility.

  I have done a lot of work to learn each of these utilities and create a process that gets to output that I think will work. To create reliable DASH artifacts. But it seems I am doing something wrong. Here are examples of command lines I run.

  For step one I create three different files... for three different rates and sizes... (these may not be the exact command lines, they are from old notes... but the parameter values are the ones I am using for sure)

  ffmpeg -y -i C:\_SIMULATED_SHARE_C\ff\big_buck_bunny.mp4 -c:v libx264 -x264opts keyint=24:min-keyint=24:no-scenecut -b:v 1500k -maxrate 1500k -bufsize 1000k -vf "scale=-1:720" C:\_SIMULATED_SHARE_C\ff\output\outputfile720.mp4
  
  
  
  ffmpeg -y -i C:\_SIMULATED_SHARE_C\ff\big_buck_bunny.mp4 -c:v libx264 -x264opts keyint=24:min-keyint=24:no-scenecut -b:v 800k -maxrate 800k -bufsize 500k -vf "scale=-1:540" C:\_SIMULATED_SHARE_C\ff\output\outputfile540.mp4
  
  
  
  ffmpeg -y -i C:\_SIMULATED_SHARE_C\ff\big_buck_bunny.mp4 -c:v libx264 -x264opts keyint=24:min-keyint=24:no-scenecut -b:v 400k -maxrate 400k -bufsize 400k -vf "scale=-1:360" C:\_SIMULATED_SHARE_C\ff\output\outputfile360.mp4

  So that gives you the three files that need to be fragmented. So I fragment them using the mp4fragment utility from Bento4. That command line is simple so I will not offer an example.

  The I use Bento4s mp4dash to create the dash. Something like this...

  "E:\\_INSTALL\\Bento4\\Bento4-SDK-1-4-3-607.x86-microsoft-win32-vs2010\\Bento4-SDK-1-4-3-607.x86-microsoft-win32-vs2010\\bin\\mp4dash.exe --mpd-name=\"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\dash\\something.mpd\" -f -o \"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\dash\" \"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\bunny360f.mp4\" \"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\bunny540f.mp4\" \"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\bunny720f.mp4\" "

  This runs ok. And creates the mpd file, an audio folder with segment files, an a video folder with 3 folders each with the segment files.

  I am able to configure IIS to play the mpd ok. As well as the segement files. The video plays fine... and then it stops. It freezes between 50 and 60 seconds. Right before it freezes I can confirm that the onratechange event of the HTML5 video tag is raised. This is an event that is raised with the rate of play is changed. I do nothing to change this it just happens.

  The actual video in question is about 1hr long. Also I am using the dash.js library as a player.

  If you can identify any components or specifics I should change please let me know !

  UPDATE ON MAR 7, 2016

  As requested here is the MPD content...

  <?xml version="1.0" ?>
  
  <mpd mediapresentationduration="PT51M51.909S" minbuffertime="PT0.80S" profiles="urn:mpeg:dash:profile:isoff-live:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011">
  
    
  
    <period>
  
      
  
      <adaptationset lang="en" mimetype="audio/mp4" segmentalignment="true" startwithsap="1">
  
        <segmenttemplate duration="801" initialization="$RepresentationID$/init.mp4" media="$RepresentationID$/seg-$Number$.m4f" startnumber="1" timescale="1000"></segmenttemplate>
  
        <representation audiosamplingrate="48000" bandwidth="132491" codecs="mp4a.40.2">
  
          <audiochannelconfiguration schemeiduri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"></audiochannelconfiguration>
  
        </representation>
  
      </adaptationset>
  
      
  
      <adaptationset maxheight="720" maxwidth="1280" mimetype="video/mp4" minheight="540" minwidth="640" segmentalignment="true" startwithsap="1">
  
        <segmenttemplate duration="801" initialization="$RepresentationID$/init.mp4" media="$RepresentationID$/seg-$Number$.m4f" startnumber="1" timescale="1000"></segmenttemplate>
  
        <representation bandwidth="785536" codecs="avc1.64001F" framerate="30000/1001" height="540" scantype="progressive" width="960"></representation>
  
        <representation bandwidth="403537" codecs="avc1.64001E" framerate="30000/1001" height="360" scantype="progressive" width="640"></representation>
  
        <representation bandwidth="1496514" codecs="avc1.64001F" framerate="30000/1001" height="720" scantype="progressive" width="1280"></representation>
  
      </adaptationset>
  
    </period>
  
  </mpd>

  UPDATE ON MAR 7, 2016

  Here is the console output for two different videos that just stop for some unknown reason between 50 and 60s

  ///ABS VIDEO 
  
  
  
  58393][rulesController] [RULES]: playbackTimeRule [Fragment:download,82,MediaSegment,0video] Default 
  
  dash.all.js:11 [58395][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,82,MediaSegment,0,video] Default 
  
  dash.all.js:11 [58401][indexHandler][audio] Getting the request for time: 82.082 
  
  dash.all.js:11 [58403][indexHandler][audio] Index for time 82.082 is 81 
  
  dash.all.js:11 [58405][indexHandler][audio] SegmentTemplate: 81.08099999999999 / 1361.235 
  
  dash.all.js:11 [58406][indexHandler][audio] SegmentTemplate: 82.082 / 1361.235 
  
  dash.all.js:11 [58407][rulesController] [RULES]: playbackTimeRule [Fragment:download,82,MediaSegment,0audio] Default 
  
  dash.all.js:11 [58417][fragmentLoader] loaded video:MediaSegment:82.082 (200, 18ms, 2ms) 
  
  dash.all.js:11 [58419][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,82,MediaSegment,0,audio] Default 
  
  dash.all.js:11 [58435][fragmentLoader] loaded audio:MediaSegment:82.082 (200, 14ms, 0ms) 
  
  dash.all.js:11 [59390][indexHandler][video] Getting the request for time: 83.083 
  
  dash.all.js:11 [59392][indexHandler][video] Index for time 83.083 is 82 
  
  dash.all.js:11 [59393][indexHandler][video] SegmentTemplate: 82.082 / 1361.235 
  
  dash.all.js:11 [59394][indexHandler][video] SegmentTemplate: 83.08299999999998 / 1361.235 
  
  dash.all.js:11 [59395][rulesController] [RULES]: playbackTimeRule [Fragment:download,83,MediaSegment,0video] Default 
  
  dash.all.js:11 [59396][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,83,MediaSegment,0,video] Default 
  
  dash.all.js:11 [59401][indexHandler][audio] Getting the request for time: 83.083 
  
  dash.all.js:11 [59403][indexHandler][audio] Index for time 83.083 is 82 
  
  dash.all.js:11 [59404][indexHandler][audio] SegmentTemplate: 82.082 / 1361.235 
  
  dash.all.js:11 [59406][indexHandler][audio] SegmentTemplate: 83.08299999999998 / 1361.235 
  
  dash.all.js:11 [59408][rulesController] [RULES]: playbackTimeRule [Fragment:download,83,MediaSegment,0audio] Default 
  
  dash.all.js:11 [59416][fragmentLoader] loaded video:MediaSegment:83.08299999999998 (200, 17ms, 1ms) 
  
  dash.all.js:11 [59418][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,83,MediaSegment,0,audio] Default 
  
  dash.all.js:11 [59437][fragmentLoader] loaded audio:MediaSegment:83.08299999999998 (200, 14ms, 1ms) 
  
  dash.all.js:11 [59902][scheduleController][video] Stalling Buffer 
  
  dash.all.js:11 [59903][bufferController][video] Waiting for more buffer before starting playback. 
  
  dash.all.js:11 [59906][scheduleController][audio] Stalling Buffer 
  
  dash.all.js:11 [59907][bufferController][audio] Waiting for more buffer before starting playback. 
  
  dash.all.js:11 [63684][playbackController] <video> ratechange:  0 
  
  
  
  
  
  
  
  
  
  ///HOG VIDEO
  
  
  
  [58250][rulesController] [RULES]: playbackTimeRule [Fragment:download,105,MediaSegment,0audio] Default 
  
  dash.all.js:11 [58266][fragmentLoader] loaded video:MediaSegment:84.105 (200, 28ms, 1ms) 
  
  dash.all.js:11 [58269][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,105,MediaSegment,0,audio] Default 
  
  dash.all.js:11 [58287][fragmentLoader] loaded audio:MediaSegment:84.105 (200, 14ms, 1ms) 
  
  dash.all.js:11 [58982][indexHandler][video] Getting the request for time: 84.906 
  
  dash.all.js:11 [58983][indexHandler][video] Index for time 84.906 is 105 
  
  dash.all.js:11 [58984][indexHandler][video] SegmentTemplate: 84.105 / 3111.909 
  
  dash.all.js:11 [58984][indexHandler][video] SegmentTemplate: 84.906 / 3111.909 
  
  dash.all.js:11 [58985][rulesController] [RULES]: playbackTimeRule [Fragment:download,106,MediaSegment,0video] Default 
  
  dash.all.js:11 [58986][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,106,MediaSegment,0,video] Default 
  
  dash.all.js:11 [58989][indexHandler][audio] Getting the request for time: 84.906 
  
  dash.all.js:11 [58990][indexHandler][audio] Index for time 84.906 is 105 
  
  dash.all.js:11 [58991][indexHandler][audio] SegmentTemplate: 84.105 / 3111.909 
  
  dash.all.js:11 [58992][indexHandler][audio] SegmentTemplate: 84.906 / 3111.909 
  
  dash.all.js:11 [58993][rulesController] [RULES]: playbackTimeRule [Fragment:download,106,MediaSegment,0audio] Default 
  
  dash.all.js:11 [59002][fragmentLoader] loaded video:MediaSegment:84.906 (200, 14ms, 1ms) 
  
  dash.all.js:11 [59003][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,106,MediaSegment,0,audio] Default 
  
  dash.all.js:11 [59014][fragmentLoader] loaded audio:MediaSegment:84.906 (200, 8ms, 1ms) 
  
  dash.all.js:11 [59735][scheduleController][video] Stalling Buffer 
  
  dash.all.js:11 [59736][bufferController][video] Waiting for more buffer before starting playback. 
  
  dash.all.js:11 [59738][scheduleController][audio] Stalling Buffer 
  
  dash.all.js:11 [59739][bufferController][audio] Waiting for more buffer before starting playback. 
  
  dash.all.js:11 [61695][playbackController] <video> ratechange:  0 
  
  </video></video>

• FFmpeg : avconv and ffserver - connection reset by peer

  5 mars 2016, par einsA

  I’m trying to stream a video via ffserver. My dev environment is a Raspberry Pi 2.

  I’m getting an av_interleaved_write_frame(): Connection reset by peer with the following approach :

  pi@raspberrypi:/opt/livefeed $ avconv -f mjpeg -i movie.mjpg -r 20 -b 2000k -vcodec libx264 -preset ultrafast -f flv -metadata streamName="mjpeg_to_flv" -metadata title="converting mjpeg to flv via avconv" http://localhost:8090/feed1.ffm
  
  avconv version 11.4-6:11.4-1~deb8u1+rpi1, Copyright (c) 2000-2014 the Libav developers
  
    built on Jun 16 2015 05:32:34 with gcc 4.9.2 (Raspbian 4.9.2-10)
  
  [mjpeg @ 0x20461e0] Estimating duration from bitrate, this may be inaccurate
  
  Input #0, mjpeg, from 'movie.mjpg':
  
    Duration: N/A, bitrate: N/A
  
      Stream #0.0: Video: mjpeg, yuvj422p, 768x512, 25 fps, 25 tbn
  
  [tcp @ 0x2050bc0] Connection to tcp://localhost:8090 failed (Connection refused), trying next address
  
  [libx264 @ 0x20501a0] using cpu capabilities: none!
  
  [libx264 @ 0x20501a0] profile Constrained Baseline, level 3.0
  
  [libx264 @ 0x20501a0] 264 - core 142 r2431 a5831aa - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=20 scenecut=0 intra_refresh=0 rc=abr mbtree=0 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
  
  Output #0, flv, to 'http://localhost:8090/feed1.ffm':
  
    Metadata:
  
      streamName      : mjpeg_to_flv
  
      title           : converting mjpeg to flv via avconv
  
      encoder         : Lavf56.1.0
  
      Stream #0.0: Video: libx264, yuv420p, 768x512, q=-1--1, 2000 kb/s, 20 fps, 1k tbn, 20 tbc
  
      Metadata:
  
        encoder         : Lavc56.1.0 libx264
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
  
  Press ctrl-c to stop encoding
  
  av_interleaved_write_frame(): Connection reset by peer0000.00 bitrate=   0.0kbits/s

  ---

  I am able to stream the file doing it step by step with the following settings and commands :

  ffserver

  ffserver config :

  pi@raspberrypi:/opt/livefeed $ cat ffserver_flv.conf 
  
  # FLV live feed
  
  
  
  Port 8090
  
  BindAddress 0.0.0.0
  
  MaxHTTPConnections 2000
  
  MaxClients 1000
  
  MaxBandwidth 1000
  
  CustomLog -
  
  
  
  <stream>
  
    Format status
  
    ACL allow localhost
  
    ACL allow 192.168.0.0 192.168.255.255
  
  </stream>
  
  
  
  <feed>
  
    File /tmp/feed_flv.ffm
  
    FileMaxSize 1G
  
    ACL allow 127.0.0.1
  
  </feed>
  
  
  
  <stream>
  
    Format flv
  
    Feed feed1.ffm
  
  
  
    VideoCodec libx264
  
    VideoFrameRate 30
  
    VideoBitRate 800
  
    VideoSize 720x576
  
    AVOptionVideo crf 23
  
    AVOptionVideo preset medium
  
    AVOptionVideo me_range 16
  
    AVOptionVideo qdiff 4
  
    AVOptionVideo qmin 10
  
    AVOptionVideo qmax 51
  
    AVOptionVideo flags +global_header
  
  
  
    NoAudio
  
  </stream>

  starting the server with :

  pi@raspberrypi:/opt/livefeed $ ffserver -v debug -f ffserver_flv.conf
  
  ffserver version N-78909-gfb2f164 Copyright (c) 2000-2016 the FFmpeg developers
  
    built with gcc 4.9.2 (Raspbian 4.9.2-10)
  
    configuration: --arch=armel --target-os=linux --enable-gpl --enable-libx264 --enable-nonfree --enable-libass --enable-libfaac --enable-libmp3lame --enable-libfreetype
  
    libavutil      55. 19.100 / 55. 19.100
  
    libavcodec     57. 27.101 / 57. 27.101
  
    libavformat    57. 28.100 / 57. 28.100
  
    libavdevice    57.  0.101 / 57.  0.101
  
    libavfilter     6. 39.100 /  6. 39.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  ffserver_flv.conf:6: Port option is deprecated. Use HTTPPort instead.
  
  ffserver_flv.conf:7: BindAddress option is deprecated. Use HTTPBindAddress instead.
  
  ffserver_flv.conf:48: Setting default value for video bit rate tolerance = 200000. Use NoDefaults to disable it.
  
  ffserver_flv.conf:48: Setting default value for video rate control equation = tex^qComp. Use NoDefaults to disable it.
  
  ffserver_flv.conf:48: Setting default value for video max rate = 49537952. Use NoDefaults to disable it.
  
  ffserver_flv.conf:48: Setting default value for video buffer size = 1600000. Use NoDefaults to disable it.
  
  Sat Mar  5 13:29:46 2016 [file @ 0x2f3d0d0]Setting default whitelist 'file'
  
  Sat Mar  5 13:29:46 2016 [ffm @ 0x2f40bc0]Format ffm probed with size=2048 and score=101
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'time_base' to value '1/30'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'b' to value '800000'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'video_size' to value '720x576'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'me_range' to value '16'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'qdiff' to value '4'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'qmin' to value '10'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'qmax' to value '51'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'flags' to value '+global_header'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'bt' to value '200000'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'rc_eq' to value 'tex^qComp'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'maxrate' to value '1600000'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'bufsize' to value '1600000'
  
  Sat Mar  5 13:29:46 2016 [AVIOContext @ 0x2f41450]Statistics: 32768 bytes read, 0 seeks
  
  Sat Mar  5 13:29:46 2016 FFserver started.

  avconv

  I’m converting a mjpeg movie (movie.mjpeg) via avconv to flv (movie.flv) :

  pi@raspberrypi:/opt/livefeed $ avconv -f mjpeg -i movie.mjpg -r 20 -b 2000k -vcodec libx264 -preset ultrafast -f flv -metadata streamName="mjpeg_to_flv" -metadata title="converting mjpeg to flv via avconv" movie.flv
  
  avconv version 11.4-6:11.4-1~deb8u1+rpi1, Copyright (c) 2000-2014 the Libav developers
  
    built on Jun 16 2015 05:32:34 with gcc 4.9.2 (Raspbian 4.9.2-10)
  
  [mjpeg @ 0x34f1e0] Estimating duration from bitrate, this may be inaccurate
  
  Input #0, mjpeg, from 'movie.mjpg':
  
    Duration: N/A, bitrate: N/A
  
      Stream #0.0: Video: mjpeg, yuvj422p, 768x512, 25 fps, 25 tbn
  
  [libx264 @ 0x3591a0] using cpu capabilities: none!
  
  [libx264 @ 0x3591a0] profile Constrained Baseline, level 3.0
  
  [libx264 @ 0x3591a0] 264 - core 142 r2431 a5831aa - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=20 scenecut=0 intra_refresh=0 rc=abr mbtree=0 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
  
  Output #0, flv, to 'movie.flv':
  
    Metadata:
  
      streamName      : mjpeg_to_flv
  
      title           : converting mjpeg to flv via avconv
  
      encoder         : Lavf56.1.0
  
      Stream #0.0: Video: libx264, yuv420p, 768x512, q=-1--1, 2000 kb/s, 20 fps, 1k tbn, 20 tbc
  
      Metadata:
  
        encoder         : Lavc56.1.0 libx264
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
  
  Press ctrl-c to stop encoding
  
  frame=   93 fps=  9 q=-1.0 Lsize=    1078kB time=4.60 bitrate=1919.3kbits/s    ts/s    
  
  video:1076kB audio:0kB other streams:0kB global headers:0kB muxing overhead: 0.201471%
  
  [libx264 @ 0x3591a0] frame I:1     Avg QP:19.00  size: 35978
  
  [libx264 @ 0x3591a0] frame P:92    Avg QP:21.43  size: 11574
  
  [libx264 @ 0x3591a0] mb I  I16..4: 100.0%  0.0%  0.0%
  
  [libx264 @ 0x3591a0] mb P  I16..4: 25.9%  0.0%  0.0%  P16..4: 53.6%  0.0%  0.0%  0.0%  0.0%    skip:20.5%
  
  [libx264 @ 0x3591a0] final ratefactor: 20.82
  
  [libx264 @ 0x3591a0] coded y,uvDC,uvAC intra: 28.9% 46.9% 18.2% inter: 33.7% 39.8% 2.8%
  
  [libx264 @ 0x3591a0] i16 v,h,dc,p: 29% 30% 28% 14%
  
  [libx264 @ 0x3591a0] i8c dc,h,v,p: 53% 26% 15%  7%
  
  [libx264 @ 0x3591a0] kb/s:1893.82

  send flv to ffserver

  pi@raspberrypi:/opt/livefeed $ ffmpeg -stream_loop -1 -i movie.flv http://localhost:8090/feed1.ffm
  
  ffmpeg version N-78909-gfb2f164 Copyright (c) 2000-2016 the FFmpeg developers
  
    built with gcc 4.9.2 (Raspbian 4.9.2-10)
  
    configuration: --arch=armel --target-os=linux --enable-gpl --enable-libx264 --enable-nonfree --enable-libass --enable-libfaac --enable-libmp3lame --enable-libfreetype
  
    libavutil      55. 19.100 / 55. 19.100
  
    libavcodec     57. 27.101 / 57. 27.101
  
    libavformat    57. 28.100 / 57. 28.100
  
    libavdevice    57.  0.101 / 57.  0.101
  
    libavfilter     6. 39.100 /  6. 39.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  Input #0, flv, from 'movie.flv':
  
    Metadata:
  
      streamName      : mjpeg_to_flv
  
      title           : converting mjpeg to flv via avconv
  
      encoder         : Lavf56.1.0
  
    Duration: 00:00:04.65, start: 0.000000, bitrate: 1898 kb/s
  
      Stream #0:0: Video: h264 (Constrained Baseline), yuv420p, 768x512, 2000 kb/s, 20 fps, 20 tbr, 1k tbn, 40 tbc
  
  [tcp @ 0x30d2920] Connection to tcp://localhost:8090 failed (Connection refused), trying next address
  
  [tcp @ 0x329a080] Connection to tcp://localhost:8090 failed (Connection refused), trying next address
  
  [libx264 @ 0x325fb30] using cpu capabilities: ARMv6 NEON
  
  [libx264 @ 0x325fb30] profile High, level 3.1
  
  [libx264 @ 0x325fb30] 264 - core 148 r2665 a01e339 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 vbv_maxrate=1600 vbv_bufsize=1600 crf_max=0.0 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
  
  Output #0, ffm, to 'http://localhost:8090/feed1.ffm':
  
    Metadata:
  
      streamName      : mjpeg_to_flv
  
      title           : converting mjpeg to flv via avconv
  
      creation_time   : now
  
      encoder         : Lavf57.28.100
  
      Stream #0:0: Video: h264 (libx264), yuv420p, 720x576, q=10-51, 800 kb/s, 20 fps, 1000k tbn, 30 tbc
  
      Metadata:
  
        encoder         : Lavc57.27.101 libx264
  
      Side data:
  
        cpb: bitrate max/min/avg: 1600000/0/800000 buffer size: 1600000 vbv_delay: -1
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  
  Press [q] to stop, [?] for help
  
  Past duration 0.999992 too large
  
      Last message repeated 2 times
  
  Past duration 0.999992 too large      4kB time=00:00:00.00 bitrate=N/A dup=5 drop=0 speed=   0x    
  
      Last message repeated 3 times
  
  Past duration 0.999992 too large      4kB time=00:00:00.00 bitrate=N/A dup=9 drop=0 speed=   
  
  [croped]    
  
  video:968kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3.719636%
  
  [libx264 @ 0x325fb30] frame I:11    Avg QP:22.49  size: 13888
  
  [libx264 @ 0x325fb30] frame P:487   Avg QP:16.10  size:  1382
  
  [libx264 @ 0x325fb30] frame B:980   Avg QP:21.31  size:   168
  
  [libx264 @ 0x325fb30] consecutive B-frames: 11.0%  1.8%  0.4% 86.9%
  
  [libx264 @ 0x325fb30] mb I  I16..4: 11.5% 81.2%  7.3%
  
  [libx264 @ 0x325fb30] mb P  I16..4:  1.6%  5.0%  0.2%  P16..4: 12.0%  1.6%  0.7%  0.0%  0.0%    skip:79.0%
  
  [libx264 @ 0x325fb30] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  5.8%  0.1%  0.0%  direct: 0.0%  skip:94.1%  L0:55.3% L1:44.4% BI: 0.3%
  
  [libx264 @ 0x325fb30] 8x8 transform intra:75.4% inter:88.7%
  
  [libx264 @ 0x325fb30] coded y,uvDC,uvAC intra: 38.6% 48.2% 14.9% inter: 1.5% 2.3% 0.0%
  
  [libx264 @ 0x325fb30] i16 v,h,dc,p: 28% 32% 11% 28%
  
  [libx264 @ 0x325fb30] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 21% 31%  3%  4%  4%  4%  4%  4%
  
  [libx264 @ 0x325fb30] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 25% 23%  4%  7%  5%  7%  4%  3%
  
  [libx264 @ 0x325fb30] i8c dc,h,v,p: 63% 20% 14%  4%
  
  [libx264 @ 0x325fb30] Weighted P-Frames: Y:1.6% UV:1.2%
  
  [libx264 @ 0x325fb30] ref P L0: 67.5% 11.6% 13.8%  6.8%  0.3%
  
  [libx264 @ 0x325fb30] ref B L0: 58.7% 40.4%  0.9%
  
  [libx264 @ 0x325fb30] ref B L1: 96.2%  3.8%
  
  [libx264 @ 0x325fb30] kb/s:160.83
  
  Exiting normally, received signal 2.

  On another machine I can open the flv-stream via VLC by opening the address http://ip.running.ffserver:8090/live.flv.

  So why does it works step by step via the movie.flv file and not directly ?

  What is the difference between

  • avconv -f mjpeg -i movie.mjpg -r 20 -b 2000k -vcodec libx264 -preset ultrafast -f flv -metadata streamName="mjpeg_to_flv" -metadata title="converting mjpeg to flv via avconv" movie.flv
    followed by
    ffmpeg -stream_loop -1 -i movie.flv http://localhost:8090/feed1.ffm and

  • avconv -f mjpeg -i movie.mjpg -r 20 -b 2000k -vcodec libx264 -preset ultrafast -f flv -metadata streamName="mjpeg_to_flv" -metadata title="converting mjpeg to flv via avconv" http://localhost:8090/feed1.ffm ? Why ends this command with a av_interleaved_write_frame(): Connection reset by peer ?

  ---

  Informations and logs

  • ffserver log (both times) :

    Sat Mar  5 13:54:08 2016 127.0.0.1 - - [POST] "/feed1.ffm HTTP/1.1" 200 4096

  • ffserver version

    pi@raspberrypi:/opt/livefeed $ ffserver -version
    
    ffserver version N-78909-gfb2f164 Copyright (c) 2000-2016 the FFmpeg developers
    
    built with gcc 4.9.2 (Raspbian 4.9.2-10)
    
    configuration: --arch=armel --target-os=linux --enable-gpl --enable-libx264 --enable-nonfree --enable-libass --enable-libfaac --enable-libmp3lame --enable-libfreetype
    
    libavutil      55. 19.100 / 55. 19.100
    
    libavcodec     57. 27.101 / 57. 27.101
    
    libavformat    57. 28.100 / 57. 28.100
    
    libavdevice    57.  0.101 / 57.  0.101
    
    libavfilter     6. 39.100 /  6. 39.100
    
    libswscale      4.  0.100 /  4.  0.100
    
    libswresample   2.  0.101 /  2.  0.101
    
    libpostproc    54.  0.100 / 54.  0.100

  • ffmpeg version

    pi@raspberrypi:/opt/livefeed $ ffmpeg -version
    
    ffmpeg version N-78909-gfb2f164 Copyright (c) 2000-2016 the FFmpeg developers
    
    built with gcc 4.9.2 (Raspbian 4.9.2-10)
    
    configuration: --arch=armel --target-os=linux --enable-gpl --enable-libx264 --enable-nonfree --enable-libass --enable-libfaac --enable-libmp3lame --enable-libfreetype
    
    libavutil      55. 19.100 / 55. 19.100
    
    libavcodec     57. 27.101 / 57. 27.101
    
    libavformat    57. 28.100 / 57. 28.100
    
    libavdevice    57.  0.101 / 57.  0.101
    
    libavfilter     6. 39.100 /  6. 39.100
    
    libswscale      4.  0.100 /  4.  0.100
    
    libswresample   2.  0.101 /  2.  0.101
    
    libpostproc    54.  0.100 / 54.  0.100

  • avconv version :

    pi@raspberrypi:/opt/livefeed $ avconv -version
    
    avconv version 11.4-6:11.4-1~deb8u1+rpi1, Copyright (c) 2000-2014 the Libav developers
    
    built on Jun 16 2015 05:32:34 with gcc 4.9.2 (Raspbian 4.9.2-10)
    
    avconv 11.4-6:11.4-1~deb8u1+rpi1
    
    libavutil     54.  3. 0 / 54.  3. 0
    
    libavcodec    56.  1. 0 / 56.  1. 0
    
    libavformat   56.  1. 0 / 56.  1. 0
    
    libavdevice   55.  0. 0 / 55.  0. 0
    
    libavfilter    5.  0. 0 /  5.  0. 0
    
    libavresample  2.  1. 0 /  2.  1. 0
    
    libswscale     3.  0. 0 /  3.  0. 0

  Thanks and cheers

  Clemens

• Android recording video with overlay view

  6 mars 2016, par t0m

  I am trying app in android which have functionality to capture video with overlay views. (Also i need onPreviewFrame method.)
  Via SurfaceView and JavaCV with FFmpeg :

  ---

  OpenCVCameraActivity.java :

  import android.app.Activity;
  
  import android.content.Context;
  
  import android.hardware.Camera;
  
  import android.hardware.Camera.PreviewCallback;
  
  import android.media.AudioFormat;
  
  import android.media.AudioRecord;
  
  import android.media.MediaRecorder;
  
  import android.os.Bundle;
  
  import android.util.Log;
  
  import android.view.KeyEvent;
  
  import android.view.Menu;
  
  import android.view.SurfaceHolder;
  
  import android.view.SurfaceView;
  
  import android.view.View;
  
  import android.widget.FrameLayout;
  
  import android.widget.Toast;
  
  
  
  import org.bytedeco.javacv.FFmpegFrameRecorder;
  
  import org.bytedeco.javacv.Frame;
  
  
  
  import java.io.File;
  
  import java.io.IOException;
  
  import java.nio.ByteBuffer;
  
  import java.nio.ShortBuffer;
  
  import java.util.List;
  
  
  
  
  
  @SuppressWarnings("ALL")
  
  public class OpenCVCameraActivity extends Activity {
  
  
  
      private static final String TAG = OpenCVCameraActivity.class.getSimpleName();
  
  
  
      private long startTime = 0;
  
      private boolean isPreviewOn = false;
  
  
  
      private int sampleAudioRateInHz = 44100;
  
      private Camera.Size previewSize;        //preview and Camera and Recorder width and height
  
      private int recorderFrameRate = 25;
  
  
  
      // audio data getting thread
  
      private AudioRecord audioRecord;
  
      private AudioRecordRunnable audioRecordRunnable;
  
      private Thread audioThread;
  
      private volatile boolean runAudioThread = true;
  
  
  
      // video data getting thread
  
      private Camera mCamera;
  
      private CameraView mPreview;
  
      private FFmpegFrameRecorder recorder;
  
      private boolean recording = false;
  
      private Frame yuvImage = null;
  
  
  
      //storage
  
      private Storage storage;
  
  
  
      @Override
  
      public void onCreate(Bundle savedInstanceState) {
  
          super.onCreate(savedInstanceState);
  
          if(Static.DEBUG) Log.i(TAG, "onCreate()");
  
  
  
          Thread.setDefaultUncaughtExceptionHandler(uncaughtExceptionHandler);
  
  
  
          setContentView(R.layout.activity_opencv);
  
  
  
          prepareCamera();
  
      }
  
  
  
      private Thread.UncaughtExceptionHandler uncaughtExceptionHandler =
  
              new Thread.UncaughtExceptionHandler() {
  
                  public void uncaughtException(Thread thread, Throwable ex) {
  
                      if(Static.DEBUG) Log.e(TAG, "Uncaught exception", ex);
  
                  }
  
              };
  
  
  
      @Override
  
      protected void onRestart() {
  
          super.onRestart();
  
          if (Static.DEBUG) Log.i(TAG, "onRestart()");
  
      }
  
  
  
      @Override
  
      protected void onStart() {
  
          super.onStart();
  
          if (Static.DEBUG) Log.i(TAG, "onStart()");
  
      }
  
  
  
      @Override
  
      protected void onResume() {
  
          super.onResume();
  
          if (Static.DEBUG) Log.i(TAG, "onResume()");
  
  
  
          storage = new Storage(this);
  
          if(storage.mExternalStorageAvailable == true && storage.mExternalStorageWriteable == false)
  
              Static.showToast(this, getString(R.string.errExternalStorageReadOnly), Toast.LENGTH_LONG);
  
          else if (storage.mExternalStorageAvailable == false && storage.mExternalStorageWriteable == false)
  
              Static.showToast(this, getString(R.string.errExternalStorage), Toast.LENGTH_LONG);
  
      }
  
  
  
      @Override
  
      public boolean onCreateOptionsMenu(Menu menu) {
  
          if (Static.DEBUG) Log.i(TAG, "onCreateOptionsMenu()");
  
          return super.onCreateOptionsMenu(menu);
  
      }
  
  
  
      @Override
  
      protected void onPause() {
  
          super.onPause();
  
          if (Static.DEBUG) Log.i(TAG, "onPause()");
  
      }
  
  
  
      @Override
  
      protected void onStop() {
  
          super.onStop();
  
          if (Static.DEBUG) Log.i(TAG, "onStop()");
  
      }
  
  
  
      @Override
  
      protected void onDestroy() {
  
          super.onDestroy();
  
          if (Static.DEBUG) Log.i(TAG, "onDestroy()");
  
  
  
          recording = false;
  
  
  
          if (mPreview != null) {
  
              mPreview.stopPreview();
  
          }
  
  
  
          if (mCamera != null) {
  
              mCamera.stopPreview();
  
              mCamera.release();
  
              mCamera = null;
  
          }
  
      }
  
  
  
      /** Prepare camera object.
  
       * */
  
      private void prepareCamera() {
  
          //1. Open camera object
  
          try {
  
              mCamera = getCameraInstance(this);
  
          } catch (Exception e) {
  
              e.printStackTrace();
  
              Static.showToast(this, e.getMessage(), Toast.LENGTH_LONG);
  
              finish();
  
              return;
  
          }
  
  
  
          setOptimalCameraParams();
  
  
  
          //2. Connect Preview + 3. Start Preview + 8e Stop preview v metode surfaceChanged
  
          mPreview = new CameraView(this, mCamera);
  
          FrameLayout preview = (FrameLayout) findViewById(R.id.cameraPreview);
  
          preview.addView(mPreview); //surfaceView to FrameLayout
  
          if(Static.DEBUG) Log.i(TAG, "camera preview start: OK");
  
      }
  
  
  
      /**
  
       * A safe way to get an instance of the Camera object.
  
       */
  
      @SuppressWarnings("deprecation")
  
      public static Camera getCameraInstance(Context ctx) throws Exception {
  
          Camera c = Camera.open();
  
          if (c == null)
  
              throw new Exception(ctx.getString(R.string.errCameraNotAvailable));
  
  
  
          if(Static.DEBUG) Log.i(TAG, "camera open");
  
          return c; // attempt to get a Camera instance, otherwise null
  
      }
  
  
  
      /**Method set optmial parameters. Depend on HW possibilities.*/
  
      @SuppressWarnings("deprecation")
  
      private void setOptimalCameraParams(){
  
  
  
          // Camera parameters
  
          Camera.Parameters params = mCamera.getParameters();
  
          List<string> focusModes = params.getSupportedFocusModes();
  
          if (Static.DEBUG) Log.i(TAG, "focusModes():" + focusModes.toString());
  
          if (Static.DEBUG) Log.i(TAG, "Camera parameters:\n" + params.flatten());
  
  
  
          params.setRecordingHint(true); //MediaRecorder.start() to start faster
  
  
  
          //Automatically autofocus if it's possible
  
          if (params.getSupportedFocusModes().contains(Camera.Parameters.FOCUS_MODE_CONTINUOUS_VIDEO)) {
  
              params.setFocusMode(Camera.Parameters.FOCUS_MODE_CONTINUOUS_VIDEO);
  
          } else if (focusModes.contains(Camera.Parameters.FOCUS_MODE_AUTO)) { //at least focus auto
  
              params.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);
  
          }
  
  
  
          // set Camera parameters
  
          mCamera.setParameters(params);
  
      }
  
  
  
      //---------------------------------------
  
      // initialize ffmpeg_recorder
  
      //---------------------------------------
  
      private void initRecorder() throws Exception {
  
          if(Static.DEBUG) Log.i(TAG,"init recorder");
  
  
  
          File output = null;
  
  
  
          try {
  
              output = storage.getOutputMediaFile(storage.MEDIA_TYPE_VIDEO);
  
              if(output == null)
  
                  throw new Exception();
  
          } catch (Exception e) {
  
              e.printStackTrace();
  
              throw new Exception(getString(R.string.errSetOutputFile));
  
          }
  
  
  
          if (yuvImage == null) {
  
              yuvImage = new Frame(previewSize.width, previewSize.height, Frame.DEPTH_UBYTE, 2);
  
              if(Static.DEBUG) Log.i(TAG, "create yuvImage");
  
          }
  
  
  
          if(Static.DEBUG) Log.i(TAG, "ffmpeg_url: " + output.getPath());
  
          recorder = new FFmpegFrameRecorder(output.getPath(), previewSize.width, previewSize.height, 1);
  
          //recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
  
          //recorder.setVideoOption("preset", "veryfast"); // or ultrafast or fast, etc.
  
          //recorder.setVideoQuality(0); // maximum quality, replace recorder.setVideoBitrate(16384);
  
          //recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
  
          recorder.setFormat("mp4");
  
          recorder.setSampleRate(sampleAudioRateInHz);
  
          recorder.setFrameRate(recorderFrameRate);
  
  
  
          if(Static.DEBUG) Log.i(TAG, "recorder initialize success");
  
  
  
          audioRecordRunnable = new AudioRecordRunnable();
  
          audioThread = new Thread(audioRecordRunnable);
  
          runAudioThread = true;
  
      }
  
  
  
      public void startRecording() {
  
  
  
          try {
  
              initRecorder();
  
          } catch (Exception e){
  
              e.printStackTrace();
  
              Static.showToast(this, e.getMessage(), Toast.LENGTH_LONG);
  
          }
  
  
  
          try {
  
              recorder.start();
  
              startTime = System.currentTimeMillis();
  
              recording = true;
  
              audioThread.start();
  
  
  
          } catch (FFmpegFrameRecorder.Exception e) {
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      public void stopRecording() {
  
  
  
          runAudioThread = false;
  
          try {
  
              audioThread.join();
  
          } catch (InterruptedException e) {
  
              // reset interrupt to be nice
  
              Thread.currentThread().interrupt();
  
              return;
  
          }
  
          audioRecordRunnable = null;
  
          audioThread = null;
  
  
  
          if (recorder != null &amp;&amp; recording) {
  
  
  
              recording = false;
  
              if(Static.DEBUG) Log.i(TAG,"Finishing recording, calling stop and release on recorder");
  
              try {
  
                  recorder.stop();
  
                  recorder.release();
  
              } catch (FFmpegFrameRecorder.Exception e) {
  
                  e.printStackTrace();
  
              }
  
              recorder = null;
  
  
  
          }
  
      }
  
  
  
      @Override
  
      public boolean onKeyDown(int keyCode, KeyEvent event) {
  
  
  
          if (keyCode == KeyEvent.KEYCODE_BACK) {
  
              if (recording) {
  
                  stopRecording();
  
              }
  
  
  
              finish();
  
  
  
              return true;
  
          }
  
  
  
          return super.onKeyDown(keyCode, event);
  
      }
  
  
  
      public void onClickBtnStartRecord(View v) {
  
          if (!recording) {
  
              startRecording();
  
              if(Static.DEBUG) Log.i(TAG, "Start Button Pushed");
  
          } else {
  
              // This will trigger the audio recording loop to stop and then set isRecorderStart = false;
  
              stopRecording();
  
              if(Static.DEBUG) Log.i(TAG, "Stop Button Pushed");
  
          }
  
      }
  
  
  
      //---------------------------------------------
  
      // audio thread, gets and encodes audio data
  
      //---------------------------------------------
  
      class AudioRecordRunnable implements Runnable {
  
  
  
          @Override
  
          public void run() {
  
              android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
  
  
  
              // Audio
  
              int bufferSize;
  
              ShortBuffer audioData;
  
              int bufferReadResult;
  
  
  
              bufferSize = AudioRecord.getMinBufferSize(sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
  
              audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
  
  
  
              audioData = ShortBuffer.allocate(bufferSize);
  
  
  
              if(Static.DEBUG) Log.i(TAG, "audioRecord.startRecording()");
  
              audioRecord.startRecording();
  
  
  
              /* ffmpeg_audio encoding loop */
  
              while (runAudioThread) {
  
                  //if(Static.DEBUG) Log.i(TAG,"recording? " + recording);
  
                  bufferReadResult = audioRecord.read(audioData.array(), 0, audioData.capacity());
  
                  audioData.limit(bufferReadResult);
  
                  if (bufferReadResult > 0) {
  
                      //if(Static.DEBUG) Log.i(TAG,"bufferReadResult: " + bufferReadResult);
  
                      // If "recording" isn't true when start this thread, it never get's set according to this if statement...!!!
  
                      // Why?  Good question...
  
                      if (recording) {
  
                          try {
  
                              recorder.recordSamples(audioData);
  
                              //if(Static.DEBUG) Log.i(TAG,"recording " + 1024*i + " to " + 1024*i+1024);
  
                          } catch (FFmpegFrameRecorder.Exception e) {
  
                              if(Static.DEBUG) Log.i(TAG,e.getMessage());
  
                              e.printStackTrace();
  
                          }
  
                      }
  
                  }
  
              }
  
              if(Static.DEBUG) Log.i(TAG,"AudioThread Finished, release audioRecord");
  
  
  
              /* encoding finish, release recorder */
  
              if (audioRecord != null) {
  
                  audioRecord.stop();
  
                  audioRecord.release();
  
                  audioRecord = null;
  
                  if(Static.DEBUG) Log.i(TAG,"audioRecord released");
  
              }
  
          }
  
      }
  
  
  
      /**TODO*/
  
      private Camera.Size getBestPreviewSize(int width, int height, Camera.Parameters parameters) {
  
          Camera.Size result=null;
  
  
  
          for (Camera.Size size : parameters.getSupportedPreviewSizes()) {
  
  
  
              if(Static.DEBUG) Log.i(TAG, size.width + "/" + size.height);
  
  
  
              if (size.width&lt;=width &amp;&amp; size.height&lt;=height) {
  
                  if (result==null) {
  
                      result=size;
  
                  } else {
  
                      int resultArea=result.width*result.height;
  
                      int newArea=size.width*size.height;
  
  
  
                      if (newArea>resultArea) {
  
                          result=size;
  
                      }
  
                  }
  
              }
  
          }
  
  
  
          return(result);
  
      }
  
  
  
      //---------------------------------------------
  
      // camera thread, gets and encodes video data
  
      //---------------------------------------------
  
      private class CameraView extends SurfaceView implements SurfaceHolder.Callback, PreviewCallback {
  
  
  
          private final String TAG = CameraView.class.getSimpleName();
  
  
  
          private SurfaceHolder mHolder;
  
          private Camera mCamera;
  
  
  
          public CameraView(Context context, Camera camera) {
  
              super(context);
  
              if(Static.DEBUG) Log.i(TAG, "camera view");
  
  
  
              mCamera = camera;
  
  
  
              // Install a SurfaceHolder.Callback so we get notified when the
  
              // underlying surface is created and destroyed.
  
              mHolder = getHolder();
  
              mHolder.addCallback(CameraView.this);
  
              mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
  
              mCamera.setPreviewCallback(CameraView.this);
  
          }
  
  
  
          @Override
  
          public void surfaceCreated(SurfaceHolder holder) {
  
              try {
  
                  stopPreview();
  
                  mCamera.setPreviewDisplay(holder);
  
              } catch (IOException exception) {
  
                  mCamera.release();
  
                  mCamera = null;
  
              }
  
          }
  
  
  
          public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
  
              if(Static.DEBUG) Log.i(TAG, "surfaceChanged() => w=" + w + ", h=" + h);
  
  
  
              // If your preview can change or rotate, take care of those events here.
  
              // Make sure to stop the preview before resizing or reformatting it.
  
  
  
              if (mHolder.getSurface() == null){
  
                  // preview surface does not exist
  
                  return;
  
              }
  
  
  
              // stop preview before making changes
  
              try {
  
                  stopPreview();
  
              } catch (Exception e){
  
                  // ignore: tried to stop a non-existent preview
  
              }
  
  
  
              // start preview with new settings
  
              try {
  
                  Camera.Parameters params = mCamera.getParameters();
  
  
  
                  previewSize = getBestPreviewSize(w, h, params);
  
                  if(Static.DEBUG) Log.i(TAG, "getBestPreviewSize() => w=" + previewSize.width + ", h=" + previewSize.height);
  
                  if (previewSize != null)
  
                      params.setPreviewSize(previewSize.width, previewSize.height);
  
  
  
                  params.setPreviewFrameRate(recorderFrameRate);
  
                  if(Static.DEBUG) Log.i(TAG,"Preview Framerate: " + params.getPreviewFrameRate());
  
                  mCamera.setParameters(params);
  
                  mCamera.setPreviewDisplay(holder);
  
                  mCamera.setPreviewCallback(CameraView.this);
  
                  startPreview();
  
              } catch (Exception e){
  
                  if(Static.DEBUG) Log.i(TAG, "Could not set preview display in surfaceChanged");
  
                  e.printStackTrace();
  
              }
  
  
  
          }
  
  
  
          @Override
  
          public void surfaceDestroyed(SurfaceHolder holder) {
  
              try {
  
                  mHolder.addCallback(null);
  
                  mCamera.setPreviewCallback(null);
  
              } catch (RuntimeException e) {
  
                  // The camera has probably just been released, ignore.
  
              }
  
          }
  
  
  
          public void startPreview() {
  
              if (!isPreviewOn &amp;&amp; mCamera != null) {
  
                  isPreviewOn = true;
  
                  mCamera.startPreview();
  
              }
  
          }
  
  
  
          public void stopPreview() {
  
              if (isPreviewOn &amp;&amp; mCamera != null) {
  
                  isPreviewOn = false;
  
                  mCamera.stopPreview();
  
              }
  
          }
  
  
  
          @Override
  
          public void onPreviewFrame(byte[] data, Camera camera) {
  
              if (audioRecord == null || audioRecord.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING) {
  
                  startTime = System.currentTimeMillis();
  
                  return;
  
              }
  
  
  
              // get video data
  
              if (yuvImage != null &amp;&amp; recording) {
  
                  ((ByteBuffer)yuvImage.image[0].position(0)).put(data);
  
  
  
                  try {
  
                      long t = 1000 * (System.currentTimeMillis() - startTime);
  
                      if(Static.DEBUG) Log.i(TAG,"Writing Frame on timestamp: "+t);
  
                      if (t > recorder.getTimestamp()) {
  
                          recorder.setTimestamp(t);
  
                      }
  
                      recorder.record(yuvImage);
  
                  } catch (FFmpegFrameRecorder.Exception e) {
  
                      if(Static.DEBUG) Log.i(TAG,e.getMessage());
  
                      e.printStackTrace();
  
                  }
  
              }
  
          }
  
      }
  
  }
  
  </string>

  activity_opencv.xml :

  <?xml version="1.0" encoding="utf-8"?>
  
  
  
  <relativelayout>
  
  
  
      <framelayout>
  
      </framelayout>
  
  
  
      &lt;ImageButton<br />
  
          android:id=&quot;@+id/btnStartRecord&quot;<br />
  
          android:layout_width=&quot;70dp&quot;<br />
  
          android:layout_height=&quot;70dp&quot;<br />
  
          android:scaleType=&quot;fitXY&quot;<br />
  
          android:src=&quot;@drawable/record_icon&quot;<br />
  
          android:background=&quot;@null&quot;<br />
  
          android:text=&quot;@string/btnStartRecord&quot;<br />
  
          android:onClick=&quot;onClickBtnStartRecord&quot;<br />
  
          android:clickable=&quot;true&quot;<br />
  
          android:layout_centerVertical=&quot;true&quot;<br />
  
          android:layout_alignParentRight=&quot;true&quot;<br />
  
          android:layout_alignParentEnd=&quot;true&quot;/&gt;
  
  
  
      <textview></textview>
  
  
  
  </relativelayout>

  Overlay views working, but recorded video is without overlay views.