Recherche avancée

Sur d’autres sites (8507)

• Converted Avi to Mp4 using FFMPEG, Converted video not working in html 5 Tag

  21 mars 2015, par Suprabhat

  I have a section in my web page where user can upload any types of videos of any format , currently only restricted to .mp4 and .avi. After successfull upload i have displayed the same video to the user. I have bind the path in HTML5 video so that the user can view the content he/she has uploded. Video with extension .mp4 no doubt are working properly as HTML5 support them. Tricky part is it don’t support Avi files. Now here what the problem has arised. In order to display avi videos i have used FFMPEG to convert videos with extension .avi to .mp4. With lots of googling and reading forum, i have succesfully converted avi videos to mp4 . Here’s what i have used :-

  1. ffmpeg -i input.avi -acodec libfaac -b:a 128k -vcodec mpeg4 -b:v 1200k -flags +aic+mv4 output.mp4

  2. ffmpeg -i input.avi -c:v libx264 -b:a 128k -vcodec mpeg4 -b:v 1200k -flags +aic+mv4 output.mp4

  Above two are working perfectly, they have succesfully converted the video. But when i run them on browser in HTML5 and in new tab (Flash Player Plugin Installed), HTML5 doesn’t play it and flash player return an error message "Video can’t be played because the file is corrupt". But when i played them on KMplayer and in Window media player they are running perfectly.

  I have been to various threads in stackoverflow related to convert avi to mp4 and here i found following in one of the forum. where one of user has accepted this a correct answer but it ain’t worked out for me.

  3. ffmpeg -y -i sample.avi -b:v 1500k -vcodec libx264 -vpre slow -vpre baseline -g 30 sample.mp4

  Above argument returned me following error "File for preset ’slow’ not found".

  Following my futher searches i came across this thread ffmpeg convert mov file to mp4 for HTML5 video tag IE9. Here following argument worked perfectly and it able to convert video in such way that it is playble on browser.

  4. ffmpeg -y -i input.avi -vcodec libx264 -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -vf scale=-1:480 -threads 0 -acodec libvo_aacenc -b:a 128k -pix_fmt yuv420p output.mp4.

  Problem i faced here was video is converted to 420p reso which quality is noty upto mark. Smaller resolution videos are been enlarged and seems pixelated

  Thus, i had to finally put up a question. I will be very obliged if someone can give a solution for above problem. I need to convert an avi video to mp4 supported by HTML5 video tag. It should able to play it on browser and during conversion of video it should maintain original audio and video quality plus resolution.

  Thanks

  My C# Code :

          public void Create(string input, string output, string parametri, string ThumbnailPhysicalPath, int ConvertType)
  
          {
  
              ffmpeg = new Process();
  
  
  
              if (ConvertType == Convert.ToInt32(ConversionType.Thumbnail))
  
                  ffmpeg.StartInfo.Arguments = " -i \"" + input + "\" -vframes 1 \"" + output + "\"";
  
              else if (ConvertType == Convert.ToInt32(ConversionType.AviToMp4))
  
                  ffmpeg.StartInfo.Arguments = " -i \"" + input + "\" -c:v libx264 -b:a 128k -vcodec mpeg4 -b:v 1200k -flags +aic+mv4 \"" + output + "\"";
  
                  //ffmpeg.StartInfo.Arguments = " -i \"" + input + "\" -vcodec libx264 -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -vf scale=-1:480 -threads 0 -acodec libvo_aacenc -b:a 128k -pix_fmt yuv420p \"" + output + "\"";
  
              ffmpeg.StartInfo.FileName = ThumbnailPhysicalPath + @"ffmpeg.exe";
  
              ffmpeg.StartInfo.UseShellExecute = false;
  
              ffmpeg.StartInfo.RedirectStandardOutput = true;
  
              ffmpeg.StartInfo.RedirectStandardError = true;
  
              ffmpeg.StartInfo.CreateNoWindow = true;
  
              try
  
              {
  
                  ffmpeg.Start();
  
                  ffmpeg.WaitForExit();
  
                  string error = ffmpeg.StandardError.ReadToEnd();
  
              }
  
              catch (Exception Ex)
  
              {
  
                  Common.WriteLog("Exception occurred during conversion. Error Message :- " + Ex.Message + "\n Input Parameter :- " + input+ "\n Output Paramenter :- "+ output);
  
              }
  
              finally
  
              {
  
                  ffmpeg.Close();
  
                  if (ConvertType == Convert.ToInt32(ConversionType.AviToMp4))
  
                      UpdateConvertedVideoDetails(input,output);
  
              }
  
          }

  Command Prompt FFMPEG Output :-

  Sample 3 Result :-

  D:\Client\WebSite\Converter_Tools>ffmpeg -y -i sample.avi -b:v 1500k -vcodec libx264 -vpre slow -vpre baseline -g 30 sample.mp4
  
  ffmpeg version N-70239-g111d79a Copyright (c) 2000-2015 the FFmpeg developers
  
    built with gcc 4.9.2 (GCC)
  
    configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libblu
  
  ray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrw
  
  b --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --
  
  enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enab
  
  le-zlib
  
    libavutil      54. 19.100 / 54. 19.100
  
    libavcodec     56. 26.100 / 56. 26.100
  
    libavformat    56. 23.105 / 56. 23.105
  
    libavdevice    56.  4.100 / 56.  4.100
  
    libavfilter     5. 11.101 /  5. 11.101
  
    libswscale      3.  1.101 /  3.  1.101
  
    libswresample   1.  1.100 /  1.  1.100
  
    libpostproc    53.  3.100 / 53.  3.100
  
  [avi @ 037c8480] non-interleaved AVI
  
  Guessed Channel Layout for  Input Stream #0.1 : mono
  
  Input #0, avi, from 'sample.avi':
  
    Duration: 00:00:34.00, start: 0.000000, bitrate: 1433 kb/s
  
      Stream #0:0: Video: cinepak (cvid / 0x64697663), rgb24, 320x240, 15 fps, 15 tbr, 15 tbn, 15 tbc
  
      Stream #0:1: Audio: pcm_u8 ([1][0][0][0] / 0x0001), 22050 Hz, 1 channels, u8, 176 kb/s
  
  File for preset 'slow' not found

  Sample 4 Result :-

  D:\Client\WebSite\Converter_Tools>ffmpeg -y -i input.avi -vcodec libx264 -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -vf scale=-1:480 -threads 0 -acodec libvo_
  
  aacenc -b:a 128k -pix_fmt yuv420p output.mp4
  
  ffmpeg version N-70239-g111d79a Copyright (c) 2000-2015 the FFmpeg developers
  
    built with gcc 4.9.2 (GCC)
  
    configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libblu
  
  ray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrw
  
  b --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --
  
  enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enab
  
  le-zlib
  
    libavutil      54. 19.100 / 54. 19.100
  
    libavcodec     56. 26.100 / 56. 26.100
  
    libavformat    56. 23.105 / 56. 23.105
  
    libavdevice    56.  4.100 / 56.  4.100
  
    libavfilter     5. 11.101 /  5. 11.101
  
    libswscale      3.  1.101 /  3.  1.101
  
    libswresample   1.  1.100 /  1.  1.100
  
    libpostproc    53.  3.100 / 53.  3.100
  
  Input #0, avi, from 'input.avi':
  
    Duration: 00:00:03.93, start: 0.000000, bitrate: 3255 kb/s
  
      Stream #0:0: Video: msrle ([1][0][0][0] / 0x0001), pal8, 300x250, 3301 kb/s, 15 fps, 15 tbr, 15 tbn, 15 tbc
  
  [libx264 @ 002ec860] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
  
  [libx264 @ 002ec860] profile High, level 2.2
  
  [libx264 @ 002ec860] 264 - core 144 r2525 40bb568 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=5 deblock=1:0:0 analyse=0x3:0x113 me=umh subm
  
  e=8 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 i
  
  nterlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=50 rc
  
  =cbr mbtree=1 bitrate=500 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=500 vbv_bufsize=1000 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
  
  Output #0, mp4, to 'output.mp4':
  
    Metadata:
  
      encoder         : Lavf56.23.105
  
      Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 576x480, q=-1--1, 500 kb/s, 15 fps, 15360 tbn, 15 tbc
  
      Metadata:
  
        encoder         : Lavc56.26.100 libx264
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (msrle (native) -> h264 (libx264))
  
  Press [q] to stop, [?] for help
  
  frame=   59 fps= 30 q=-1.0 Lsize=     229kB time=00:00:03.80 bitrate= 493.5kbits/s
  
  video:227kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.637976%
  
  [libx264 @ 002ec860] frame I:3     Avg QP:26.53  size: 10657
  
  [libx264 @ 002ec860] frame P:25    Avg QP:30.49  size:  5608
  
  [libx264 @ 002ec860] frame B:31    Avg QP:32.26  size:  1935
  
  [libx264 @ 002ec860] consecutive B-frames: 22.0% 16.9% 20.3% 40.7%
  
  [libx264 @ 002ec860] mb I  I16..4: 16.7% 69.0% 14.4%
  
  [libx264 @ 002ec860] mb P  I16..4: 11.1% 29.9%  3.8%  P16..4: 21.3%  6.8%  2.6%  0.0%  0.0%    skip:24.6%
  
  [libx264 @ 002ec860] mb B  I16..4:  1.7%  3.0%  0.3%  B16..8: 29.7%  5.6%  0.8%  direct: 2.1%  skip:56.8%  L0:50.5% L1:45.6% BI: 3.9%
  
  [libx264 @ 002ec860] 8x8 transform intra:66.5% inter:79.4%
  
  [libx264 @ 002ec860] direct mvs  spatial:93.5% temporal:6.5%
  
  [libx264 @ 002ec860] coded y,uvDC,uvAC intra: 40.3% 48.8% 25.7% inter: 12.4% 8.4% 1.4%
  
  [libx264 @ 002ec860] i16 v,h,dc,p: 19% 59%  6% 17%
  
  [libx264 @ 002ec860] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 25% 16%  7%  8%  6% 11%  7% 10%
  
  [libx264 @ 002ec860] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 21%  9%  7%  9%  8% 10%  7% 10%
  
  [libx264 @ 002ec860] i8c dc,h,v,p: 41% 33% 13% 13%
  
  [libx264 @ 002ec860] Weighted P-Frames: Y:0.0% UV:0.0%
  
  [libx264 @ 002ec860] ref P L0: 67.6%  8.1%  9.6%  5.4%  6.6%  2.8%
  
  [libx264 @ 002ec860] ref B L0: 84.6% 10.5%  3.9%  1.0%
  
  [libx264 @ 002ec860] ref B L1: 95.9%  4.1%
  
  [libx264 @ 002ec860] kb/s:472.20

• Muxing Android MediaCodec encoded H264 packets into RTMP

  31 décembre 2015, par Vadym

  I am coming from a thread Encoding H.264 from camera with Android MediaCodec. My setup is very similar. However, I attempt to write mux the encoded frames and with javacv and broadcast them via rtmp.

  RtmpClient.java

  ...
  
  private volatile BlockingQueue mFrameQueue = new LinkedBlockingQueue(MAXIMUM_VIDEO_FRAME_BACKLOG);
  
  ...
  
  private void startStream() throws FrameRecorder.Exception, IOException {
  
      if (TextUtils.isEmpty(mDestination)) {
  
          throw new IllegalStateException("Cannot start RtmpClient without destination");
  
      }
  
  
  
      if (mCamera == null) {
  
          throw new IllegalStateException("Cannot start RtmpClient without camera.");
  
      }
  
  
  
      Camera.Parameters cameraParams = mCamera.getParameters();
  
  
  
      mRecorder = new FFmpegFrameRecorder(
  
              mDestination,
  
              mVideoQuality.resX,
  
              mVideoQuality.resY,
  
              (mAudioQuality.channelType.equals(AudioQuality.CHANNEL_TYPE_STEREO) ? 2 : 1));
  
  
  
      mRecorder.setFormat("flv");
  
  
  
      mRecorder.setFrameRate(mVideoQuality.frameRate);
  
      mRecorder.setVideoBitrate(mVideoQuality.bitRate);
  
      mRecorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
  
  
  
      mRecorder.setSampleRate(mAudioQuality.samplingRate);
  
      mRecorder.setAudioBitrate(mAudioQuality.bitRate);
  
      mRecorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
  
  
  
      mVideoStream = new VideoStream(mRecorder, mVideoQuality, mFrameQueue, mCamera);
  
      mAudioStream = new AudioStream(mRecorder, mAudioQuality);
  
  
  
      mRecorder.start();
  
  
  
      // Setup a bufferred preview callback
  
      setupCameraCallback(mCamera, mRtmpClient, DEFAULT_PREVIEW_CALLBACK_BUFFERS,
  
              mVideoQuality.resX * mVideoQuality.resY * ImageFormat.getBitsPerPixel(
  
                      cameraParams.getPreviewFormat())/8);
  
  
  
      try {
  
          mVideoStream.start();
  
          mAudioStream.start();
  
      }
  
      catch(Exception e) {
  
          e.printStackTrace();
  
          stopStream();
  
      }
  
  }
  
  ...
  
  @Override
  
  public void onPreviewFrame(byte[] data, Camera camera) {
  
      boolean frameQueued = false;
  
  
  
      if (mRecorder == null || data == null) {
  
          return;
  
      }
  
  
  
      frameQueued = mFrameQueue.offer(data);
  
  
  
      // return the buffer to be reused - done in videostream
  
      //camera.addCallbackBuffer(data);
  
  }
  
  ...

  VideoStream.java

  ...
  
  @Override
  
  public void run() {
  
      try {
  
          mMediaCodec = MediaCodec.createEncoderByType("video/avc");
  
          MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", mVideoQuality.resX, mVideoQuality.resY);
  
          mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, mVideoQuality.bitRate);
  
          mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, mVideoQuality.frameRate);
  
          mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);
  
          mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);
  
          mMediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
  
          mMediaCodec.start();
  
      }
  
      catch(IOException e) {
  
          e.printStackTrace();
  
      }
  
  
  
      long startTimestamp = System.currentTimeMillis();
  
      long frameTimestamp = 0;
  
      byte[] rawFrame = null;
  
  
  
      try {
  
          while (!Thread.interrupted()) {
  
              rawFrame = mFrameQueue.take();
  
  
  
              frameTimestamp = 1000 * (System.currentTimeMillis() - startTimestamp);
  
  
  
              encodeFrame(rawFrame, frameTimestamp);
  
  
  
              // return the buffer to be reused
  
              mCamera.addCallbackBuffer(rawFrame);
  
          }
  
      }
  
      catch (InterruptedException ignore) {
  
          // ignore interrup while waiting
  
      }
  
  
  
      // Clean up video stream allocations
  
      try {
  
          mMediaCodec.stop();
  
          mMediaCodec.release();
  
          mOutputStream.flush();
  
          mOutputStream.close();
  
      } catch (Exception e){
  
          e.printStackTrace();
  
      }
  
  }
  
  ...
  
  private void encodeFrame(byte[] input, long timestamp) {
  
      try {
  
          ByteBuffer[] inputBuffers = mMediaCodec.getInputBuffers();
  
          ByteBuffer[] outputBuffers = mMediaCodec.getOutputBuffers();
  
  
  
          int inputBufferIndex = mMediaCodec.dequeueInputBuffer(0);
  
  
  
          if (inputBufferIndex >= 0) {
  
              ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
  
              inputBuffer.clear();
  
              inputBuffer.put(input);
  
              mMediaCodec.queueInputBuffer(inputBufferIndex, 0, input.length, timestamp, 0);
  
          }
  
  
  
          MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
  
  
  
          int outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, 0);
  
  
  
          if (outputBufferIndex >= 0) {
  
              while (outputBufferIndex >= 0) {
  
                  ByteBuffer outputBuffer = outputBuffers[outputBufferIndex];
  
  
  
                  // Should this be a direct byte buffer?
  
                  byte[] outData = new byte[bufferInfo.size - bufferInfo.offset];
  
                  outputBuffer.get(outData);
  
  
  
                  mFrameRecorder.record(outData, bufferInfo.offset, outData.length, timestamp);
  
  
  
                  mMediaCodec.releaseOutputBuffer(outputBufferIndex, false);
  
                  outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, 0);
  
              }
  
          }
  
          else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
  
              outputBuffers = mMediaCodec.getOutputBuffers();
  
          } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
  
              // ignore for now
  
          }
  
      } catch (Throwable t) {
  
          t.printStackTrace();
  
      }
  
  
  
  }
  
  ...

  FFmpegFrameRecorder.java

  ...
  
  // Hackish codec copy frame recording function
  
  public boolean record(byte[] encodedData, int offset, int length, long frameCount) throws Exception {
  
      int ret;
  
  
  
      if (encodedData == null) {
  
          return false;
  
      }
  
  
  
      av_init_packet(video_pkt);
  
  
  
      // this is why i wondered whether I should get outputbuffer data into direct byte buffer
  
      video_outbuf.put(encodedData, 0, encodedData.length);
  
  
  
      video_pkt.data(video_outbuf);
  
      video_pkt.size(video_outbuf_size);
  
  
  
      video_pkt.pts(frameCount);
  
      video_pkt.dts(frameCount);
  
  
  
      video_pkt.stream_index(video_st.index());
  
  
  
      synchronized (oc) {
  
          /* write the compressed frame in the media file */
  
          if (interleaved && audio_st != null) {
  
              if ((ret = av_interleaved_write_frame(oc, video_pkt)) < 0) {
  
                  throw new Exception("av_interleaved_write_frame() error " + ret + " while writing interleaved video frame.");
  
              }
  
          } else {
  
              if ((ret = av_write_frame(oc, video_pkt)) < 0) {
  
                  throw new Exception("av_write_frame() error " + ret + " while writing video frame.");
  
              }
  
          }
  
      }
  
      return (video_pkt.flags() & AV_PKT_FLAG_KEY) == 1;
  
  }
  
  ...

  When I try to stream the video and run ffprobe on it, I get the following output :

  ffprobe version 2.5.3 Copyright (c) 2007-2015 the FFmpeg developers
  
    built on Jan 19 2015 12:56:57 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-55)
  
    configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --enable-bzlib --disable-crystalhd --enable-libass --enable-libdc1394 --enable-libfaac --enable-nonfree --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-avresample --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --enable-libcaca --shlibdir=/usr/lib64 --enable-runtime-cpudetect
  
    libavutil      54. 15.100 / 54. 15.100
  
    libavcodec     56. 13.100 / 56. 13.100
  
    libavformat    56. 15.102 / 56. 15.102
  
    libavdevice    56.  3.100 / 56.  3.100
  
    libavfilter     5.  2.103 /  5.  2.103
  
    libavresample   2.  1.  0 /  2.  1.  0
  
    libswscale      3.  1.101 /  3.  1.101
  
    libswresample   1.  1.100 /  1.  1.100
  
    libpostproc    53.  3.100 / 53.  3.100
  
  Metadata:
  
    Server                NGINX RTMP (github.com/arut/nginx-rtmp-module)
  
    width                 320.00
  
    height                240.00
  
    displayWidth          320.00
  
    displayHeight         240.00
  
    duration              0.00
  
    framerate             0.00
  
    fps                   0.00
  
    videodatarate         261.00
  
    videocodecid          7.00
  
    audiodatarate         62.00
  
    audiocodecid          10.00
  
    profile
  
    level
  
  [live_flv @ 0x1edb0820] Could not find codec parameters for stream 0 (Video: none, none, 267 kb/s): unknown codec
  
  Consider increasing the value for the 'analyzeduration' and 'probesize' options
  
  Input #0, live_flv, from 'rtmp://<server>/input/<stream>':
  
    Metadata:
  
      Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
  
      displayWidth    : 320
  
      displayHeight   : 240
  
      fps             : 0
  
      profile         :
  
      level           :
  
    Duration: 00:00:00.00, start: 16.768000, bitrate: N/A
  
      Stream #0:0: Video: none, none, 267 kb/s, 1k tbr, 1k tbn, 1k tbc
  
      Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp, 63 kb/s
  
  Unsupported codec with id 0 for input stream 0
  
  </stream></server>

  I am not, by any means, an expert in H264 or video encoding. I know that the encoded frames that come out from MediaCodec contain SPS NAL, PPS NAL, and frame NAL units. I’ve also written the MediaCodec output into a file and was able to play it back (I did have to specify the format and framerate as otherwise it would play too fast).

  My assumption is that things should work (see how little I know :)). Knowing that SPS and PPS are written out, decoder should know enough. Yet, ffprobe fails to recognize codec, fps, and other video information. Do I need to pass packet flag information to FFmpegFrameRecorder.java:record() function ? Or should I use direct buffer ? Any suggestion will be appreciated ! I should figure things out with a hint.

  PS : I know that some codecs use Planar and other SemiPlanar color formats. That distinction will come later if I get past this. Also, I didn’t go the Surface to MediaCodec way because I need to support API 17 and it requires more changes than this route, which I think helps me understand the more basic flow. Agan, I appreciate any suggestions. Please let me know if something needs to be clarified.

  Update #1

  So having done more testing, I see that my encoder outputs the following frames :

  000000016742800DDA0507E806D0A1350000000168CE06E2
  
  0000000165B840A6F1E7F0EA24000AE73BEB5F51CC7000233A84240...
  
  0000000141E2031364E387FD4F9BB3D67F51CC7000279B9F9CFE811...
  
  0000000141E40304423FFFFF0B7867F89FAFFFFFFFFFFCBE8EF25E6...
  
  0000000141E602899A3512EF8AEAD1379F0650CC3F905131504F839...
  
  ...

  The very first frame contains SPS and PPS. From what I was able to see, these are transmitted only once. The rest are NAL types 1 and 5. So, my assumption is that, for ffprobe to see stream info not only when the stream starts, I should capture SPS and PPS frames and re-transmit them myself periodically, after a certain number of frames, or perhaps before every I-frame. What do you think ?

  Update #2

  Unable to validate that I’m writing frames successfully. After having tried to read back the written packet, I cannot validate written bytes. As strange, on successful write of IPL image and streaming, I also cannot print out bytes of encoded packet after avcodec_encode_video2. Hit the official dead end.

• Video Recording on Android from Frames using javaCV

  23 juin 2016, par jawad bin zafar

  I am using this library javaCV to record video on android. They have provided a sample VideoRecording Activity But There is some bug which I could not figure out what I am doing it wrong or what is missing which causing this bug.

  package org.bytedeco.javacv.recordactivity;
  
  
  
  import android.app.Activity;
  
  import android.content.Context;
  
  import android.content.pm.ActivityInfo;
  
  import android.hardware.Camera;
  
  import android.hardware.Camera.PreviewCallback;
  
  import android.media.AudioFormat;
  
  import android.media.AudioRecord;
  
  import android.media.MediaRecorder;
  
  import android.os.Bundle;
  
  import android.os.PowerManager;
  
  import android.util.Log;
  
  import android.view.Display;
  
  import android.view.KeyEvent;
  
  import android.view.LayoutInflater;
  
  import android.view.SurfaceHolder;
  
  import android.view.SurfaceView;
  
  import android.view.View;
  
  import android.view.View.OnClickListener;
  
  import android.view.WindowManager;
  
  import android.widget.Button;
  
  import android.widget.LinearLayout;
  
  import android.widget.RelativeLayout;
  
  
  
  import java.io.IOException;
  
  import java.nio.ShortBuffer;
  
  
  
  import org.bytedeco.javacv.FFmpegFrameRecorder;
  
  
  
  import static org.bytedeco.javacpp.opencv_core.*;
  
  
  
  public class RecordActivity extends Activity implements OnClickListener {
  
  
  
      private final static String CLASS_LABEL = "RecordActivity";
  
      private final static String LOG_TAG = CLASS_LABEL;
  
  
  
      private PowerManager.WakeLock mWakeLock;
  
  
  
      private String ffmpeg_link = "/mnt/sdcard/stream.flv";
  
  
  
      long startTime = 0;
  
      boolean recording = false;
  
  
  
      private volatile FFmpegFrameRecorder recorder;
  
  
  
      private boolean isPreviewOn = false;
  
  
  
      private int sampleAudioRateInHz = 44100;
  
      private int imageWidth = 320;
  
      private int imageHeight = 240;
  
      private int frameRate = 30;
  
  
  
      /* audio data getting thread */
  
      private AudioRecord audioRecord;
  
      private AudioRecordRunnable audioRecordRunnable;
  
      private Thread audioThread;
  
      volatile boolean runAudioThread = true;
  
  
  
      /* video data getting thread */
  
      private Camera cameraDevice;
  
      private CameraView cameraView;
  
  
  
      private IplImage yuvIplimage = null;
  
  
  
      /* layout setting */
  
      private final int bg_screen_bx = 232;
  
      private final int bg_screen_by = 128;
  
      private final int bg_screen_width = 700;
  
      private final int bg_screen_height = 500;
  
      private final int bg_width = 1123;
  
      private final int bg_height = 715;
  
      private final int live_width = 640;
  
      private final int live_height = 480;
  
      private int screenWidth, screenHeight;
  
      private Button btnRecorderControl;
  
  
  
      /** The number of seconds in the continuous record loop (or 0 to disable loop). */
  
      final int RECORD_LENGTH = 10;
  
      IplImage[] images;
  
      long[] timestamps;
  
      ShortBuffer[] samples;
  
      int imagesIndex, samplesIndex;
  
  
  
      @Override
  
      public void onCreate(Bundle savedInstanceState) {
  
          super.onCreate(savedInstanceState);
  
          setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
  
  
  
          setContentView(R.layout.main);
  
  
  
          PowerManager pm = (PowerManager) getSystemService(Context.POWER_SERVICE); 
  
          mWakeLock = pm.newWakeLock(PowerManager.SCREEN_BRIGHT_WAKE_LOCK, CLASS_LABEL); 
  
          mWakeLock.acquire(); 
  
  
  
          initLayout();
  
      }
  
  
  
  
  
      @Override
  
      protected void onResume() {
  
          super.onResume();
  
  
  
          if (mWakeLock == null) {
  
             PowerManager pm = (PowerManager) getSystemService(Context.POWER_SERVICE);
  
             mWakeLock = pm.newWakeLock(PowerManager.SCREEN_BRIGHT_WAKE_LOCK, CLASS_LABEL);
  
             mWakeLock.acquire();
  
          }
  
      }
  
  
  
      @Override
  
      protected void onPause() {
  
          super.onPause();
  
  
  
          if (mWakeLock != null) {
  
              mWakeLock.release();
  
              mWakeLock = null;
  
          }
  
      }
  
  
  
      @Override
  
      protected void onDestroy() {
  
          super.onDestroy();
  
  
  
          recording = false;
  
  
  
          if (cameraView != null) {
  
              cameraView.stopPreview();
  
          }
  
  
  
          if(cameraDevice != null) {
  
             cameraDevice.stopPreview();
  
             cameraDevice.release();
  
             cameraDevice = null;
  
          }
  
  
  
          if (mWakeLock != null) {
  
              mWakeLock.release();
  
              mWakeLock = null;
  
          }
  
      }
  
  
  
  
  
      private void initLayout() {
  
  
  
          /* get size of screen */
  
          Display display = ((WindowManager) getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay();
  
          screenWidth = display.getWidth();
  
          screenHeight = display.getHeight();
  
          RelativeLayout.LayoutParams layoutParam = null; 
  
          LayoutInflater myInflate = null; 
  
          myInflate = (LayoutInflater) getSystemService(Context.LAYOUT_INFLATER_SERVICE);
  
          RelativeLayout topLayout = new RelativeLayout(this);
  
          setContentView(topLayout);
  
          LinearLayout preViewLayout = (LinearLayout) myInflate.inflate(R.layout.main, null);
  
          layoutParam = new RelativeLayout.LayoutParams(screenWidth, screenHeight);
  
          topLayout.addView(preViewLayout, layoutParam);
  
  
  
          /* add control button: start and stop */
  
          btnRecorderControl = (Button) findViewById(R.id.recorder_control);
  
          btnRecorderControl.setText("Start");
  
          btnRecorderControl.setOnClickListener(this);
  
  
  
          /* add camera view */
  
          int display_width_d = (int) (1.0 * bg_screen_width * screenWidth / bg_width);
  
          int display_height_d = (int) (1.0 * bg_screen_height * screenHeight / bg_height);
  
          int prev_rw, prev_rh;
  
          if (1.0 * display_width_d / display_height_d > 1.0 * live_width / live_height) {
  
              prev_rh = display_height_d;
  
              prev_rw = (int) (1.0 * display_height_d * live_width / live_height);
  
          } else {
  
              prev_rw = display_width_d;
  
              prev_rh = (int) (1.0 * display_width_d * live_height / live_width);
  
          }
  
          layoutParam = new RelativeLayout.LayoutParams(prev_rw, prev_rh);
  
          layoutParam.topMargin = (int) (1.0 * bg_screen_by * screenHeight / bg_height);
  
          layoutParam.leftMargin = (int) (1.0 * bg_screen_bx * screenWidth / bg_width);
  
  
  
          cameraDevice = Camera.open();
  
          Log.i(LOG_TAG, "cameara open");
  
          cameraView = new CameraView(this, cameraDevice);
  
          topLayout.addView(cameraView, layoutParam);
  
          Log.i(LOG_TAG, "cameara preview start: OK");
  
      }
  
  
  
      //---------------------------------------
  
      // initialize ffmpeg_recorder
  
      //---------------------------------------
  
      private void initRecorder() {
  
  
  
          Log.w(LOG_TAG,"init recorder");
  
  
  
          if (RECORD_LENGTH > 0) {
  
              imagesIndex = 0;
  
              images = new IplImage[RECORD_LENGTH * frameRate];
  
              timestamps = new long[images.length];
  
              for (int i = 0; i < images.length; i++) {
  
                  images[i] = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_8U, 2);
  
                  timestamps[i] = -1;
  
              }
  
          } else if (yuvIplimage == null) {
  
              yuvIplimage = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_8U, 2);
  
              Log.i(LOG_TAG, "create yuvIplimage");
  
          }
  
  
  
          Log.i(LOG_TAG, "ffmpeg_url: " + ffmpeg_link);
  
          recorder = new FFmpegFrameRecorder(ffmpeg_link, imageWidth, imageHeight, 1);
  
          recorder.setFormat("flv");
  
          recorder.setSampleRate(sampleAudioRateInHz);
  
          // Set in the surface changed method
  
          recorder.setFrameRate(frameRate);
  
  
  
          Log.i(LOG_TAG, "recorder initialize success");
  
  
  
          audioRecordRunnable = new AudioRecordRunnable();
  
          audioThread = new Thread(audioRecordRunnable);
  
          runAudioThread = true;
  
      }
  
  
  
      public void startRecording() {
  
  
  
          initRecorder();
  
  
  
          try {
  
              recorder.start();
  
              startTime = System.currentTimeMillis();
  
              recording = true;
  
              audioThread.start();
  
  
  
          } catch (FFmpegFrameRecorder.Exception e) {
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      public void stopRecording() {
  
  
  
          runAudioThread = false;
  
          try {
  
              audioThread.join();
  
          } catch (InterruptedException e) {
  
              e.printStackTrace();
  
          }
  
          audioRecordRunnable = null;
  
          audioThread = null;
  
  
  
          if (recorder != null && recording) {
  
              if (RECORD_LENGTH > 0) {
  
                  Log.v(LOG_TAG,"Writing frames");
  
                  try {
  
                      int firstIndex = imagesIndex % samples.length;
  
                      int lastIndex = (imagesIndex - 1) % images.length;
  
                      if (imagesIndex <= images.length) {
  
                          firstIndex = 0;
  
                          lastIndex = imagesIndex - 1;
  
                      }
  
                      if ((startTime = timestamps[lastIndex] - RECORD_LENGTH * 1000000L) < 0) {
  
                          startTime = 0;
  
                      }
  
                      if (lastIndex < firstIndex) {
  
                          lastIndex += images.length;
  
                      }
  
                      for (int i = firstIndex; i <= lastIndex; i++) {
  
                          long t = timestamps[i % timestamps.length] - startTime;
  
                          if (t >= 0) {
  
                              if (t > recorder.getTimestamp()) {
  
                                  recorder.setTimestamp(t);
  
                              }
  
                              recorder.record(images[i % images.length]);
  
                          }
  
                      }
  
  
  
                      firstIndex = samplesIndex % samples.length;
  
                      lastIndex = (samplesIndex - 1) % samples.length;
  
                      if (samplesIndex <= samples.length) {
  
                          firstIndex = 0;
  
                          lastIndex = samplesIndex - 1;
  
                      }
  
                      if (lastIndex < firstIndex) {
  
                          lastIndex += samples.length;
  
                      }
  
                      for (int i = firstIndex; i <= lastIndex; i++) {
  
                          recorder.record(samples[i % samples.length]);
  
                      }
  
                  } catch (FFmpegFrameRecorder.Exception e) {
  
                      Log.v(LOG_TAG,e.getMessage());
  
                      e.printStackTrace();
  
                  }
  
              }
  
  
  
              recording = false;
  
              Log.v(LOG_TAG,"Finishing recording, calling stop and release on recorder");
  
              try {
  
                  recorder.stop();
  
                  recorder.release();
  
              } catch (FFmpegFrameRecorder.Exception e) {
  
                  e.printStackTrace();
  
              }
  
              recorder = null;
  
  
  
          }
  
      }
  
  
  
      @Override
  
      public boolean onKeyDown(int keyCode, KeyEvent event) {
  
  
  
          if (keyCode == KeyEvent.KEYCODE_BACK) {
  
              if (recording) {
  
                  stopRecording();
  
              }
  
  
  
              finish();
  
  
  
              return true;
  
          }
  
  
  
          return super.onKeyDown(keyCode, event);
  
      }
  
  
  
  
  
      //---------------------------------------------
  
      // audio thread, gets and encodes audio data
  
      //---------------------------------------------
  
      class AudioRecordRunnable implements Runnable {
  
  
  
          @Override
  
          public void run() {
  
              android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
  
  
  
              // Audio
  
              int bufferSize;
  
              ShortBuffer audioData;
  
              int bufferReadResult;
  
  
  
              bufferSize = AudioRecord.getMinBufferSize(sampleAudioRateInHz, 
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
  
              audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleAudioRateInHz, 
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
  
  
  
              if (RECORD_LENGTH > 0) {
  
                  samplesIndex = 0;
  
                  samples = new ShortBuffer[RECORD_LENGTH * sampleAudioRateInHz * 2 / bufferSize + 1];
  
                  for (int i = 0; i < samples.length; i++) {
  
                      samples[i] = ShortBuffer.allocate(bufferSize);
  
                  }
  
              } else {
  
                  audioData = ShortBuffer.allocate(bufferSize);
  
              }
  
  
  
              Log.d(LOG_TAG, "audioRecord.startRecording()");
  
              audioRecord.startRecording();
  
  
  
              /* ffmpeg_audio encoding loop */
  
              while (runAudioThread) {
  
                  if (RECORD_LENGTH > 0) {
  
                      audioData = samples[samplesIndex++ % samples.length];
  
                      audioData.position(0).limit(0);
  
                  }
  
                  //Log.v(LOG_TAG,"recording? " + recording);
  
                  bufferReadResult = audioRecord.read(audioData.array(), 0, audioData.capacity());
  
                  audioData.limit(bufferReadResult);
  
                  if (bufferReadResult > 0) {
  
                      Log.v(LOG_TAG,"bufferReadResult: " + bufferReadResult);
  
                      // If "recording" isn't true when start this thread, it never get's set according to this if statement...!!!
  
                      // Why?  Good question...
  
                      if (recording) {
  
                          if (RECORD_LENGTH <= 0) try {
  
                              recorder.record(audioData);
  
                              //Log.v(LOG_TAG,"recording " + 1024*i + " to " + 1024*i+1024);
  
                          } catch (FFmpegFrameRecorder.Exception e) {
  
                              Log.v(LOG_TAG,e.getMessage());
  
                              e.printStackTrace();
  
                          }
  
                      }
  
                  }
  
              }
  
              Log.v(LOG_TAG,"AudioThread Finished, release audioRecord");
  
  
  
              /* encoding finish, release recorder */
  
              if (audioRecord != null) {
  
                  audioRecord.stop();
  
                  audioRecord.release();
  
                  audioRecord = null;
  
                  Log.v(LOG_TAG,"audioRecord released");
  
              }
  
          }
  
      }
  
  
  
      //---------------------------------------------
  
      // camera thread, gets and encodes video data
  
      //---------------------------------------------
  
      class CameraView extends SurfaceView implements SurfaceHolder.Callback, PreviewCallback {
  
  
  
          private SurfaceHolder mHolder;
  
          private Camera mCamera;
  
  
  
          public CameraView(Context context, Camera camera) {
  
              super(context);
  
              Log.w("camera","camera view");
  
              mCamera = camera;
  
              mHolder = getHolder();
  
              mHolder.addCallback(CameraView.this);
  
              mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
  
              mCamera.setPreviewCallback(CameraView.this);
  
          }
  
  
  
          @Override
  
          public void surfaceCreated(SurfaceHolder holder) {
  
              try {
  
                  stopPreview();
  
                  mCamera.setPreviewDisplay(holder);
  
              } catch (IOException exception) {
  
                  mCamera.release();
  
                  mCamera = null;
  
              }
  
          }
  
  
  
          public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
  
              Log.v(LOG_TAG,"Setting imageWidth: " + imageWidth + " imageHeight: " + imageHeight + " frameRate: " + frameRate);
  
              Camera.Parameters camParams = mCamera.getParameters();
  
              camParams.setPreviewSize(imageWidth, imageHeight);
  
  
  
              Log.v(LOG_TAG,"Preview Framerate: " + camParams.getPreviewFrameRate());
  
  
  
              camParams.setPreviewFrameRate(frameRate);
  
              mCamera.setParameters(camParams);
  
              startPreview();
  
          }
  
  
  
          @Override
  
          public void surfaceDestroyed(SurfaceHolder holder) {
  
              try {
  
                  mHolder.addCallback(null);
  
                  mCamera.setPreviewCallback(null);
  
              } catch (RuntimeException e) {
  
                  // The camera has probably just been released, ignore.
  
              }
  
          }
  
  
  
          public void startPreview() {
  
              if (!isPreviewOn && mCamera != null) {
  
                  isPreviewOn = true;
  
                  mCamera.startPreview();
  
              }
  
          }
  
  
  
          public void stopPreview() {
  
              if (isPreviewOn && mCamera != null) {
  
                  isPreviewOn = false;
  
                  mCamera.stopPreview();
  
              }
  
          }
  
  
  
          @Override
  
          public void onPreviewFrame(byte[] data, Camera camera) {
  
              if (audioRecord == null || audioRecord.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING) {
  
                  startTime = System.currentTimeMillis();
  
                  return;
  
              }
  
              if (RECORD_LENGTH > 0) {
  
                  int i = imagesIndex++ % images.length;
  
                  yuvIplimage = images[i];
  
                  timestamps[i] = 1000 * (System.currentTimeMillis() - startTime);
  
              }
  
              /* get video data */
  
              if (yuvIplimage != null && recording) {
  
                  yuvIplimage.getByteBuffer().put(data);
  
  
  
                  if (RECORD_LENGTH <= 0) try {
  
                      Log.v(LOG_TAG,"Writing Frame");
  
                      long t = 1000 * (System.currentTimeMillis() - startTime);
  
                      if (t > recorder.getTimestamp()) {
  
                          recorder.setTimestamp(t);
  
                      }
  
                      recorder.record(yuvIplimage);
  
                  } catch (FFmpegFrameRecorder.Exception e) {
  
                      Log.v(LOG_TAG,e.getMessage());
  
                      e.printStackTrace();
  
                  }
  
              }
  
          }
  
      }
  
  
  
      @Override
  
      public void onClick(View v) {
  
          if (!recording) {
  
              startRecording();
  
              Log.w(LOG_TAG, "Start Button Pushed");
  
              btnRecorderControl.setText("Stop");
  
          } else {
  
              // This will trigger the audio recording loop to stop and then set isRecorderStart = false;
  
              stopRecording();
  
              Log.w(LOG_TAG, "Stop Button Pushed");
  
              btnRecorderControl.setText("Start");
  
          }
  
      }
  
  }

  I am getting this error

  The type org.bytedeco.javacpp.avutil$AVFrame cannot be resolved. It is indirectly referenced from required .class files

  at following line in the above code

  if (RECORD_LENGTH <= 0) try {
  
      recorder.record(audioData);
  
      //Log.v(LOG_TAG,"recording " + 1024*i + " to " + 1024*i+1024);
  
   }

  at recorder.record(audioData). I don’t know what I am doing wrong here. New to JavaCV. Any help will be appreciated.