Recherche avancée

Sur d’autres sites (2560)

• FFmpeg muxing to avi

  2 septembre 2015, par Uncia

  sI have program, that succefully shows h264 stream using SDL : I’m getting h264 frame, decode it using ffmpeg and draw using SDL.
  Also I can write frames to file (using fwrite) and play this file through ffplay.

  But I want to mux data to the avi and face some problems in av_write_frame.

  Here is my code :

  ...
  
  /*Initializing format context - outFormatContext is the member of my class*/
  
  AVOutputFormat *outFormat;
  
  outFormat = av_guess_format(NULL,"out.avi",NULL);
  
  outFormat->video_codec = AV_CODEC_ID_H264;
  
  outFormat->audio_codec = AV_CODEC_ID_NONE;
  
  avformat_alloc_output_context2(&outFormatContext, outFormat, NULL, "out.avi");
  
  AVCodec *outCodec;
  
  AVStream *outStream = add_stream(outFormatContext, &outCodec, outFormatContext->oformat->video_codec);
  
  avcodec_open2(outStream->codec, outCodec, NULL);
  
  av_dump_format(outFormatContext, 0, "out.avi", 1);
  
  if (avio_open(&outFormatContext->pb, "out.avi", AVIO_FLAG_WRITE) < 0) 
  
      throw Exception("Couldn't open file");
  
  if (avformat_write_header(outFormatContext, NULL) < 0)
  
      throw Exception("Couldn't write to file");
  
   //I don't have exceptions here - so there is 6KB header in out.avi.
  
   ...
  
  
  
  static AVStream *add_stream(AVFormatContext *oc, AVCodec **codec,
  
                          enum AVCodecID codec_id)
  
  {
  
  AVCodecContext *c;
  
  AVStream *st;
  
  /* find the encoder */
  
  *codec = avcodec_find_encoder(codec_id);
  
  if (!(*codec)) 
  
      throw("Could not find encoder");
  
  st = avformat_new_stream(oc, *codec);
  
  if (!st) 
  
      throw ("Could not allocate stream");
  
  st->id = oc->nb_streams-1;
  
  c = st->codec;
  
  c->bit_rate = 400000;
  
  /* Resolution must be a multiple of two. */
  
  c->width    = 1920;
  
  c->height   = 1080;
  
  c->pix_fmt  = PIX_FMT_YUV420P;
  
  c->flags = 0;
  
  c->time_base.num = 1;
  
  c->time_base.den = 25;
  
  c->gop_size      = 12; /* emit one intra frame every twelve frames at most */
  
  return st;
  
  }
  
  ...
  
  /* Part of decoding loop. There is AVPacket packet - h264 packet;
  
  int ret = av_write_frame(outFormatContext, &packet); //it return -22 code - Invadlid argument;
  
  if (avcodec_decode_video2(pCodecCtx, pFrame, &frameDecoded, &packet) < 0) 
  
      return;
  
  if (frameDecoded)
  
  {
  
     //SDL stuff
  
  }

  Also i tried to use avcodec_encode_video2 (encode pFrame back to the H264) next to the SDL stuff but encoding is not working - i’ve got empty packets :( It is the second problem.

  Using av_interleaved_write_frame causes acces violation.

  Code of the muxing part i copied from ffmpeg muxing example (https://www.ffmpeg.org/doxygen/2.1/doc_2examples_2muxing_8c-example.html)

• APPLY Strong to Buffer rule. Quit Switching Bit rates MPEG DASH

  14 juillet 2015, par Vinay

  I am using mpeg dash for adaptive bit rate streaming of video from my server.

  I have used ffmpeg and MP4Box to generate 4 different quality video files from my source .mp4

  The .mpd file generated has the below code

  <?xml version="1.0"?>
  
  
  
  <mpd xmlns="urn:mpeg:dash:schema:mpd:2011" minbuffertime="PT1.500000S" type="static" mediapresentationduration="PT0H3M1.42S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011">
  
   <programinformation moreinformationurl="http://gpac.sourceforge.net">
  
    
  
   </programinformation>
  
  
  
   <period duration="PT0H3M1.42S">
  
    <adaptationset segmentalignment="true" maxwidth="1920" maxheight="1080" maxframerate="24" par="16:9" lang="und" subsegmentstartswithsap="1">
  
     <representation mimetype="video/mp4" codecs="avc1.64000d" width="320" height="240" framerate="24" sar="1:1" startwithsap="1" bandwidth="375715">
  
      <baseurl>400_dashinit.mp4</baseurl>
  
      <segmentbase indexrangeexact="true" indexrange="904-1403">
  
        <initialization range="0-903"></initialization>
  
      </segmentbase>
  
     </representation>
  
     <representation mimetype="video/mp4" codecs="avc1.640015" width="420" height="270" framerate="24" sar="1:1" startwithsap="1" bandwidth="644824">
  
      <baseurl>700_dashinit.mp4</baseurl>
  
      <segmentbase indexrangeexact="true" indexrange="905-1404">
  
        <initialization range="0-904"></initialization>
  
      </segmentbase>
  
     </representation>
  
     <representation mimetype="video/mp4" codecs="avc1.64001f" width="1024" height="576" framerate="24" sar="1:1" startwithsap="1" bandwidth="1349484">
  
      <baseurl>1500_dashinit.mp4</baseurl>
  
      <segmentbase indexrangeexact="true" indexrange="905-1404">
  
        <initialization range="0-904"></initialization>
  
      </segmentbase>
  
     </representation>
  
     <representation mimetype="video/mp4" codecs="avc1.64001f" width="1280" height="720" framerate="24" sar="1:1" startwithsap="1" bandwidth="2264379">
  
      <baseurl>2500_dashinit.mp4</baseurl>
  
      <segmentbase indexrangeexact="true" indexrange="905-1404">
  
        <initialization range="0-904"></initialization>
  
      </segmentbase>
  
     </representation>
  
     <representation mimetype="video/mp4" codecs="avc1.640028" width="1920" height="1080" framerate="24" sar="1:1" startwithsap="1" bandwidth="3633049">
  
      <baseurl>4000_dashinit.mp4</baseurl>
  
      <segmentbase indexrangeexact="true" indexrange="906-1405">
  
        <initialization range="0-905"></initialization>
  
      </segmentbase>
  
     </representation>
  
    </adaptationset>
  
   </period>
  
  </mpd>

  I am using video.js along with dash.js to playback the mpeg dash content on client side. The issue is that the video doesn’t playback perfectly when i simulate network conditions from chrome dev tools.

  It works at times and it doesn’t at others. For ex the stream starts with bit rate of 400kbps and then detects enough bandwidth available so it switches to 2500kbps. Then when i bring down my bandwidth to 400kbps again then the video freezes at some point of time.

  At times the video freezes after few initial seconds of playback when it tries to switch the stream. I think there might be some command line parameter that i am missing while generating my video files via ffmpeg or generating .mpd file via MP4Box.

  below are the commands i use for ffmpeg and MP4Box

  ffmpeg -y -i inputfile -c:a libfdk_aac -ac 2 -ab 128k -c:v libx264 -r 24 – g 24 -b:v 1500k -maxrate 1500k -bufsize 1000k -vf "scale=-1:720" outputfile.mp4
  
  
  
  
  
  MP4Box -dash [DURATION] -rap -frag-rap -profile [PROFILE] -out [path/to/outpout.file] [path/to/input1.file] [path/to/input2.file] [path/to/input3.file]

  Also while i am generating .mpd files via MP4Box i am getting below warning in console

  [DASH]: Files have non-proportional track layouts (320x240 vs 420x270) but sample size and aspect ratio match, assuming precision issue
  
  [DASH]: Files have non-proportional track layouts (320x240 vs 1024x576) but sample size and aspect ratio match, assuming precision issue
  
  [DASH]: Files have non-proportional track layouts (320x240 vs 1280x720) but sample size and aspect ratio match, assuming precision issue
  
  [DASH]: Files have non-proportional track layouts (320x240 vs 1920x1080) but sample size and aspect ratio match, assuming precision issue

  Whenever the video stops playing the chrome console has these logs

  Number of times the buffer has run dry: 25
  
  Apply STRONG to buffer rule.
  
  Quit switching bit rates.

  I don’t have any clue as to why the buffers run dry and it stops switching the bit rates.

  Anything that is predominantly wrong in the process ?

• Screen Recording with FFmpeg-Lib with c++

  27 janvier 2020, par Baschdel

  I’m trying to record the whole desktop stream with FFmpeg on Windows.
  I found a working example here. The Problem is that some og the functions depricated. So I tried to replace them with the updated ones.

  But there are some slight problems. The error "has triggered a breakpoint." occurse and also "not able to read the location."
  The bigger problem is that I don’t know if this is the right way to do this..

  My code looks like this :

  using namespace std;
  
  
  
  /* initialize the resources*/
  
  Recorder::Recorder()
  
  {
  
  
  
      av_register_all();
  
      avcodec_register_all();
  
      avdevice_register_all();
  
      cout<<"\nall required functions are registered successfully";
  
  }
  
  
  
  /* uninitialize the resources */
  
  Recorder::~Recorder()
  
  {
  
  
  
      avformat_close_input(&pAVFormatContext);
  
      if( !pAVFormatContext )
  
      {
  
          cout<<"\nfile closed sucessfully";
  
      }
  
      else
  
      {
  
          cout<<"\nunable to close the file";
  
          exit(1);
  
      }
  
  
  
      avformat_free_context(pAVFormatContext);
  
      if( !pAVFormatContext )
  
      {
  
          cout<<"\navformat free successfully";
  
      }
  
      else
  
      {
  
          cout<<"\nunable to free avformat context";
  
          exit(1);
  
      }
  
  
  
  }
  
  
  
  /* establishing the connection between camera or screen through its respective folder */
  
  int Recorder::openCamera()
  
  {
  
  
  
      value = 0;
  
      options = NULL;
  
      pAVFormatContext = NULL;
  
  
  
      pAVFormatContext = avformat_alloc_context();//Allocate an AVFormatContext.
  
  
  
      openScreen(pAVFormatContext);
  
  
  
      /* set frame per second */
  
      value = av_dict_set( &options,"framerate","30",0 );
  
      if(value < 0)
  
      {
  
        cout<<"\nerror in setting dictionary value";
  
         exit(1);
  
      }
  
  
  
      value = av_dict_set( &options, "preset", "medium", 0 );
  
      if(value < 0)
  
      {
  
        cout<<"\nerror in setting preset values";
  
        exit(1);
  
      }
  
  
  
  //  value = avformat_find_stream_info(pAVFormatContext,NULL);
  
      if(value < 0)
  
      {
  
        cout<<"\nunable to find the stream information";
  
        exit(1);
  
      }
  
  
  
      VideoStreamIndx = -1;
  
  
  
      /* find the first video stream index . Also there is an API available to do the below operations */
  
      for(int i = 0; i < pAVFormatContext->nb_streams; i++ ) // find video stream posistion/index.
  
      {
  
        if( pAVFormatContext->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO )
  
        {
  
           VideoStreamIndx = i;
  
           break;
  
        }
  
  
  
      } 
  
  
  
      if( VideoStreamIndx == -1)
  
      {
  
        cout<<"\nunable to find the video stream index. (-1)";
  
        exit(1);
  
      }
  
  
  
      // assign pAVFormatContext to VideoStreamIndx
  
      pAVCodecContext = pAVFormatContext->streams[VideoStreamIndx]->codec;
  
  
  
      pAVCodec = avcodec_find_decoder(pAVCodecContext->codec_id);
  
      if( pAVCodec == NULL )
  
      {
  
        cout<<"\nunable to find the decoder";
  
        exit(1);
  
      }
  
  
  
      value = avcodec_open2(pAVCodecContext , pAVCodec , NULL);//Initialize the AVCodecContext to use the given AVCodec.
  
      if( value < 0 )
  
      {
  
        cout<<"\nunable to open the av codec";
  
        exit(1);
  
      }
  
  }
  
  
  
  /* initialize the video output file and its properties  */
  
  int Recorder::init_outputfile()
  
  {
  
      outAVFormatContext = NULL;
  
      value = 0;
  
      output_file = "output.mp4";
  
  
  
      avformat_alloc_output_context2(&outAVFormatContext, NULL, NULL, output_file);
  
      if (!outAVFormatContext)
  
      {
  
          cout<<"\nerror in allocating av format output context";
  
          exit(1);
  
      }
  
  
  
  /* Returns the output format in the list of registered output formats which best matches the provided parameters, or returns NULL if there is no match. */
  
      output_format = av_guess_format(NULL, output_file ,NULL);
  
      if( !output_format )
  
      {
  
       cout<<"\nerror in guessing the video format. try with correct format";
  
       exit(1);
  
      }
  
  
  
      video_st = avformat_new_stream(outAVFormatContext ,NULL);
  
      if( !video_st )
  
      {
  
          cout<<"\nerror in creating a av format new stream";
  
          exit(1);
  
      }
  
  
  
      if (codec_id == AV_CODEC_ID_H264)
  
      {
  
          av_opt_set(outAVCodecContext->priv_data, "preset", "slow", 0);
  
      }
  
  
  
      outAVCodec = avcodec_find_encoder(AV_CODEC_ID_MPEG4);
  
      if (!outAVCodec)
  
      {
  
          cout << "\nerror in finding the av codecs. try again with correct codec";
  
          exit(1);
  
      }
  
  
  
      outAVCodecContext = avcodec_alloc_context3(outAVCodec);
  
      if( !outAVCodecContext )
  
      {
  
          cout<<"\nerror in allocating the codec contexts";
  
          exit(1);
  
      }
  
  
  
      /* set property of the video file */
  
      outAVCodecContext = video_st->codec;
  
      outAVCodecContext->codec_id = AV_CODEC_ID_MPEG4;// AV_CODEC_ID_MPEG4; // AV_CODEC_ID_H264 // AV_CODEC_ID_MPEG1VIDEO
  
      outAVCodecContext->codec_type = AVMEDIA_TYPE_VIDEO;
  
      outAVCodecContext->pix_fmt  = AV_PIX_FMT_YUV420P;
  
      outAVCodecContext->bit_rate = 400000; // 2500000
  
      outAVCodecContext->width = 1920;
  
      outAVCodecContext->height = 1080;
  
      outAVCodecContext->gop_size = 3;
  
      outAVCodecContext->max_b_frames = 2;
  
      outAVCodecContext->time_base.num = 1;
  
      outAVCodecContext->time_base.den = 30; //15fps
  
  
  
  
  
      /* Some container formats (like MP4) require global headers to be present
  
         Mark the encoder so that it behaves accordingly. */
  
  
  
      if ( outAVFormatContext->oformat->flags & AVFMT_GLOBALHEADER)
  
      {
  
          outAVCodecContext->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
  
      }
  
  
  
      value = avcodec_open2(outAVCodecContext, outAVCodec, NULL);
  
      if( value < 0)
  
      {
  
          cout<<"\nerror in opening the avcodec";
  
          exit(1);
  
      }
  
  
  
      /* create empty video file */
  
      if ( !(outAVFormatContext->flags & AVFMT_NOFILE) )
  
      {
  
       if( avio_open2(&outAVFormatContext->pb , output_file , AVIO_FLAG_WRITE ,NULL, NULL) < 0 )
  
       {
  
        cout<<"\nerror in creating the video file";
  
        exit(1);
  
       }
  
      }
  
  
  
      if(!outAVFormatContext->nb_streams)
  
      {
  
          cout<<"\noutput file dose not contain any stream";
  
          exit(1);
  
      }
  
  
  
      /* imp: mp4 container or some advanced container file required header information*/
  
      value = avformat_write_header(outAVFormatContext , &options);
  
      if(value < 0)
  
      {
  
          cout<<"\nerror in writing the header context";
  
          exit(1);
  
      }
  
  
  
      /*
  
      // uncomment here to view the complete video file informations
  
      cout<<"\n\nOutput file information :\n\n";
  
      av_dump_format(outAVFormatContext , 0 ,output_file ,1);
  
      */
  
  }
  
  
  
  int Recorder::stop() {
  
      threading = false;
  
  
  
      demux->join();
  
      rescale->join();
  
      mux->join();
  
  
  
      return 0;
  
  }
  
  
  
  int Recorder::start() {
  
      initVideoThreads();
  
      return 0;
  
  }
  
  
  
  int Recorder::initVideoThreads() {
  
      demux = new thread(&Recorder::demuxVideoStream, this, pAVCodecContext, pAVFormatContext, VideoStreamIndx);
  
  
  
      rescale = new thread(&Recorder::rescaleVideoStream, this, pAVCodecContext, outAVCodecContext);
  
  
  
      demux = new thread(&Recorder::encodeVideoStream, this, outAVCodecContext);
  
      return 0;
  
  }
  
  
  
  void Recorder::demuxVideoStream(AVCodecContext* codecContext, AVFormatContext* formatContext, int streamIndex)
  
  {
  
      // init packet
  
      AVPacket* packet = (AVPacket*)av_malloc(sizeof(AVPacket));
  
      av_init_packet(packet);
  
  
  
      int ctr = 0;
  
  
  
      while (threading)
  
      {
  
          if (av_read_frame(formatContext, packet) < 0) {
  
              exit(1);
  
          }
  
  
  
          if (packet->stream_index == streamIndex)
  
          {
  
              int return_value; // = 0;
  
              ctr++;
  
  
  
              do
  
              {
  
                  return_value = avcodec_send_packet(codecContext, packet);
  
              } while (return_value == AVERROR(EAGAIN) && threading);
  
  
  
              //int i = avcodec_send_packet(codecContext, packet);
  
              if (return_value < 0 && threading) { // call Decoder
  
                  cout << "unable to decode video";
  
                  exit(1);
  
              }
  
          }
  
      }
  
  
  
      avcodec_send_packet(codecContext, NULL); // flush decoder
  
  
  
      // return 0;
  
  }
  
  
  
  void Recorder::rescaleVideoStream(AVCodecContext* inCodecContext, AVCodecContext* outCodecContext)
  
  {
  
      bool closing = false;
  
      AVFrame* inFrame = av_frame_alloc();
  
      if (!inFrame)
  
      {
  
          cout << "\nunable to release the avframe resources";
  
          exit(1);
  
      }
  
  
  
      int nbytes = av_image_get_buffer_size(outAVCodecContext->pix_fmt, outAVCodecContext->width, outAVCodecContext->height, 32);
  
      uint8_t* video_outbuf = (uint8_t*)av_malloc(nbytes);
  
      if (video_outbuf == NULL)
  
      {
  
          cout << "\nunable to allocate memory";
  
          exit(1);
  
      }
  
  
  
      AVFrame* outFrame = av_frame_alloc();//Allocate an AVFrame and set its fields to default values.
  
      if (!outFrame)
  
      {
  
          cout << "\nunable to release the avframe resources for outframe";
  
          exit(1);
  
      }
  
  
  
      // Setup the data pointers and linesizes based on the specified image parameters and the provided array.
  
      int value = av_image_fill_arrays(outFrame->data, outFrame->linesize, video_outbuf, AV_PIX_FMT_YUV420P, outAVCodecContext->width, outAVCodecContext->height, 1); // returns : the size in bytes required for src
  
      if (value < 0)
  
      {
  
          cout << "\nerror in filling image array";
  
      }
  
      int ctr = 0;
  
  
  
      while (threading || !closing) {
  
          int value = avcodec_receive_frame(inCodecContext, inFrame);
  
          if (value == 0) {
  
              ctr++;
  
              SwsContext* swsCtx_ = sws_getContext(inCodecContext->width,
  
                  inCodecContext->height,
  
                  inCodecContext->pix_fmt,
  
                  outAVCodecContext->width,
  
                  outAVCodecContext->height,
  
                  outAVCodecContext->pix_fmt,
  
                  SWS_BICUBIC, NULL, NULL, NULL);
  
              sws_scale(swsCtx_, inFrame->data, inFrame->linesize, 0, inCodecContext->height, outFrame->data, outFrame->linesize);
  
  
  
  
  
              int return_value;
  
              do
  
              {
  
                  return_value = avcodec_send_frame(outCodecContext, outFrame);
  
              } while (return_value == AVERROR(EAGAIN) && threading);
  
          }
  
          closing = (value == AVERROR_EOF);
  
      }
  
      avcodec_send_frame(outCodecContext, NULL);
  
  
  
  
  
      // av_free(video_outbuf);
  
  
  
      // return 0;
  
  }
  
  
  
  void Recorder::encodeVideoStream(AVCodecContext* codecContext)
  
  {
  
      bool closing = true;
  
      AVPacket* packet = (AVPacket*)av_malloc(sizeof(AVPacket));
  
      av_init_packet(packet);
  
  
  
      int ctr = 0;
  
  
  
      while (threading || !closing) {
  
          packet->data = NULL;    // packet data will be allocated by the encoder
  
          packet->size = 0;
  
          ctr++;
  
          int value = avcodec_receive_packet(codecContext, packet);
  
          if (value == 0) {
  
              if (packet->pts != AV_NOPTS_VALUE)
  
                  packet->pts = av_rescale_q(packet->pts, video_st->codec->time_base, video_st->time_base);
  
              if (packet->dts != AV_NOPTS_VALUE)
  
                  packet->dts = av_rescale_q(packet->dts, video_st->codec->time_base, video_st->time_base);
  
  
  
              //printf("Write frame %3d (size= %2d)\n", j++, packet->size / 1000);
  
              if (av_write_frame(outAVFormatContext, packet) != 0)
  
              {
  
                  cout << "\nerror in writing video frame";
  
              }
  
          }
  
  
  
          closing = (value == AVERROR_EOF);
  
      }
  
  
  
      value = av_write_trailer(outAVFormatContext);
  
      if (value < 0)
  
      {
  
          cout << "\nerror in writing av trailer";
  
          exit(1);
  
      }
  
  
  
      // av_free(packet);
  
  
  
      // return 0;
  
  }
  
  
  
  
  
  int Recorder::openScreen(AVFormatContext* pFormatCtx) {
  
      /*
  
  
  
      X11 video input device.
  
      To enable this input device during configuration you need libxcb installed on your system. It will be automatically detected during configuration.
  
      This device allows one to capture a region of an X11 display.
  
      refer : https://www.ffmpeg.org/ffmpeg-devices.html#x11grab
  
      */
  
      /* current below is for screen recording. to connect with camera use v4l2 as a input parameter for av_find_input_format */
  
      pAVInputFormat = av_find_input_format("gdigrab");
  
      //value = avformat_open_input(&pAVFormatContext, ":0.0+10,250", pAVInputFormat, NULL);
  
  
  
      value = avformat_open_input(&pAVFormatContext, "desktop", pAVInputFormat, NULL);
  
      if (value != 0)
  
      {
  
          cout << "\nerror in opening input device";
  
          exit(1);
  
      }
  
      return 0;
  
  }