Recherche avancée

Sur d’autres sites (2833)

• LibVLC DXVA2 Windows C++ Hardware Acceleration

  15 novembre 2016, par Sergiu Vlad

  I want to decode a h264 RTSP stream using DXVA2 Accelerated decoder. In VLC it does work but in my code have the following output :

  avcodec generic warning: threaded frame decoding is not compatible with DXVA2, disabled
  
  avcodec generic debug: available hardware decoder output format 61 (dxva2_vld)
  
  avcodec generic debug: available hardware decoder output format 128 (d3d11va_vld)
  
  avcodec generic debug: available software decoder output format 12 (yuvj420p)
  
  core generic debug: looking for hw decoder module matching "any": 1 candidates
  
  core generic debug: no hw decoder modules matched

  Is this a problem with NV12 conversion ? Is there a way to force DXVA2 ?

  Thanks

  CODE :

      #include 
  
  #include 
  
  #include 
  
  #include 
  
  #include 
  
  #include <iostream>
  
  
  
  #include 
  
  #include <ctime>
  
  #include <vlc></vlc>vlc.h>
  
  
  
  // RTSP address
  
  const char* rtspAddress = "rtsp://admin:admin@192.168.0.245/live_st1";
  
  // Video resolution WxH
  
  cv::Size rtspRes(1920, 1080);
  
  
  
  struct VideoDataStruct
  
  {
  
      int param;
  
  };
  
  
  
  int done = 0;
  
  libvlc_media_player_t *mp;
  
  unsigned int videoBufferSize = 0;
  
  uint8_t *videoBuffer = 0;
  
  
  
  void cbVideoPrerender(void *p_video_data, uint8_t **pp_pixel_buffer, int size)
  
  {
  
      if (size > videoBufferSize || !videoBuffer)
  
      {
  
          printf("Reallocate raw video buffer %d bytes\n", size);
  
          free(videoBuffer);
  
          videoBuffer = (uint8_t *)malloc(size);
  
          videoBufferSize = size;
  
      }
  
  
  
      // videoBuffer = (uint8_t *)malloc(size);
  
      *pp_pixel_buffer = videoBuffer;
  
  }
  
  void cbVideoPostrender(void *p_video_data, uint8_t *p_pixel_buffer, int width, int height, int pixel_pitch, int size, int64_t pts)
  
  {
  
      // Unlocking
  
      //CloseHandle(hMutex);
  
  
  
  }
  
  
  
  //static void handleEvent(const libvlc_event_t* pEvt, void* pUserData)
  
  //{
  
  //    libvlc_time_t time;
  
  //    switch (pEvt->type)
  
  //    {
  
  //    case libvlc_MediaPlayerTimeChanged:
  
  //        time = libvlc_media_player_get_time(mp);
  
  //        printf("MediaPlayerTimeChanged %lld ms\n", (long long)time);
  
  //        break;
  
  //    case libvlc_MediaPlayerEndReached:
  
  //        printf("MediaPlayerEndReached\n");
  
  //        done = 1;
  
  //        break;
  
  //    default:
  
  //        printf("%s\n", libvlc_event_type_name(pEvt->type));
  
  //    }
  
  //}
  
  
  
  clock_t startTime[8];
  
  int framesReceivedThisSecond[8];
  
  
  
  int main()
  
  {
  
  
  
  
  
      // VLC pointers
  
      libvlc_instance_t *inst;
  
      libvlc_media_t *m;
  
      void *pUserData = 0;
  
  
  
      VideoDataStruct dataStruct;
  
  
  
      // VLC options
  
      char smem_options[1000];
  
  
  
      // RV24
  
      sprintf(smem_options
  
              , "#transcode{vcodec=RV24}:smem{"
  
                "video-prerender-callback=%lld,"
  
                "video-postrender-callback=%lld,"
  
                "video-data=%lld,"
  
                "no-time-sync},"
  
              , (long long int)(intptr_t)(void*)&amp;cbVideoPrerender
  
              , (long long int)(intptr_t)(void*)&amp;cbVideoPostrender
  
              , (long long int)(intptr_t)(void*)&amp;dataStruct
  
              );
  
  
  
      const char * const vlc_args[] = {
  
          "-I", "dummy",            // Don't use any interface
  
          "--ignore-config",        // Don't use VLC's config
  
          "--extraintf=logger",     // Log anything
  
          "--verbose=4",            // Be verbose
  
          //        "--ffmpeg-hw",
  
          //        "-vvv",
  
          "--ffmpeg-hw",
  
          "--avcodec-hw=any",
  
          "--sout", smem_options    // Stream to memory
  
  
  
      };
  
  
  
      // Launch VLC
  
      inst = libvlc_new(sizeof(vlc_args) / sizeof(vlc_args[0]), vlc_args);
  
  
  
      // Create a new item
  
      m = libvlc_media_new_location(inst, rtspAddress);
  
  
  
  
  
  
  
      // Create a media player playing environement
  
      mp = libvlc_media_player_new_from_media(m);
  
  
  
      //    libvlc_event_manager_t* eventManager = libvlc_media_player_event_manager(mp);
  
      //    libvlc_event_attach(eventManager, libvlc_MediaPlayerTimeChanged, handleEvent, pUserData);
  
      //    libvlc_event_attach(eventManager, libvlc_MediaPlayerEndReached, handleEvent, pUserData);
  
      //    libvlc_event_attach(eventManager, libvlc_MediaPlayerPositionChanged, handleEvent, pUserData);
  
  
  
      //libvlc_video_set_format(mp, "RV24", 240, 320, 240 * 3 );
  
  
  
      // play the media_player
  
      libvlc_media_player_play(mp);
  
  
  
  
  
      // Create a window for displaying the video
  
      std::string winName("Demo Video");
  
      cv::namedWindow(winName, cv::WINDOW_AUTOSIZE);
  
  
  
      cv::Mat frame;
  
      int key = 0;
  
  
  
      // Endless loop, press Esc to quit
  
      while (key != 27)
  
      {
  
  
  
          // Check for invalid input
  
          if (videoBuffer)
  
          {
  
              // CV_8UC3 = 8 bits, 3 chanels
  
              frame = cv::Mat(rtspRes, CV_8UC3, videoBuffer);
  
          }
  
  
  
  
  
          if (frame.rows == 0 || frame.cols == 0)
  
              continue;
  
  
  
          int stream_number=0;
  
          framesReceivedThisSecond[stream_number]++;
  
          long    duration = ( std::clock() - startTime[stream_number] );
  
          if(duration>1000)
  
          {
  
              std::cout&lt;&lt;"\n"&lt;&lt;"FPS "&lt;code></ctime></iostream>

  FULL LOG :

  core libvlc debug: Copyright © 1996-2016 the VideoLAN team
  
  core libvlc debug: revision 2.2.3-37-g888b7e89
  
  core libvlc debug: configured with ../extras/package/win32/../../../configure  '--enable-update-check' '--enable-lua' '--enable-faad' '--enable-flac' '--enable-theora' '--enable-twolame' '--enable-quicktime' '--enable-avcodec' '--enable-merge-ffmpeg' '--enable-dca' '--enable-mpc' '--enable-libass' '--enable-x264' '--enable-schroedinger' '--enable-realrtsp' '--enable-live555' '--enable-dvdread' '--enable-shout' '--enable-goom' '--enable-caca' '--disable-sdl' '--enable-qt' '--enable-skins2' '--enable-sse' '--enable-mmx' '--enable-libcddb' '--enable-zvbi' '--disable-telx' '--enable-nls' '--host=i686-w64-mingw32' '--build=x86_64-pc-linux-gnu' 'build_alias=x86_64-pc-linux-gnu' 'host_alias=i686-w64-mingw32' 'PKG_CONFIG_PATH=/opt/intel/opencl:' 'PKG_CONFIG_LIBDIR=/home/svlad/DEV/vlc/contrib/i686-w64-mingw32/lib/pkgconfig'
  
  core libvlc debug: using multimedia timers as clock source
  
  core libvlc debug:  min period: 1 ms, max period: 1000000 ms
  
  core libvlc debug: searching plug-in modules
  
  core libvlc debug: loading plugins cache file C:\DEV\VLCStreamProcessor\build-untitled-Desktop_Qt_5_5_1_MSVC2013_32bit-Debug\debug\plugins\plugins.dat
  
  core libvlc debug: recursively browsing `C:\DEV\VLCStreamProcessor\build-untitled-Desktop_Qt_5_5_1_MSVC2013_32bit-Debug\debug\plugins'
  
  core libvlc debug: saving plugins cache C:\DEV\VLCStreamProcessor\build-untitled-Desktop_Qt_5_5_1_MSVC2013_32bit-Debug\debug\plugins\plugins.dat
  
  core libvlc debug: plug-ins loaded: 418 modules
  
  core libvlc debug: translation test: code is "C"
  
  core libvlc debug: CPU has capabilities MMX MMXEXT SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 FPU 
  
  core playlist debug: creating audio output
  
  core audio output debug: looking for audio output module matching "any": 6 candidates
  
  directsound audio output debug: found 2 devices
  
  core audio output debug: using audio output module "directsound"
  
  core playlist debug: keeping audio output
  
  core interface debug: looking for interface module matching "logger,none": 16 candidates
  
  logger interface: VLC media player - 2.2.4 Weatherwax
  
  logger interface: Copyright © 1996-2016 the VideoLAN team
  
  logger interface: 
  
  Warning: if you cannot access the GUI anymore, open a command-line window, go to the directory where you installed VLC and run "vlc -I qt"
  
  logger interface: using logger.
  
  logger interface warning: no log filename provided, using `vlc-log.txt'
  
  logger interface debug: opening logfile `vlc-log.txt'
  
  core libvlc debug: VLC media player - 2.2.4 Weatherwax
  
  core libvlc debug: Copyright © 1996-2016 the VideoLAN team
  
  core libvlc debug: revision 2.2.3-37-g888b7e89
  
  core libvlc debug: configured with ../extras/package/win32/../../../configure  '--enable-update-check' '--enable-lua' '--enable-faad' '--enable-flac' '--enable-theora' '--enable-twolame' '--enable-quicktime' '--enable-avcodec' '--enable-merge-ffmpeg' '--enable-dca' '--enable-mpc' '--enable-libass' '--enable-x264' '--enable-schroedinger' '--enable-realrtsp' '--enable-live555' '--enable-dvdread' '--enable-shout' '--enable-goom' '--enable-caca' '--disable-sdl' '--enable-qt' '--enable-skins2' '--enable-sse' '--enable-mmx' '--enable-libcddb' '--enable-zvbi' '--disable-telx' '--enable-nls' '--host=i686-w64-mingw32' '--build=x86_64-pc-linux-gnu' 'build_alias=x86_64-pc-linux-gnu' 'host_alias=i686-w64-mingw32' 'PKG_CONFIG_PATH=/opt/intel/opencl:' 'PKG_CONFIG_LIBDIR=/home/svlad/DEV/vlc/contrib/i686-w64-mingw32/lib/pkgconfig'
  
  core interface debug: using interface module "logger"
  
  core generic debug: creating audio output
  
  core audio output debug: looking for audio output module matching "any": 6 candidates
  
  directsound audio output debug: found 2 devices
  
  core audio output debug: using audio output module "directsound"
  
  core generic debug: keeping audio output
  
  core input debug: Creating an input for 'rtsp://192.168.0.245/live_st1'
  
  core stream output debug: using sout chain=`transcode{vcodec=RV24}:smem{video-prerender-callback=15667245,video-postrender-callback=15667630,video-data=3471900,no-time-sync},'
  
  core stream output debug: stream=`smem'
  
  core stream out debug: looking for sout stream module matching "smem": 22 candidates
  
  core stream out debug: set config option: sout-smem-video-prerender-callback to 15667245
  
  core stream out debug: set config option: sout-smem-video-postrender-callback to 15667630
  
  core stream out debug: set config option: sout-smem-video-data to 3471900
  
  core stream out debug: set config option: sout-smem-time-sync to (null)
  
  core stream out debug: using sout stream module "stream_out_smem"
  
  core stream output debug: stream=`transcode'
  
  core stream out debug: looking for sout stream module matching "transcode": 22 candidates
  
  core stream out debug: set config option: sout-transcode-vcodec to RV24
  
  stream_out_transcode stream out debug: Checking video codec mapping for RV24 got RV24 
  
  stream_out_transcode stream out debug: codec video=RV24 0x0 scaling: 0.000000 0kb/s
  
  core stream out debug: using sout stream module "stream_out_transcode"
  
  core input debug: using timeshift granularity of 50 MiB, in path 'C:\Users\BCDVideo\AppData\Local\Temp'
  
  core input debug: `rtsp://admin:admin@192.168.0.245/live_st1' gives access `rtsp' demux `' path `admin:admin@192.168.0.245/live_st1'
  
  core input debug: specified demux `any'
  
  core input debug: creating demux: access='rtsp' demux='any' location='admin:admin@192.168.0.245/live_st1' file='\\admin:admin@192.168.0.245\live_st1'
  
  core demux debug: looking for access_demux module matching "rtsp": 12 candidates
  
  live555 demux debug: version 2016.01.12
  
  live555 demux debug: RTP subsession 'video/H264'
  
  core input debug: selecting program id=0
  
  live555 demux debug: setup start: 0.000000 stop:0.000000
  
  live555 demux debug: We have a timeout of 60 seconds
  
  live555 demux debug: spawned timeout thread
  
  live555 demux debug: play start: 0.000000 stop:0.000000
  
  core demux debug: using access_demux module "live555"
  
  core decoder debug: looking for packetizer module matching "any": 23 candidates
  
  packetizer_h264 decoder debug: found NAL_SPS (sps_id=0)
  
  packetizer_h264 decoder debug: found NAL_PPS (pps_id=0 sps_id=0)
  
  core decoder debug: using packetizer module "packetizer_h264"
  
  core input debug: starting in sync mode
  
  core demux meta debug: looking for meta reader module matching "any": 2 candidates
  
  lua demux meta debug: Trying Lua scripts in C:\Users\BCDVideo\AppData\Roaming\vlc\lua\meta\reader
  
  lua demux meta debug: Trying Lua scripts in C:\DEV\VLCStreamProcessor\build-untitled-Desktop_Qt_5_5_1_MSVC2013_32bit-Debug\debug\lua\meta\reader
  
  core demux meta debug: no meta reader modules matched
  
  core input debug: `rtsp://admin:admin@192.168.0.245/live_st1' successfully opened
  
  core input debug: switching to async mode
  
  core input debug: Buffering 0%
  
  core input debug: Buffering 3%
  
  core input debug: Buffering 10%
  
  core stream output debug: adding a new sout input (sout_input:00c21c50)
  
  core input debug: Buffering 13%
  
  stream_out_transcode stream out debug: creating video transcoding from fcc=`h264' to fcc=`RV24'
  
  core input debug: Buffering 16%
  
  core generic debug: looking for decoder module matching "any": 43 candidates
  
  core input debug: Buffering 20%
  
  core input debug: Buffering 23%
  
  avcodec generic debug: CPU flags: 0x0107d3db
  
  core input debug: Buffering 30%
  
  avcodec generic debug: trying to use direct rendering
  
  core input debug: Buffering 33%
  
  avcodec generic debug: allowing 4 thread(s) for decoding
  
  core input debug: Buffering 36%
  
  avcodec generic warning: threaded frame decoding is not compatible with DXVA2, disabled
  
  core input debug: Buffering 40%
  
  avcodec generic debug: avcodec codec (H264 - MPEG-4 AVC (part 10)) started
  
  core input debug: Buffering 43%
  
  avcodec generic debug: using slice thread mode with 4 threads
  
  core input debug: Buffering 50%
  
  core generic debug: using decoder module "avcodec"
  
  core input debug: Buffering 53%
  
  core encoder debug: looking for encoder module matching "any": 20 candidates
  
  core input debug: Buffering 56%
  
  core input debug: Buffering 60%
  
  avcodec encoder debug: CPU flags: 0x0107d3db
  
  core input debug: Buffering 63%
  
  avcodec encoder debug: Time base for probing setted to 1/1000000
  
  core input debug: Buffering 70%
  
  avcodec encoder debug: Time base set to 1/1000000
  
  core input debug: Buffering 73%
  
  avcodec encoder debug: found encoder Raw video
  
  core input debug: Buffering 76%
  
  core encoder debug: using encoder module "avcodec"
  
  core input debug: Buffering 80%
  
  core encoder debug: removing module "avcodec"
  
  core input debug: Buffering 83%
  
  core input debug: Buffering 90%
  
  core input debug: Buffering 93%
  
  core input debug: Buffering 96%
  
  core input debug: Stream buffering done (1002 ms in 834 ms)
  
  core input debug: Decoder wait done in 0 ms
  
  avcodec generic debug: available hardware decoder output format 61 (dxva2_vld)
  
  avcodec generic debug: available hardware decoder output format 128 (d3d11va_vld)
  
  avcodec generic debug: available software decoder output format 12 (yuvj420p)
  
  core generic debug: looking for hw decoder module matching "any": 1 candidates
  
  core generic debug: no hw decoder modules matched
  
  avcodec generic warning: plane 0 not aligned
  
  avcodec generic warning: disabling direct rendering
  
  stream_out_transcode stream out debug: decoder aspect is 1.761468:1
  
  stream_out_transcode stream out debug: source pixel aspect is 0.990826:1
  
  stream_out_transcode stream out debug: scaled pixel aspect is 0.990826:1
  
  stream_out_transcode stream out debug: source 1920x1080, destination 1920x1080
  
  stream_out_transcode stream out debug: source fps 30/1, destination 30/1
  
  stream_out_transcode stream out: input interval 33333 (base 1)
  
  stream_out_transcode stream out: output interval 33333 (base 1)
  
  stream_out_transcode stream out debug: encoder aspect is 1920:1090
  
  core filter debug: looking for video filter2 module matching "any": 58 candidates
  
  swscale filter debug: 1920x1080 (1920x1090) chroma: J420 -> 1920x1080 (1920x1090) chroma: RV24 with scaling using Bicubic (good quality)
  
  core filter debug: using video filter2 module "swscale"
  
  core stream out debug: Filter 'Swscale' (0089a2cc) appended to chain
  
  stream_out_transcode stream out debug: destination (after video filters) 1920x1090
  
  core encoder debug: looking for encoder module matching "any": 20 candidates
  
  avcodec encoder debug: CPU flags: 0x0107d3db
  
  avcodec encoder debug: Time base for probing setted to 1/30
  
  avcodec encoder debug: Time base set to 1/30
  
  avcodec encoder debug: found encoder Raw video
  
  core encoder debug: using encoder module "avcodec"
  
  stream_out_transcode stream out warning: Reseting video sync
  
  stream_out_transcode stream out warning: Reseting video sync
  
  stream_out_transcode stream out warning: Reseting video sync
  
  stream_out_transcode stream out warning: Reseting video sync
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518755, last: 1518769)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518756, last: 1518769)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518757, last: 1518769)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518758, last: 1518769)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518759, last: 1518769)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518760, last: 1518769)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518761, last: 1518769)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518762, last: 1518769)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518763, last: 1518769)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518764, last: 1518769)
  
  stream_out_transcode stream out warning: Reseting video sync
  
  stream_out_transcode stream out warning: Reseting video sync
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518784, last: 1518795)
  
  live555 demux debug: tk->rtpSource->hasBeenSynchronizedUsingRTCP()
  
  core input error: ES_OUT_RESET_PCR called
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518785, last: 1518795)
  
  core input debug: Buffering 0%
  
  core input debug: Buffering 3%
  
  core input debug: Buffering 6%
  
  core input debug: Buffering 10%
  
  core input debug: Buffering 13%
  
  core input debug: Buffering 20%
  
  core input debug: Buffering 23%
  
  core input debug: Buffering 26%
  
  core input debug: Buffering 30%
  
  core input debug: Buffering 33%
  
  core input debug: Buffering 40%
  
  core input debug: Buffering 43%
  
  core input debug: Buffering 46%
  
  core input debug: Buffering 50%
  
  core input debug: Buffering 53%
  
  core input debug: Buffering 60%
  
  core input debug: Buffering 63%
  
  core input debug: Buffering 66%
  
  core input debug: Buffering 70%
  
  core input debug: Buffering 73%
  
  core input debug: Buffering 80%
  
  core input debug: Buffering 83%
  
  core input debug: Buffering 86%
  
  core input debug: Buffering 90%
  
  core input debug: Buffering 93%
  
  core input debug: Stream buffering done (1002 ms in 817 ms)
  
  core input debug: Decoder wait done in 0 ms
  
  stream_out_transcode stream out warning: Reseting video sync
  
  stream_out_transcode stream out warning: Reseting video sync
  
  stream_out_transcode stream out warning: Reseting video sync
  
  stream_out_transcode stream out warning: Reseting video sync
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518834, last: 1518837)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518835, last: 1518837)
  
  avcodec encoder warning: almost fed libavcodec with a frame in the past (current: 1518836, last: 1518837)
  
  avcodec encoder warning: almost fed libavcodec with two frames with the same PTS (1518837)
  
  stream_out_transcode stream out warning: Reseting video sync
  
  stream_out_transcode stream out warning: Reseting video sync

• ffmpeg - convert image sequence to video with reversed order

  8 avril 2017, par 0__

  Looking at the docs, it is not apparent to me whether ffmpeg would allow me to convert an image sequence to a video in reverse order, for example using this sequence :

  frame-1000.jpg
  
  frame-999.jpg
  
  frame-998.jpg
  
  ...
  
  frame-1.jpg

  Is it possible to give a "step direction" for the frame indices ?

• Libav (ffmpeg) copying decoded video timestamps to encoder

  31 octobre 2016, par Jason C

  I am writing an application that decodes a single video stream from an input file (any codec, any container), does a bunch of image processing, and encodes the results to an output file (single video stream, Quicktime RLE, MOV). I am using ffmpeg’s libav 3.1.5 (Windows build for now, but the application will be cross-platform).

  There is a 1:1 correspondence between input and output frames and I want the frame timing in the output to be identical to the input. I am having a really, really hard time accomplishing this. So my general question is : How do I reliably (as in, in all cases of inputs) set the output frame timing identical to the input ?

  It took me a very long time to slog through the API and get to the point I am at now. I put together a minimal test program to work with :

  #include <cstdio>
  
  
  
  extern "C" {
  
  #include <libavcodec></libavcodec>avcodec.h>
  
  #include <libavformat></libavformat>avformat.h>
  
  #include <libavutil></libavutil>avutil.h>
  
  #include <libavutil></libavutil>imgutils.h>
  
  #include <libswscale></libswscale>swscale.h>
  
  }
  
  
  
  using namespace std;
  
  
  
  
  
  struct DecoderStuff {
  
      AVFormatContext *formatx;
  
      int nstream;
  
      AVCodec *codec;
  
      AVStream *stream;
  
      AVCodecContext *codecx;
  
      AVFrame *rawframe;
  
      AVFrame *rgbframe;
  
      SwsContext *swsx;
  
  };
  
  
  
  
  
  struct EncoderStuff {
  
      AVFormatContext *formatx;
  
      AVCodec *codec;
  
      AVStream *stream;
  
      AVCodecContext *codecx;
  
  };
  
  
  
  
  
  template <typename t="t">
  
  static void dump_timebase (const char *what, const T *o) {
  
      if (o)
  
          printf("%s timebase: %d/%d\n", what, o->time_base.num, o->time_base.den);
  
      else
  
          printf("%s timebase: null object\n", what);
  
  }
  
  
  
  
  
  // reads next frame into d.rawframe and d.rgbframe. returns false on error/eof.
  
  static bool read_frame (DecoderStuff &amp;d) {
  
  
  
      AVPacket packet;
  
      int err = 0, haveframe = 0;
  
  
  
      // read
  
      while (!haveframe &amp;&amp; err >= 0 &amp;&amp; ((err = av_read_frame(d.formatx, &amp;packet)) >= 0)) {
  
         if (packet.stream_index == d.nstream) {
  
             err = avcodec_decode_video2(d.codecx, d.rawframe, &amp;haveframe, &amp;packet);
  
         }
  
         av_packet_unref(&amp;packet);
  
      }
  
  
  
      // error output
  
      if (!haveframe &amp;&amp; err != AVERROR_EOF) {
  
          char buf[500];
  
          av_strerror(err, buf, sizeof(buf) - 1);
  
          buf[499] = 0;
  
          printf("read_frame: %s\n", buf);
  
      }
  
  
  
      // convert to rgb
  
      if (haveframe) {
  
          sws_scale(d.swsx, d.rawframe->data, d.rawframe->linesize, 0, d.rawframe->height,
  
                    d.rgbframe->data, d.rgbframe->linesize);
  
      }
  
  
  
      return haveframe;
  
  
  
  }
  
  
  
  
  
  // writes an output frame, returns false on error.
  
  static bool write_frame (EncoderStuff &amp;e, AVFrame *inframe) {
  
  
  
      // see note in so post about outframe here
  
      AVFrame *outframe = av_frame_alloc();
  
      outframe->format = inframe->format;
  
      outframe->width = inframe->width;
  
      outframe->height = inframe->height;
  
      av_image_alloc(outframe->data, outframe->linesize, outframe->width, outframe->height,
  
                     AV_PIX_FMT_RGB24, 1);
  
      //av_frame_copy(outframe, inframe);
  
      static int count = 0;
  
      for (int n = 0; n &lt; outframe->width * outframe->height; ++ n) {
  
          outframe->data[0][n*3+0] = ((n+count) % 100) ? 0 : 255;
  
          outframe->data[0][n*3+1] = ((n+count) % 100) ? 0 : 255;
  
          outframe->data[0][n*3+2] = ((n+count) % 100) ? 0 : 255;
  
      }
  
      ++ count;
  
  
  
      AVPacket packet;
  
      av_init_packet(&amp;packet);
  
      packet.size = 0;
  
      packet.data = NULL;
  
  
  
      int err, havepacket = 0;
  
      if ((err = avcodec_encode_video2(e.codecx, &amp;packet, outframe, &amp;havepacket)) >= 0 &amp;&amp; havepacket) {
  
          packet.stream_index = e.stream->index;
  
          err = av_interleaved_write_frame(e.formatx, &amp;packet);
  
      }
  
  
  
      if (err &lt; 0) {
  
          char buf[500];
  
          av_strerror(err, buf, sizeof(buf) - 1);
  
          buf[499] = 0;
  
          printf("write_frame: %s\n", buf);
  
      }
  
  
  
      av_packet_unref(&amp;packet);
  
      av_freep(&amp;outframe->data[0]);
  
      av_frame_free(&amp;outframe);
  
  
  
      return err >= 0;
  
  
  
  }
  
  
  
  
  
  int main (int argc, char *argv[]) {
  
  
  
      const char *infile = "wildlife.wmv";
  
      const char *outfile = "test.mov";
  
      DecoderStuff d = {};
  
      EncoderStuff e = {};
  
  
  
      av_register_all();
  
  
  
      // decoder
  
      avformat_open_input(&amp;d.formatx, infile, NULL, NULL);
  
      avformat_find_stream_info(d.formatx, NULL);
  
      d.nstream = av_find_best_stream(d.formatx, AVMEDIA_TYPE_VIDEO, -1, -1, &amp;d.codec, 0);
  
      d.stream = d.formatx->streams[d.nstream];
  
      d.codecx = avcodec_alloc_context3(d.codec);
  
      avcodec_parameters_to_context(d.codecx, d.stream->codecpar);
  
      avcodec_open2(d.codecx, NULL, NULL);
  
      d.rawframe = av_frame_alloc();
  
      d.rgbframe = av_frame_alloc();
  
      d.rgbframe->format = AV_PIX_FMT_RGB24;
  
      d.rgbframe->width = d.codecx->width;
  
      d.rgbframe->height = d.codecx->height;
  
      av_frame_get_buffer(d.rgbframe, 1);
  
      d.swsx = sws_getContext(d.codecx->width, d.codecx->height, d.codecx->pix_fmt,
  
                              d.codecx->width, d.codecx->height, AV_PIX_FMT_RGB24,
  
                              SWS_POINT, NULL, NULL, NULL);
  
      //av_dump_format(d.formatx, 0, infile, 0);
  
      dump_timebase("in stream", d.stream);
  
      dump_timebase("in stream:codec", d.stream->codec); // note: deprecated
  
      dump_timebase("in codec", d.codecx);
  
  
  
      // encoder
  
      avformat_alloc_output_context2(&amp;e.formatx, NULL, NULL, outfile);
  
      e.codec = avcodec_find_encoder(AV_CODEC_ID_QTRLE);
  
      e.stream = avformat_new_stream(e.formatx, e.codec);
  
      e.codecx = avcodec_alloc_context3(e.codec);
  
      e.codecx->bit_rate = 4000000; // arbitrary for qtrle
  
      e.codecx->width = d.codecx->width;
  
      e.codecx->height = d.codecx->height;
  
      e.codecx->gop_size = 30; // 99% sure this is arbitrary for qtrle
  
      e.codecx->pix_fmt = AV_PIX_FMT_RGB24;
  
      e.codecx->time_base = d.stream->time_base; // ???
  
      e.codecx->flags |= (e.formatx->flags &amp; AVFMT_GLOBALHEADER) ? AV_CODEC_FLAG_GLOBAL_HEADER : 0;
  
      avcodec_open2(e.codecx, NULL, NULL);
  
      avcodec_parameters_from_context(e.stream->codecpar, e.codecx); 
  
      //av_dump_format(e.formatx, 0, outfile, 1);
  
      dump_timebase("out stream", e.stream);
  
      dump_timebase("out stream:codec", e.stream->codec); // note: deprecated
  
      dump_timebase("out codec", e.codecx);
  
  
  
      // open file and write header
  
      avio_open(&amp;e.formatx->pb, outfile, AVIO_FLAG_WRITE); 
  
      avformat_write_header(e.formatx, NULL);
  
  
  
      // frames
  
      while (read_frame(d) &amp;&amp; write_frame(e, d.rgbframe))
  
          ;
  
  
  
      // write trailer and close file
  
      av_write_trailer(e.formatx);
  
      avio_closep(&amp;e.formatx->pb); 
  
  
  
  }
  
  </typename></cstdio>

  A few notes about that :

  • Since all of my attempts at frame timing so far have failed, I’ve removed almost all timing-related stuff from this code to start with a clean slate.
  • Almost all error checking and cleanup omitted for brevity.
  • The reason I allocate a new output frame with a new buffer in write_frame, rather than using inframe directly, is because this is more representative of what my real application is doing. My real app also uses RGB24 internally, hence the conversions here.
  • The reason I generate a weird pattern in outframe, rather than using e.g. av_copy_frame, is because I just wanted a test pattern that compressed well with Quicktime RLE (my test input ends up generating a 1.7GB output file otherwise).
  • The input video I am using, "wildlife.wmv", can be found here. I’ve hard-coded the filenames.
  • I am aware that avcodec_decode_video2 and avcodec_encode_video2 are deprecated, but don’t care. They work fine, I’ve already struggled too much getting my head around the latest version of the API, ffmpeg changes their API with nearly every release, and I really don’t feel like dealing with avcodec_send_* and avcodec_receive_* right now.
  • I think I’m supposed to be finishing off by passing a NULL frame to avcodec_encode_video2 to flush some buffers or something but I’m a bit confused about that. Unless somebody feels like explaining that let’s ignore it for now, it’s a separate question. The docs are as vague about this point as they are about everything else.
  • My test input file’s frame rate is 29.97.

  ---

  Now, as for my current attempts. The following timing related fields are present in the above code, with details/confusion in bold. There’s a lot of them, because the API is mind-bogglingly convoluted :

  • main: d.stream->time_base : Input video stream time base. For my test input file this is 1/1000.
  • main: d.stream->codec->time_base : Not sure what this is (I never could make sense of why AVStream has an AVCodecContext field when you always use your own new context anyways) and also the codec field is deprecated. For my test input file this is 1/1000.
  • main: d.codecx->time_base : Input codec context time-base. For my test input file this is 0/1. Am I supposed to set it ?
  • main: e.stream->time_base : Time base of the output stream I create. What do I set this to ?
  • main: e.stream->codec->time_base : Time base of the deprecated and mysterious codec field of the output stream I create. Do I set this to anything ?
  • main: e.codecx->time_base : Time base of the encoder context I create. What do I set this to ?
  • read_frame: packet.dts : Decoding timestamp of packet read.
  • read_frame: packet.pts : Presentation timestamp of packet read.
  • read_frame: packet.duration : Duration of packet read.
  • read_frame: d.rawframe->pts : Presentation timestamp of raw frame decoded. This is always 0. Why isn’t it read by the decoder...?
  • read_frame: d.rgbframe->pts / write_frame: inframe->pts : Presentation timestamp of decoded frame converted to RGB. Not set to anything currently.
  • read_frame: d.rawframe->pkt_* : Fields copied from packet, discovered after reading this post. They are set correctly but I don’t know if they are useful.
  • write_frame: outframe->pts : Presentation timestamp of frame being encoded. Should I set this to something ?
  • write_frame: outframe->pkt_* : Timing fields from a packet. Should I set these ? They seem to be ignored by the encoder.
  • write_frame: packet.dts : Decoding timestamp of packet being encoded. What do I set it to ?
  • write_frame: packet.pts : Presentation timestamp of packet being encoded. What do I set it to ?
  • write_frame: packet.duration : Duration of packet being encoded. What do I set it to ?

  I have tried the following, with the described results. Note that inframe is d.rgbframe :

  1.  
     • Init e.stream->time_base = d.stream->time_base
     • Init e.codecx->time_base = d.codecx->time_base
     • Set d.rgbframe->pts = packet.dts in read_frame
     • Set outframe->pts = inframe->pts in write_frame
     • Result : Warning that encoder time base is not set (since d.codecx->time_base was 0/1), seg fault.
  2.  
     • Init e.stream->time_base = d.stream->time_base
     • Init e.codecx->time_base = d.stream->time_base
     • Set d.rgbframe->pts = packet.dts in read_frame
     • Set outframe->pts = inframe->pts in write_frame
     • Result : No warnings, but VLC reports frame rate as 480.048 (no idea where this number came from) and file plays too fast. Also the encoder sets all the timing fields in packet to 0, which was not what I expected. (Edit : Turns out this is because av_interleaved_write_frame, unlike av_write_frame, takes ownership of the packet and swaps it with a blank one, and I was printing the values after that call. So they are not ignored.)
  3.  
     • Init e.stream->time_base = d.stream->time_base
     • Init e.codecx->time_base = d.stream->time_base
     • Set d.rgbframe->pts = packet.dts in read_frame
     • Set any of pts/dts/duration in packet in write_frame to anything.
     • Result : Warnings about packet timestamps not set. Encoder seems to reset all packet timing fields to 0, so none of this has any effect.
  4.  
     • Init e.stream->time_base = d.stream->time_base
     • Init e.codecx->time_base = d.stream->time_base
     • I found these fields, pkt_pts, pkt_dts, and pkt_duration in AVFrame after reading this post, so I tried copying those all the way through to outframe.
     • Result : Really had my hopes up, but ended up with same results as attempt 3 (packet timestamp not set warning, incorrect results).

  I tried various other hand-wavey permutations of the above and nothing worked. What I want to do is create an output file that plays back with the same timing and frame rate as the input (29.97 constant frame rate in this case).

  So how do I do this ? Of the zillions of timing related fields here, what do I do to make the output be the same as the input ? And how do I do it in such a way that handles arbitrary video input formats that may store their time stamps and time bases in different places ? I need this to always work.

  ---

  For reference, here is a table of all the packet and frame timestamps read from the video stream of my test input file, to give a sense of what my test file looks like. None of the input packet pts’ are set, same with frame pts, and for some reason the duration of the first 108 frames is 0. VLC plays the file fine and reports the frame rate as 29.9700089 :

  • Table is here since it was too large for this post.