Recherche avancée

Sur d’autres sites (3395)

• C++ ffmpeg and SDL2 video rendering memory leak

  10 avril 2017, par kj192

  I have made a small program what plays a video in SDL2.0 and FFmpeg.
  The software does work and do it is purpose.
  I have left the software running and I have faced a huge memory consumption and started to look online what can I do against it.
  I have used the following tutorials :
  http://www.developersite.org/906-59411-FFMPEG
  http://ardrone-ailab-u-tokyo.blogspot.co.uk/2012/07/212-ardrone-20-video-decording-ffmpeg.html

  I wonder if someone can give advice what do I do wrong. I have tried valgrind but I can’t find any information. I have did try to comment out sections and what I have seen even if I’m not rendering to the display the memory usage is growing and after delete something still not been freed up :

  if (av_read_frame(pFormatCtx, &packet) >= 0) 

  the whole source code is here :
  main :

  #include 
  
  #include <ios>
  
  #include <iostream>
  
  #include <fstream>
  
  #include 
  
  #include <sdl2></sdl2>SDL.h>
  
  #include "video.h"
  
  using namespace std;
  
  
  
  void memory()
  
  {
  
  using std::ios_base;
  
  using std::ifstream;
  
  using std::string;
  
  
  
  double vm_usage     = 0.0;
  
  double resident_set = 0.0;
  
  
  
  // 'file' stat seems to give the most reliable results
  
  //
  
  ifstream stat_stream("/proc/self/stat",ios_base::in);
  
  
  
  // dummy vars for leading entries in stat that we don't care about
  
  //
  
  string pid, comm, state, ppid, pgrp, session, tty_nr;
  
  string tpgid, flags, minflt, cminflt, majflt, cmajflt;
  
  string utime, stime, cutime, cstime, priority, nice;
  
  string O, itrealvalue, starttime;
  
  
  
  // the two fields we want
  
  //
  
  unsigned long vsize;
  
  long rss;
  
  
  
  stat_stream >> pid >> comm >> state >> ppid >> pgrp >> session >> tty_nr
  
              >> tpgid >> flags >> minflt >> cminflt >> majflt >> cmajflt
  
              >> utime >> stime >> cutime >> cstime >> priority >> nice
  
              >> O >> itrealvalue >> starttime >> vsize >> rss; // don't care about the rest
  
  
  
  stat_stream.close();
  
  
  
  long page_size_kb = sysconf(_SC_PAGE_SIZE) / 1024; // in case x86-64 is configured to use 2MB pages
  
  vm_usage     = vsize / 1024.0;
  
  resident_set = rss * page_size_kb;
  
  std::cout&lt;&lt;"VM: " &lt;&lt; vm_usage &lt;&lt; " RE:"&lt;&lt; resident_set &lt;&lt; std::endl;
  
  }
  
  
  
  
  
  int main()
  
  {
  
  //This example using 1280x800 video
  
  av_register_all();
  
  if( SDL_Init(SDL_INIT_VIDEO | SDL_INIT_AUDIO | SDL_INIT_TIMER ))
  
  {
  
      fprintf(stderr, "Could not initialize SDL - %s\n", SDL_GetError());
  
      exit(1);
  
  }
  
  SDL_Window* sdlWindow = SDL_CreateWindow("Video Window", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 1280, 800, SDL_WINDOW_OPENGL);
  
  if( !sdlWindow )
  
  {
  
      fprintf(stderr, "SDL: could not set video mode - exiting\n");
  
      exit(1);
  
  }
  
  SDL_Renderer* sdlRenderer = SDL_CreateRenderer(sdlWindow, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC | SDL_RENDERER_TARGETTEXTURE);
  
  SDL_Texture* sdlTexture = SDL_CreateTexture(sdlRenderer, SDL_PIXELFORMAT_YV12, SDL_TEXTUREACCESS_STREAMING, 1280, 800);
  
  if(!sdlTexture)
  
  {
  
      return -1;
  
  }
  
  SDL_SetTextureBlendMode(sdlTexture,SDL_BLENDMODE_BLEND );
  
  //VIDEO RESOLUTION
  
  SDL_Rect sdlRect;
  
  sdlRect.x = 0;
  
  sdlRect.y = 0;
  
  sdlRect.w = 1280;
  
  sdlRect.h = 800;    
  
  memory();
  
  for(int i = 1; i &lt; 6; i++)
  
  {
  
      memory();   
  
      video* vid = new video("vid.mp4");  
  
      while (!vid -> getFinished())
  
      {
  
          memory();
  
          vid -> Update(sdlTexture);
  
          SDL_RenderCopy(sdlRenderer,sdlTexture,&amp;sdlRect,&amp;sdlRect);
  
          SDL_RenderPresent(sdlRenderer);
  
      }
  
      delete vid;
  
      memory();
  
  }   
  
  SDL_DestroyTexture(sdlTexture);
  
  SDL_DestroyRenderer(sdlRenderer);
  
  SDL_DestroyWindow(sdlWindow);
  
  SDL_Quit();
  
  return 0;
  
  }
  
  </fstream></iostream></ios>

  video.cpp

  #include "video.h"
  
  
  
  video::video(const std::string& name) : _finished(false)
  
  {
  
  av_register_all();
  
  pFormatCtx = NULL;
  
  pCodecCtxOrig = NULL;
  
  pCodecCtx = NULL;
  
  pCodec = NULL;
  
  pFrame = NULL;
  
  sws_ctx = NULL;
  
  if (avformat_open_input(&pFormatCtx, name.c_str(), NULL, NULL) != 0)
  
  {
  
  _finished = true; // Couldn't open file
  
  }
  
  // Retrieve stream information
  
  if (avformat_find_stream_info(pFormatCtx, NULL) < 0)
  
  {
  
  _finished = true; // Couldn't find stream information
  
  }
  
  videoStream = -1;
  
  for (i = 0; i < pFormatCtx->nb_streams; i++)
  
  {
  
      if (pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO) 
  
      {
  
          videoStream = i;
  
          break;
  
      }
  
  }
  
  if (videoStream == -1)
  
  {
  
      _finished = true; // Didn't find a video stream
  
  }
  
  // Get a pointer to the codec context for the video stream
  
  pCodecCtxOrig = pFormatCtx->streams[videoStream]->codec;
  
  // Find the decoder for the video stream
  
  pCodec = avcodec_find_decoder(pCodecCtxOrig->codec_id);
  
  if (pCodec == NULL) 
  
  {
  
      fprintf(stderr, "Unsupported codec!\n");
  
      _finished = true; // Codec not found
  
  }
  
  pCodecCtx = avcodec_alloc_context3(pCodec);
  
  if (avcodec_copy_context(pCodecCtx, pCodecCtxOrig) != 0) 
  
  {
  
      fprintf(stderr, "Couldn't copy codec context");
  
      _finished = true; // Error copying codec context
  
  }
  
  // Open codec
  
  if (avcodec_open2(pCodecCtx, pCodec, NULL) < 0)
  
  {
  
      _finished = true; // Could not open codec
  
  }
  
  // Allocate video frame
  
  pFrame = av_frame_alloc();
  
  sws_ctx = sws_getContext(pCodecCtx->width, pCodecCtx->height,
  
  pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height,
  
  AV_PIX_FMT_YUV420P,
  
  SWS_BILINEAR,
  
  NULL,
  
  NULL,
  
  NULL);
  
  yPlaneSz = pCodecCtx->width * pCodecCtx->height;
  
  uvPlaneSz = pCodecCtx->width * pCodecCtx->height / 4;
  
  yPlane = (Uint8*)malloc(yPlaneSz);
  
  uPlane = (Uint8*)malloc(uvPlaneSz);
  
  vPlane = (Uint8*)malloc(uvPlaneSz);
  
  if (!yPlane || !uPlane || !vPlane) 
  
  {
  
      fprintf(stderr, "Could not allocate pixel buffers - exiting\n");
  
      exit(1);
  
  }
  
  uvPitch = pCodecCtx->width / 2;
  
  }
  
  void video::Update(SDL_Texture* texture)
  
  {
  
  if (av_read_frame(pFormatCtx, &packet) >= 0) 
  
  {
  
      // Is this a packet from the video stream?
  
      if (packet.stream_index == videoStream) 
  
      {
  
          avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet);
  
          // Did we get a video frame?
  
          if (frameFinished) 
  
          {
  
              AVPicture pict;
  
              pict.data[0] = yPlane;
  
              pict.data[1] = uPlane;
  
              pict.data[2] = vPlane;
  
              pict.linesize[0] = pCodecCtx->width;
  
              pict.linesize[1] = uvPitch;
  
              pict.linesize[2] = uvPitch;
  
              // Convert the image into YUV format that SDL uses
  
              sws_scale(sws_ctx, (uint8_t const * const *) pFrame->data,pFrame->linesize, 0, pCodecCtx->height, pict.data,pict.linesize);
  
              SDL_UpdateYUVTexture(texture,NULL,yPlane,pCodecCtx->width,uPlane,uvPitch,vPlane,uvPitch);
  
          }
  
      }
  
      // Free the packet that was allocated by av_read_frame
  
      av_packet_unref(&packet);
  
      av_freep(&packet);
  
  }
  
  else
  
  {
  
      av_packet_unref(&packet);
  
      av_freep(&packet);
  
      _finished = true;
  
  }
  
  }
  
  bool video::getFinished()
  
  {
  
  return _finished;
  
  }
  
  video::~video()
  
  {
  
  av_packet_unref(&packet);
  
  av_freep(&packet);
  
  av_frame_free(&pFrame);
  
  av_freep(&pFrame);
  
  free(yPlane);
  
  free(uPlane);
  
  free(vPlane);
  
  // Close the codec
  
  avcodec_close(pCodecCtx);
  
  avcodec_close(pCodecCtxOrig);
  
  sws_freeContext(sws_ctx);
  
  // Close the video file
  
  for (int i = 0; i < pFormatCtx->nb_streams; i++) 
  
  {
  
      AVStream *stream = pFormatCtx->streams[i];
  
      avcodec_close(stream->codec);
  
  }
  
  avformat_close_input(&pFormatCtx);
  
  /*av_dict_free(&optionsDict);   
  
  sws_freeContext(sws_ctx);
  
  av_free_packet(&packet);
  
  av_free(pFrameYUV);
  
  av_free(buffer);
  
  avcodec_close(pCodecCtx);
  
  avformat_close_input(&pFormatCtx);*/
  
  }

  video.h
  

  #include <string>
  
  #include <sdl2></sdl2>SDL.h>
  
  #ifdef __cplusplus
  
  extern "C" {
  
  #endif
  
  #include <libavcodec></libavcodec>avcodec.h> 
  
  #include <libavformat></libavformat>avformat.h>
  
  #include <libswscale></libswscale>swscale.h>
  
  #ifdef __cplusplus 
  
  }
  
  #endif
  
  
  
  class video
  
  {
  
     private:
  
      bool _finished;
  
      AVFormatContext *pFormatCtx;
  
      int videoStream;
  
      unsigned i;
  
      AVCodecContext *pCodecCtxOrig;
  
      AVCodecContext *pCodecCtx;
  
      AVCodec *pCodec;
  
      AVFrame *pFrame;
  
      AVPacket packet;
  
      int frameFinished;
  
      struct SwsContext *sws_ctx;
  
      Uint8 *yPlane, *uPlane, *vPlane;
  
      size_t yPlaneSz, uvPlaneSz;
  
      int uvPitch;
  
     public:
  
      video(const std::string&amp; name);
  
      ~video();
  
      void Update(SDL_Texture* texture);
  
      bool getFinished();
  
  };
  
  </string>

  I’m looking forward to your answers

• Using ffmpeg to convert sound files for use in an android app

  10 janvier 2012, par stefs

  short : i'm trying to simply play a sound file converted with ffmpeg in my android app, but happen to have problems getting it to work.

  long : we have an iphone app and an android app doing the same thing, and i have to port the feature playing a sound on an user interaction. i have the source file in the aiff format, and tried to convert it to mp3 for android. but the app keeps crashing when it tries to load the file

  AssetFileDescriptor fileDescriptor = context.getResources().openRawResourceFd(resid);
  
  final MediaPlayer mp = new MediaPlayer();
  
  mp.setDataSource(fileDescriptor.getFileDescriptor(), fileDescriptor.getStartOffset(), fileDescriptor.getLength());
  
  fileDescriptor.close();
  
  mp.prepare();

  more specifically, mp.setDataSource crashes. some digging around led me to believe that something's wrong with the encoding. the sound file itself resides in res/raw.

  11-29 17:11:48.012: ERROR/SoundManager(15580): java.io.IOException: setDataSourceFD failed.: status=0x80000000
  
  11-29 17:11:48.012: ERROR/SoundManager(15580):     at android.media.MediaPlayer.setDataSource(Native Method)
  
  ...

  what i tried :

  • using a different mp3 that's already used with the same code in a different place. this works.
  • converted it to wav file. this didn't cause the app to crash, but it neither played a sound. that might be a different problem.
  • converted it to ogg ; crashed

  so, the the ffmpeg conversion parameters are as follows :

  $ ffmpeg -i click_24db.aif -f mp3 ~/foobar/wheel_click.mp3
  
  ffmpeg version 0.7.8, Copyright (c) 2000-2011 the FFmpeg developers
  
    built on Nov 24 2011 14:31:00 with gcc 4.2.1 (Apple Inc. build 5666) (dot 3)
  
    configuration: --prefix=/opt/local --enable-gpl --enable-postproc --enable-swscale --enable-avfilter --enable-libmp3lame --enable-libvorbis --enable-libtheora --enable-libdirac --enable-libschroedinger --enable-libopenjpeg --enable-libxvid --enable-libx264 --enable-libvpx --enable-libspeex --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/gcc-4.2 --arch=x86_64 --enable-yasm
  
    libavutil    50. 43. 0 / 50. 43. 0
  
    libavcodec   52.123. 0 / 52.123. 0
  
    libavformat  52.111. 0 / 52.111. 0
  
    libavdevice  52.  5. 0 / 52.  5. 0
  
    libavfilter   1. 80. 0 /  1. 80. 0
  
    libswscale    0. 14. 1 /  0. 14. 1
  
    libpostproc  51.  2. 0 / 51.  2. 0
  
  Input #0, aiff, from 'click_24db.aif':
  
    Duration: 00:00:00.01, start: 0.000000, bitrate: 1570 kb/s
  
      Stream #0.0: Audio: pcm_s16be, 44100 Hz, 2 channels, s16, 1411 kb/s
  
  Output #0, mp3, to '/Users/xyz/foobar/wheel_click.mp3':
  
    Metadata:
  
      TSSE            : Lavf52.111.0
  
      Stream #0.0: Audio: libmp3lame, 44100 Hz, 2 channels, s16, 64 kb/s
  
  Stream mapping:
  
    Stream #0.0 -> #0.0
  
  Press [q] to stop, [?] for help
  
  size=       1kB time=00:00:00.05 bitrate=  92.9kbits/s    
  
  video:0kB audio:0kB global headers:0kB muxing overhead 45.563549%

  the resulting file plays nice in itunes, does not play in vlc and crashes when loaded with the android.media.MediaPlayer (note : i first tried it with the SoundPool lib, with both mp3 and ogg, but that didn't work either).

  i also tried the following paramters, which didn't work :

  ffmpeg -i inputfile.aif -f mp3 -acodec libmp3lame -ab 192000 -ar 44100 outputfile.mp3

  i'm working on osx, built ffmpeg with macports today, android api level is 7 (google api, 2.1-update1). looking at the "supported formats" table on dev.android didn't indicate my file to be out of the spec, but i may be mistaken in that.

  i don't have the slightest clue regarding bitrates and so on, so could anybody please point me to the right combination of ffmpeg parameters to get a working mp3 for android ? i don't care if the resulting file would be mp3, ogg or 3gp or whatever.

• Dealing with problems in FLAC audio files with ffmpeg

  15 janvier 2020, par Seamus

  I have gotten a set of FLAC (audio) files from a friend. I copied them to my Sonos music library, and got set to enjoy a nice album. Unfortunately, Sonos would not play the files. As a result I have been getting to know ffmpeg.

  Sonos’ complaint with the FLAC files was that it was "encoded at an unsupported sample rate". With rolling eyes and shaking head, I note that the free VLC media player happily plays these files, but the product I’ve paid for (Sonos) - does not. But I digress...

  ffprobe revealed that the FLAC files contain both an Audio channel and a Video channel :

  $ ffprobe -hide_banner  -show_streams "/path/to/Myaudio.flac"
  
  
  
  
  
      Duration: 00:02:23.17, start: 0.000000, bitrate: 6176 kb/s  
  
      Stream #0:0: Audio: flac, 176400 Hz, stereo, s32 (24 bit)  
  
      Stream #0:1: Video: mjpeg (Progressive), yuvj444p(pc, bt470bg/unknown/unknown), 450x446 [SAR 72:72 DAR 225:223], 90k tbr, 90k tbn, 90k tbc (attached pic)  
  
      Metadata:  
  
        comment         : Cover (front)  

  Cool ! I guess this is how some audio players are able to display the ’album artwork’ when they play a song ? Note also that the Audio stream is reported at 176400 Hz ! Apparently I’m out of touch ; I thought that 44.1khz sampling rate effectively removed all of the ’sampling artifacts’ we could hear. Anyway, I learned that Sonos would support a max of 48kHz sampling rate, and this (the 176.4kHz rate) is what Sonos was unhappy about. I used ffmpeg to ’dumb it down’ for them :

  $ ffmpeg -i "/path/to/Myaudio.flac" -sample_fmt s32 -ar 48000 "/path/to/Myaudio48K.flac"

  This seemed to work - at least I got a FLAC file that Sonos would play. However, I also got what looks like a warning of some sort :

  [swscaler @ 0x108e0d000] deprecated pixel format used, make sure you did set range correctly
  [flac @ 0x7feefd812a00] Frame rate very high for a muxer not efficiently supporting it.
  Please consider specifying a lower framerate, a different muxer or -vsync 2

  A bit more research turned up this answer which I don’t quite understand, and then in a comment says, "not to worry" - at least wrt the swscaler part of the warning.

  And that (finally) brings me to my questions :

  1.a. What framerate, muxer & other specifications make a graphic compatible with a majority of programs that use the graphic ?

  1.b. How should I use ffmpeg to modify the Video channel to set these specifications (ref. Q 1.a.) ?

  2.a. How do I remove the Video channel from the .flac audio file ?

  2.b. How do I add a Video channel into a .flac file ?

  EDIT :

  I asked the above (4) questions after failing to accomplish a ’direct’ conversion (a single ffmpeg command) from FLAC at 176.4 kHz to ALAC (.m4a) at 48 kHz (max supported by Sonos). I reasoned that an ’incremental’ approach through a series of conversions might get me there. With the advantage of hindsight, I now see I should have posted my original failed direct conversion incantation... we live and learn.

  That said, the accepted answer below meets my final objective to convert a FLAC file encoded at 176.4kHz to an ALAC (.m4a) at 48kHz, and preserve the cover art/video channel.