Recherche avancée

Sur d’autres sites (5789)

• AVIOContext custom stream playback glitches/corruption on UDP stream

  21 décembre 2017, par WLGfx

  I’ve written a general all round player (for Android OS) which plays file based streams as well as streams from network sources, and it all works good. File based streams I’ve got the seeking which works too.

  What it is I’m now struggling with is now I’m using AVIOContext to latch onto a UDP stream which saves the packet data partially in memory and then to transient storage. This is so I can pause live TV and also seek within it.

  However, after much faffing about, during playback (seeking is only partial at the moment), either the video frame rate will drop from 25FPS (will be deinterlaced on higher spec devices) down to between 17 and 19 frames per second, or, it will glitch and grey out.

  When I playback the record TS data from the file, it plays perfect, so the UDP buffering and writing out the overflow is sound. (This is currently not true now, only currently a minor issue)

  I’m at the point were I’ve spent a lot of time on this and I’m at a loss as to why I get either frame drops or glitches.

  The class def :

  #define PKT_SIZE (188)
  
  #define PKT_SIZE7 (PKT_SIZE * 7)
  
  
  
  #define UDP_QUEUE_SIZE (12000)
  
  #define UDP_THRESHOLD (100)
  
  
  
  #define FILE_QUEUE_PKTS (200000)
  
  
  
  #define AVIO_QUEUE_SIZE (24)
  
  #define AVIO_THRESHOLD (10)
  
  
  
  extern "C" {
  
  #include "libavformat/avio.h"
  
  };
  
  
  
  /*
  
   * PracticalSocket class as found here with examples:
  
   * http://cs.baylor.edu/~donahoo/practical/CSockets/practical/
  
   */
  
  
  
  #include "PracticalSocket.h"
  
  
  
  class FFIOBufferManager2 {
  
  
  
  public:
  
  
  
      AVIOContext *avioContext = nullptr;
  
  
  
      bool quit = false;
  
  
  
      char udp_buffer[UDP_QUEUE_SIZE][PKT_SIZE7];
  
      int udp_write_pos, udp_size; // by PKT_SIZE7
  
      char *get_udp_buffer(int index);
  
      int get_udp_buffer_size() { return udp_size; }
  
  
  
      int file_write_pos, file_size; // by PKT_SIZE7
  
      std::fstream file_out, file_in;
  
      std::mutex udp_mutex, file_mutex;
  
      std::thread udp_thread, file_thread;
  
  
  
      static void udp_thread_func(FFIOBufferManager2 *io, const char *ip, int port);
  
      static void file_thread_func(FFIOBufferManager2 *io, const char *dir);
  
      void udp_thread_run(const char *ip, int port);
  
      void file_thread_run();
  
  
  
      char avio_buffer[AVIO_QUEUE_SIZE * 7 * PKT_SIZE];
  
      int64_t avio_read_offset; // controlled by udp mutex (quickest)
  
      static int avio_read(void *ptr, uint8_t *buff, int buf_size);
  
      static int64_t avio_seek(void *ptr, int64_t pos, int whence);
  
      int avio_read_run(uint8_t *buf, int buf_size);
  
      int64_t avio_seek_run(int64_t pos, int whence);
  
      void write_udp_overflow();
  
  
  
      void start(const char *ip, int port, const char *dir);
  
  
  
      void get_size_and_pos(int64_t *size, int64_t *pos);
  
  
  
      ~FFIOBufferManager2();
  
  };

  The classes methods :

  #include 
  
  #include "FFIOBufferManager2.h"
  
  
  
  #include "LOG.h"
  
  
  
  void FFIOBufferManager2::start(const char *ip, int port, const char *dir) {
  
  
  
      file_write_pos = 0;
  
      file_size = 0;
  
  
  
      udp_write_pos = 0;
  
      udp_size = 0;
  
      avio_read_offset = 0;
  
  
  
      file_thread = std::thread(&FFIOBufferManager2::file_thread_func, this, dir);
  
      udp_thread = std::thread(&FFIOBufferManager2::udp_thread_func, this, ip, port);
  
  
  
      LOGD("Initialising avioContext");
  
  
  
      avioContext = avio_alloc_context((uint8_t*)avio_buffer,
  
                                       AVIO_QUEUE_SIZE * PKT_SIZE7,
  
                                       0,
  
                                       this,
  
                                       avio_read,
  
                                       NULL,
  
                                       avio_seek);
  
  }
  
  
  
  void FFIOBufferManager2::udp_thread_func(FFIOBufferManager2 *io, const char *ip, int port) {
  
  
  
      LOGD("AVIO UDP thread started address %s port %d - %08X", ip, port, (uint)io);
  
  
  
      io->udp_thread_run(ip, port); // run inside class
  
  
  
      LOGD("AVIO UDP thread stopped");
  
  }
  
  
  
  void FFIOBufferManager2::udp_thread_run(const char *ip, int port) {
  
  
  
      std::string addr = ip;
  
  
  
      UDPSocket socket(addr, (uint16_t)port);
  
      socket.joinGroup(addr);
  
  
  
      LOGD("UDP loop starting");
  
  
  
      while (!quit) {
  
  
  
          if (socket.recv(get_udp_buffer(udp_write_pos), PKT_SIZE7) == PKT_SIZE7) {
  
  
  
              udp_mutex.lock();
  
  
  
              udp_write_pos = (udp_write_pos + 1) % UDP_QUEUE_SIZE;
  
              udp_size++;
  
              if (udp_size >= UDP_QUEUE_SIZE) udp_size--;
  
              else avio_read_offset += PKT_SIZE7;
  
  
  
              udp_mutex.unlock();
  
          }
  
      }
  
  }
  
  
  
  void FFIOBufferManager2::file_thread_func(FFIOBufferManager2 *io, const char *dir) {
  
  
  
      LOGD("AVIO FILE thread started");
  
  
  
      std::string file = dir;
  
      const char *tsfile_name = "/tsdata.ts";
  
      file += tsfile_name;
  
  
  
      LOGD("Deleting old file %s", file.c_str());
  
  
  
      remove(file.c_str());
  
  
  
      {
  
          fstream temp; // create the ts file
  
          temp.open(file.c_str());
  
          temp.close();
  
      }
  
  
  
      LOGD("Opening %s for read and write", file.c_str());
  
  
  
      io->file_out.open(file, fstream::out | fstream::binary);
  
      io->file_in.open(file, fstream::in | fstream::binary);
  
  
  
      io->file_thread_run(); // continue inside the class to lessen pointer use
  
  
  
      LOGD("AVIO FILE thread stopped");
  
  }
  
  
  
  void FFIOBufferManager2::file_thread_run() {
  
  
  
      LOGD("FILE thread run");
  
  
  
      if (!file_out.is_open() || !file_in.is_open()) {
  
  
  
          LOGE("TS data file, error opening...");
  
          quit = true;
  
          return;
  
      }
  
  
  
      int udp_threshold = UDP_QUEUE_SIZE - (UDP_THRESHOLD * 4);
  
  
  
      while (!quit) {
  
  
  
          if (udp_size >= udp_threshold) write_udp_overflow();
  
          else usleep(1000 * 1);
  
      }
  
  }
  
  
  
  void FFIOBufferManager2::write_udp_overflow() {
  
  
  
      file_mutex.lock();
  
  
  
      udp_mutex.lock();
  
      int udp_write_pos_current = udp_write_pos;
  
      int udp_size_current = udp_size;
  
      udp_mutex.unlock();
  
  
  
      int udp_index = udp_write_pos_current - udp_size_current;
  
      if (udp_index < 0) udp_index += UDP_QUEUE_SIZE;
  
      int written = 0;
  
  
  
      //file_out.seekp((int64_t)file_write_pos * PKT_SIZE7);
  
  
  
      while (written < UDP_THRESHOLD) {
  
          file_out.write(get_udp_buffer(udp_index), PKT_SIZE7);
  
          written++;
  
          udp_index = (udp_index + 1) % UDP_QUEUE_SIZE;
  
  
  
          file_write_pos++;
  
          if (file_write_pos >= FILE_QUEUE_PKTS) {
  
              file_write_pos = 0;
  
              file_out.seekp(0);
  
          }
  
          file_size++;
  
          if (file_size > FILE_QUEUE_PKTS) file_size = FILE_QUEUE_PKTS;
  
      }
  
  
  
      udp_mutex.lock();
  
      udp_size -= UDP_THRESHOLD; // we've written this amount out
  
      udp_mutex.unlock();
  
  
  
      //file_out.flush();
  
  
  
      file_mutex.unlock();
  
  
  
      //LOGD("Written UDP overflow at %d of %d blocks file size %d",
  
      //     udp_index, written, file_size);
  
  }
  
  
  
  char *FFIOBufferManager2::get_udp_buffer(int index) {
  
      if (index < 0 || index >= UDP_QUEUE_SIZE) return nullptr;
  
      return ((char*)udp_buffer + (index * PKT_SIZE7));
  
  }
  
  
  
  /*
  
   * The avio_read and avio_seek now work on either 188 byte alignment or
  
   * byte alignment for the benefit of ffmpeg - byte positioning at the moment
  
   *
  
   * The file_mutex allows for either a read or write operation at a time
  
   */
  
  
  
  int FFIOBufferManager2::avio_read(void *ptr, uint8_t *buff, int buf_size) {
  
      FFIOBufferManager2 *io = (FFIOBufferManager2*)ptr;
  
      return io->avio_read_run(buff, buf_size);
  
  }
  
  
  
  int64_t FFIOBufferManager2::avio_seek(void *ptr, int64_t pos, int whence) {
  
      FFIOBufferManager2 *io = (FFIOBufferManager2*)ptr;
  
      return io->avio_seek_run(pos, whence);
  
  }
  
  
  
  int FFIOBufferManager2::avio_read_run(uint8_t *buf, int buf_size) {
  
  
  
      file_mutex.lock();
  
      udp_mutex.lock();
  
  
  
      int64_t cur_udp_write_pos = (int64_t) udp_write_pos * PKT_SIZE7;
  
      int64_t cur_udp_size = (int64_t) udp_size * PKT_SIZE7;
  
  
  
      int64_t cur_file_write_pos = (int64_t) file_write_pos * PKT_SIZE7;
  
      int64_t cur_file_size = (int64_t) file_size * PKT_SIZE7;
  
  
  
      int64_t cur_avio_read_offset = avio_read_offset; // already int64_t (under the udp_mutex)
  
  
  
      udp_mutex.unlock();
  
  
  
      if (cur_avio_read_offset < (AVIO_THRESHOLD * 4) * PKT_SIZE7) {
  
          file_mutex.unlock();
  
          return 0;
  
      }
  
  
  
      int64_t udp_buffer_max = (int64_t) (UDP_QUEUE_SIZE * PKT_SIZE7);
  
      int64_t file_buffer_max = (int64_t) (FILE_QUEUE_PKTS * PKT_SIZE7);
  
      uint8_t *ptr_udp_buffer = (uint8_t*)udp_buffer;
  
      int cur_written = 0;
  
  
  
      int file_reads = 0, udp_reads = 0; // for debugging
  
  
  
      int64_t cur_file_offset = cur_file_write_pos - cur_udp_size - cur_avio_read_offset;
  
      while (cur_file_offset < 0) cur_file_offset += file_buffer_max;
  
  
  
      if (cur_file_offset >= 0) {
  
          file_in.seekg(cur_file_offset);
  
  
  
          while (//cur_avio_read_offset > 0
  
                 cur_avio_read_offset > cur_udp_size
  
                 && cur_written < buf_size) { // read from file first
  
  
  
              file_in.read(&avio_buffer[cur_written], PKT_SIZE); // get 1 or 188 byte/s
  
  
  
              cur_file_offset+=PKT_SIZE;
  
              if (cur_file_offset >= file_buffer_max) { // back to file beginning
  
                  cur_file_offset = 0;
  
                  file_in.seekg(0);
  
              }
  
  
  
              cur_avio_read_offset-=PKT_SIZE;
  
  
  
              cur_written+=PKT_SIZE;
  
              file_reads+=PKT_SIZE;
  
          }
  
      }
  
  
  
      int64_t cur_udp_offset = (cur_udp_write_pos - cur_avio_read_offset);
  
      if (cur_udp_offset < 0) cur_udp_offset += udp_buffer_max;
  
  
  
      while (cur_avio_read_offset > AVIO_THRESHOLD * PKT_SIZE7
  
              && cur_avio_read_offset <= cur_udp_size
  
              && cur_written < buf_size) { // read the rest from udp buffer
  
  
  
          buf[cur_written] = ptr_udp_buffer[cur_udp_offset]; // get byte
  
  
  
          cur_udp_offset = (cur_udp_offset + 1) % udp_buffer_max;
  
          if (cur_udp_offset == 0) LOGD("AVIO UDP BUFFER to start");
  
  
  
          cur_avio_read_offset--;
  
  
  
          cur_written++;
  
          udp_reads++;
  
      }
  
  
  
      udp_mutex.lock();
  
      avio_read_offset -= cur_written;
  
      udp_mutex.unlock();
  
  
  
      file_mutex.unlock();
  
  
  
      if (cur_written) {
  
          LOGD("AVIO_READ: Written %d of %d, avio_offset %lld, file reads %d, udp reads %d, udp offset %lld, file offset %lld, file size %lld",
  
               cur_written, buf_size,
  
               cur_avio_read_offset,
  
               file_reads, udp_reads,
  
               cur_udp_write_pos, cur_file_write_pos, cur_file_size);
  
      }
  
  
  
      return cur_written;
  
  }
  
  
  
  int64_t FFIOBufferManager2::avio_seek_run(int64_t pos, int whence) {
  
      // SEEK_SET(0), SEEK_CUR(1), SEEK_END(2), AVSEEK_SIZE
  
  
  
      int64_t new_pos = -1;
  
      int64_t full_length = (udp_size + file_size) * PKT_SIZE7;
  
  
  
      switch (whence) {
  
  
  
          case AVSEEK_SIZE:
  
              LOGD("AVSEEK_SIZE pos %lld", pos);
  
              break;
  
  
  
          case SEEK_SET:
  
              LOGD("AVSEEK_SET pos %lld", pos);
  
              if (pos > full_length) new_pos = full_length;
  
              else new_pos = full_length - pos;
  
              break;
  
  
  
          case SEEK_CUR:
  
              LOGD("AVSEEK_CUR pos %lld", pos);
  
              break;
  
  
  
          case SEEK_END:
  
              LOGD("AVSEEK_END pos %lld", pos);
  
              new_pos = pos;
  
              break;
  
  
  
          default:
  
              LOGD("UNKNOWN AVIO SEEK whence %d pos %lld", whence, pos);
  
              break;
  
      }
  
  
  
      if (new_pos >= 0) {
  
  
  
          udp_mutex.lock();
  
  
  
          new_pos = (new_pos / PKT_SIZE) * PKT_SIZE; // align to packet boundary
  
          avio_read_offset = new_pos;
  
          //file_out.seekg(full_length - new_pos);
  
  
  
          udp_mutex.unlock();
  
  
  
          return full_length - new_pos;
  
      }
  
  
  
      return -1;
  
  }
  
  
  
  FFIOBufferManager2::~FFIOBufferManager2() {
  
      if (avioContext) ;// TODO whoops
  
      quit = true;
  
      if (udp_thread.joinable()) udp_thread.join();
  
      if (file_thread.joinable()) file_thread.join();
  
  }
  
  
  
  void FFIOBufferManager2::get_size_and_pos(int64_t *size, int64_t *pos) {
  
      file_mutex.lock();
  
      udp_mutex.lock();
  
  
  
      *size = (udp_size + file_size) * PKT_SIZE7;
  
      *pos = *size - avio_read_offset;
  
  
  
      udp_mutex.unlock();
  
      file_mutex.unlock();
  
  }

  It’ll play for a few seconds before any of the glitches start to appear. I have checked against the udp_buffer and the avio_buffer, but my suspicions lie with one of two things :

  1. Reading and writing to the file.
  2. the avio_read method is wrong.

  Has anybody got any input as to why this is occurring ? Any thoughts would be greatly appreciated.

  If you need any more information I’ll be glad to provide more details.

  EDIT : Seeking now actually moves to any point within the stream, but now doesn’t read from the file recording. Although that’s only a minor issue at the moment.

  The main two issues still stand, frame rate drops dramatically and the glitches after approximately 8 seconds.

• Dreamcast Serial Extractor

  31 décembre 2017, par Multimedia Mike — Sega Dreamcast

  It has not been a very productive year for blogging. But I started the year by describing an unfinished project that I developed for the Sega Dreamcast, so I may as well end the year the same way. The previous project was a media player. That initiative actually met with some amount of success and could have developed into something interesting if I had kept at it.

  By contrast, this post describes an effort that was ultimately a fool’s errand that I spent way too much time trying to make work.

  Problem Statement
  In my neverending quest to analyze the structure of video games while also hoarding a massive collection of them (though I’m proud to report that I did play at least a few of them this past year), I wanted to be able to extract the data from my many Dreamcast titles, both games and demo discs. I had a tool called the DC Coder’s Cable, a serial cable that enables communication between a Dreamcast and a PC. With the right software, you could dump an entire Dreamcast GD-ROM, which contained a gigabyte worth of sectors.

  Problem : The dumping software (named ‘dreamrip’ and written by noted game hacker BERO) operated in a very basic mode, methodically dumping sector after sector and sending it down the serial cable. This meant that it took about 28 hours to extract all the data on a single disc by running at the maximum speed of 115,200 bits/second, or about 11 kilobytes/second. I wanted to create a faster method.

  The Pitch
  I formed a mental model of dreamrip’s operation that looked like this :

  
  
  

  As an improvement, I envisioned this beautiful architecture :
  

  
  
  

  Architectural Assumptions
  My proposed architecture was predicated on the assumption that the disc reading and serial output functions were both I/O-bound operations and that the CPU would be idle much of the time. My big idea was to use that presumably idle CPU time to compress the sectors before sending them over the wire. As long as the CPU can compress the data faster than 11 kbytes/sec, it should be a win. In order to achieve this, I broke the main program into 3 threads :

  1. The first thread reads the sectors ; more specifically, it asks the drive firmware to please read the sectors and make the data available in system RAM
  2. The second thread waits for sector data to appear in memory and then compresses it
  3. The third thread takes the compressed data when it is ready and shuffles it out through the serial cable

  Simple and elegant, right ?

  For data track compression, I wanted to start with zlib in order to prove the architecture, but then also try bzip2 or lzma. As long as they could compress data faster than the serial port could write it, then it should be a win. For audio track compression, I wanted to use the Flake FLAC encoder. According to my notes, I did get both bzip2 compression and the Flake compressor working on the Dreamcast. I recall choosing Flake over the official FLAC encoder because it was much simpler and had fewer dependencies, always an important consideration for platforms such as this.

  Problems
  I worked for quite awhile on this project. I have a lot of notes recorded but a lot of the problems I had remain a bit vague in my memory. However, there was one problem I discovered that eventually sunk the entire initiative :

  The serial output operation is CPU-bound.

  My initial mental model was that the a buffer could be “handed off” to the serial subsystem and the CPU could go back to doing other work. Nope. Turns out that the CPU was participating at every step of the serial transfer.

  Further, I eventually dug into the serial driver code and learned that there was already some compression taking place via the miniLZO library.

  Lessons Learned

  • Recognize the assumptions that you’re making up front at the start of the project.
  • Prototype in order to ensure plausibility
  • Profile to make sure you’re optimizing the right thing (this is something I have learned again and again).

  Another interesting tidbit from my notes : it doesn’t matter how many sectors you read at a time, the overall speed is roughly the same. I endeavored to read 1000 2048-byte data sectors, 1 or 10 or 100 at a time, or all 1000 at once. My results :

  • 1 : 19442 ms
  • 10 : 19207 ms
  • 100 : 19194 ms
  • 1000 : 19320 ms

  No difference. That surprised me.

  Side Benefits
  At one point, I needed to understand how BERO’s dreamrip software was operating. I knew I used to have the source code but I could no longer find it. Instead, I decided to try to reverse engineer what I needed from the SH-4 binary image that I had. It wasn’t an ELF image ; rather, it was a raw binary meant to be loaded at a particular memory location which makes it extra challenging for ‘objdump’. This led to me asking my most viewed and upvoted question on Stack Overflow : “Disassembling A Flat Binary File Using objdump”. The next day, it also led me to post one of my most upvoted answers when I found the solution elsewhere.

  Strangely, I have since tried out the command line shown in my answer and have been unable to make it work. But people keep upvoting both the question and the answer.

  Eventually this all became moot when I discovered a misplaced copy of the source code on one of my computers.

  I strongly recall binging through the Alias TV show while I was slogging away on this project, so I guess that’s a positive association since I got so many fun screenshots out of it.

  The Final Resolution
  Strangely, I was still determined to make this project work even though the Dreamcast SD adapter arrived for me about halfway through the effort. Part of this was just stubbornness, but part of it was my assumptions about serial port speeds, in particular, my assumption that there was a certain speed-of-light type of limitation on serial port speeds so that the SD adapter, operating over the DC’s serial port, would not be appreciably faster than the serial cable.

  This turned out to be very incorrect. In fact, the SD adapter is capable of extracting an entire gigabyte disc image in 35-40 minutes. This is the method I have since been using to extract Dreamcast disc images.

  The post Dreamcast Serial Extractor first appeared on Breaking Eggs And Making Omelettes.

• ffmpeg drawtext RGB color codes

  19 janvier 2018, par Manish Jha

  I have an ffmpeg command which overlays an image and some text on a video where arguments are :

   -i "C:\exports\blue.mp4"
  
   -i "D:\Manish\Videos\logo.png" 
  
   -filter_complex 
  
   "[0:v]drawtext=fontfile='C\:\\Windows\\Fonts\\arial.ttf':
  
    text='falana dhimaka':
  
    fontcolor=AntiqueWhite:
  
    fontsize=20:x=1000:y=10[text]; 
  
    [text][1:v]overlay=140:5[filtered]" -map "[filtered]" -map 0:a? -codec:a copy 
  
    -preset ultrafast -ac 2 "C:\exports\blue_wm.mp4"

  This works well, but I want to be able to supply RGB Hex instead of AntiqueWhite. Any help is appreciated.