Recherche avancée

Sur d’autres sites (4368)

• mpeg2 ts android ffmpeg openmax

  10 octobre 2014, par WLGfx

  The setup is as follows :

  1. Multicast server 1000Mbs, UDP, Mpeg2-TS Part 1 (H.222) streaming live TV-channels.
  2. Quad core 1.5Ghz Android 4.2.2 GLES 2.0 renderer.
  3. FFMpeg library.
  4. Eclipse Kepler, Android SDK/NDK, etc. Running on Windows 8.1.
  5. Output screen 1920 x 1080, I am using a texture of 2048 x 1024 and getting between 35 and 45 Frames per second.

  The app :

  1. Renderer thread runs continuously and updates a single texture by uploading segments to the gpu when media images are ready.
  2. Media handler thread, downloads and processes media from server/or local storage.
  3. Video thread(s), one for buffering the UDP packets and another for decoding the packets into frames.

  I am connecting ffmpeg to the UDP stream just fine and the packets are being buffered and seemingly decoded fine. The packet buffers are plenty, no under/over-flows. The problem I am facing is it appears to be chopping up frames (ie only playing back 1 out of every so many frames). I understand that I need to distinguish I/P/B frames, but at the moment, hands up, I ain’t got a clue. I’ve even tried a hack to detect I frames to no avail. Plus, I am only rendering the frames to less than a quarter of the screen. So I’m not using full screen decoding.

  The decoded frames are also stored in separate buffers to cut out page tearing. The number of buffers I’ve changed too, from 1 to 10 with no luck.

  From what I’ve found about OpenMax IL, is it only handles MPeg2-TS Part 3 (H.264 and AAC), but you can use your own decoder. I understand that you can add your own decode component to it. Would it be worth me trying this route or should I continue on with ffmpeg ?

  The frame decoder (only the renderer will convert and scale the frames when ready)
  /*
  * This function will run through the packets and keep decoding
  * until a frame is ready first, or out of packets
  */

  while (packetsUsed[decCurrent])
  
  {
  
      hack_for_i_frame:
  
      i = avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packets[decCurrent]);
  
      packetsUsed[decCurrent] = 0; // finished with this one
  
      i = packets[decCurrent].flags & 0x0001;
  
      decCurrent++;
  
      if (decCurrent >= MAXPACKETS) decCurrent = 0;
  
      if (frameFinished)
  
      {
  
          ready_pFrame = pFrame;
  
          frameReady = true;  // notify renderer
  
          frameCounter++;
  
          if (frameCounter>=MAXFRAMES) frameCounter = 0;
  
          pFrame = pFrames[frameCounter];
  
          return 0;
  
      }
  
      else if (i)
  
          goto hack_for_i_frame;
  
  }
  
  
  
  return 0;

  The packet reader (spawned as a pthread)
  void *mainPacketReader(void *voidptr)
  
  int res ;

  while ( threadState == TS_RUNNING )
  
  {
  
      if (packetsUsed[prCurrent])
  
      {
  
          LOGE("Packet buffer overflow, dropping packet...");
  
          av_read_frame( pFormatCtx, &packet );
  
      }
  
      else if ( av_read_frame( pFormatCtx, &packets[prCurrent] ) >= 0 )
  
      {
  
          if ( packets[prCurrent].stream_index == videoStream )
  
          {
  
              packetsUsed[prCurrent] = 1; // flag as used
  
              prCurrent++;
  
              if ( prCurrent >= MAXPACKETS )
  
              {
  
                  prCurrent = 0;
  
              }
  
          }
  
  
  
          // here check if the packet is audio and add to audio buffer
  
      }
  
  }
  
  return NULL;

  And the renderer just simply does this
  // texture has already been bound before calling this function

  if ( frameReady == false ) return;
  
  
  
  AVFrame *temp;  // set to frame 'not' currently being decoded
  
  temp = ready_pFrame;
  
  
  
  sws_scale(sws_ctx,(uint8_t const* const *)temp->data,
  
          temp->linesize, 0, pCodecCtx->height,
  
          pFrameRGB->data, pFrameRGB->linesize);
  
  
  
  glTexSubImage2D(GL_TEXTURE_2D, 0,
  
          XPOS, YPOS, WID, HGT,
  
          GL_RGBA, GL_UNSIGNED_BYTE, buffer);
  
  
  
  frameReady = false;

  In the past, libvlc had audio syncing problems too, so that is my decision for going with ffmpeg and doing all the donkey work from scratch.

  If anybody has any pointers of how to stop the choppiness of the video playback (works great in VLC player) or possibly another route to go down, it would be seriously appreciated.

  EDIT I removed the hack for the I-frame (completely useless). Move the sws_scale function from the renderer to the packet decoder. And I left the udp packet reader thread alone.

  In the meantime I’ve also changed the packet reader thread and the packet decoder threads priority to real-time. Since doing that I don’t get shed loads of dropped packets.

• using audioconvert with ffmpeg

  17 juillet 2014, par lama

  I’m using this example : http://quick-apps.com/audio/ in my project to convert audio files. but my program is not using browse file, I’m recording the file using html5 and i’m getting the data of that recorded file and convert it upon user selection into audio format then display the record with the new format.
  the code of the demo is found in the github,this is how it’s sending the data of the browsed file :

  ffmpegWorker.postMessage({
  
  
  
            type: "command",
  
            arguments: arguments,
  
            files: [
  
                {
  
                    "name": fileName,
  
                    "buffer": fileBuffer
  
                }
  
            ]
  
  
  
        });

  fileName and fileBuffer are obtained by reader.onload

   fileName = file.name;
  
   fileBuffer = e.target.result;
  
   reader.readAsArrayBuffer(file);

  in my case (converting the recorded audio), im using the above code :

  fileName = encodeURIComponent('audio_recording_' + new Date().getMinutes() + '.wav');;
  
  bufferFile = event.target.result;
  
  reader.readAsDataURL(blob);

  the file is not converted.
  what is the problem ? although if i use the browse file program and browsed the recorded file and convert it, its working, but converting the same file in the backcode as shown above is not working if somebody can check the github code and get back to me please to help me with my error.
  thank you in advance

• building voice recorder using ffmpeg and html5

  15 juillet 2014, par user3789242

  I’m building a voice recorder program using html5 I have managed so far to record the voice and save the audio file as .wav I also can convert that file to a selected file format using ffmpeg. what I want to do is to let the user choose the format before recording so that when saving the audio file will be converted directly using ffmpeg into the selected format.I have been looking for demos for weeks and can’t find anything if someone can help me with a demo please.
  thank you in advance
  this is my javascript code for uploading the saved audio as .wav in a folder named upload, and the fmpeg conversion, i’m missing the call of the file saved in the folder please :

  function handleWAV(blob) {
  
  
  
  if (currentEditedSoundIndex !== -1) {
  
  $('#inFile2 tr:nth-child(' + (currentEditedSoundIndex + 1) + ')').remove();
  
  
  
  }
  
  
  
  var url = URL.createObjectURL(blob);
  
  
  
  var li = document.createElement('li');
  
  var au = document.createElement('audio');
  
  var hf = document.createElement('a');
  
  
  
  au.controls = true;
  
  au.src = url;
  
  hf.href = url;
  
  hf.download = 'audio_recording_' + new Date().getTime() + '.wav';
  
  hf.innerHTML = hf.download;
  
  li.appendChild(au);
  
  li.appendChild(hf);
  
  inFile2.appendChild(li);
  
  
  
  
  
  fileName=hf.download;
  
  
  
  var reader = new FileReader();
  
  reader.onload = function(event){
  
      var fd = new FormData();
  
      var Name = encodeURIComponent('audio_recording_' + new Date().getTime() + '.wav');
  
      console.log("name = " + Name);
  
      fd.append('fname', Name);
  
      fd.append('data', event.target.result);
  
      $.ajax({
  
          type: 'POST',
  
          url: 'upload.php',
  
          data: fd,
  
          processData: false,
  
          contentType: false,
  
          success: function(data){
  
              //console.log(data);
  
          }
  
      });
  
  };      
  
  reader.readAsDataURL(blob);
  
  
  
  
  
  
  
   var fileBuffer;
  
  
  
  
  
  
  
   // create ffmpeg worker
  
   function getFFMPEGWorker() {
  
  
  
  
  
       var ffmpegWorker = new Worker('worker.js');
  
  
  
       ffmpegWorker.addEventListener('message', function(event) {
  
           var message = event.data;
  
           console.log(message.type);
  
           if (message.type === "ready" && window.File && window.FileList && window.FileReader) {
  
               // script loaded, hide loader
  
  
  
           } else if (message.type == "stdout") {
  
               console.log(message.data);
  
           } else if (message.type == "stderr") {
  
               console.log(message.data);
  
           } else if (message.type == "done") {
  
               var code = message.data.code;
  
               console.log(code);
  
               console.log(message.data);
  
               var outFileNames = Object.keys(message.data.outputFiles);
  
  
  
               console.log(outFileNames);
  
               if (code == 0 && outFileNames.length) {
  
                   var outFileName = outFileNames[0];
  
                   console.log(outFileName);
  
                   var outFileBuffer = message.data.outputFiles[outFileName];
  
                   console.log(outFileBuffer);
  
                   var src = url;
  
                   console.log(url);
  
                   $("#downloadLink2").attr('href', src);
  
                   $("#download2").show();
  
               } else {
  
                   $("#error").show();
  
               }
  
  
  
           }
  
       }, false);
  
       return ffmpegWorker;
  
   }
  
  
  
   // create ffmpeg worker
  
   var ffmpegWorker = getFFMPEGWorker();
  
   var ffmpegRunning = false;
  
   if (ffmpegRunning) {
  
       ffmpegWorker.terminate();
  
       ffmpegWorker = getFFMPEGWorker();
  
   }
  
   ffmpegRunning = true;
  
  
  
  
  
  
  
   // hide download div
  
   $("#download2").hide();
  
  
  
   // change download file name
  
   var fileNameExt = fileName.substr(fileName.lastIndexOf('.') + 1);
  
  
  
   var outFileName = fileName.substr(0, fileName.lastIndexOf('.')) + "." + getOutFormat();
  
  
  
   $("#downloadLink2").attr("download2", outFileName);
  
   $("#downloadLink2").text(outFileName);
  
  
  
   var arguments = [];
  
   arguments.push("-i");
  
   arguments.push(fileName);
  
  
  
   arguments.push("-b:a");
  
   arguments.push(getBitrate());
  
  
  
   switch (getOutFormat()) {
  
       case "mp3":
  
           arguments.push("-acodec");
  
           arguments.push("libmp3lame");
  
           arguments.push("out.mp3");
  
           break;
  
  
  
       case "wma":
  
           arguments.push("-acodec");
  
           arguments.push("wmav1");
  
           arguments.push("out.asf");
  
           break;
  
  
  
       case "pcm":
  
           arguments.push("-f");
  
           arguments.push("s16le");
  
           arguments.push("-acodec");
  
           arguments.push("pcm_s16le");
  
           arguments.push("out.pcm");
  
   }
  
  
  
   ffmpegWorker.postMessage({
  
       type: "command",
  
       arguments: arguments,
  
       files: [
  
           {
  
               "name": fileName,
  
               "buffer": fileBuffer
  
           }
  
       ]
  
   });
  
  
  
  
  
  function getOutFormat() {
  
   return $('input[name=format]:checked').val();
  
  }
  
  
  
  function getBitrate() {
  
   return $('input[name=bitrate]:checked').val();
  
  }
  
  
  
  function readInputFile(file) {
  
  
  
  
  
   // load file content
  
   var reader = new FileReader();
  
   reader.onload = function(e) {
  
  
  
       fileName = file.name;
  
       console.log(fileName);
  
       fileBuffer = e.target.result;
  
   }
  
   reader.readAsArrayBuffer(file);
  
  
  
  }
  
  
  
  
  
  
  
  function handleFileSelect(event) {
  
   var files = event.target.files; // FileList object
  
   console.log(files);
  
   // files is a FileList of File objects. display first file name
  
   file = files[0];
  
   console.log(file);
  
   if (file) {
  
  
  
       readInputFile(file);
  
       console.log(file);
  
  
  
   }
  
   } 
  
  
  
   // setup input file listeners
  
   el=document.getElementById('inFile2');
  
  
  
   el.addEventListener('change',handleFileSelect, true);
  
  
  
   }

  note that this code was taken from a demo where the user upload the file to be converted by browisng it, I just want to cancel the browsing process and upload the saved file directly