Recherche avancée

Sur d’autres sites (2425)

• Low latency video streaming on android

  17 mai 2021, par Louis Blenner

  I'd like to be able to stream the video from my webcam to an Android app with a latency below 500ms, on my local network.

  


  To capture and send the video over the network, I use ffmpeg.

  


  ffmpeg -f v4l2 -i /dev/video0 -preset ultrafast -tune zerolatency -vcodec libx264 -an -vf format=yuv420p -f mpegts  udp://192.168.1.155:5000


  


  This command takes the webcam as an input, convert it and send it to a device using the mpegts protocol.

  


  I am able to read the video on another PC with a latency below 500 ms, using commands like

  


  gst-launch-1.0 -v udpsrc port=5000 ! video/mpegts ! tsdemux ! h264parse ! avdec_h264 ! fpsdisplaysink sync=false


  


  or

  


  mpv udp://0.0.0.0:5000 --no-cache --untimed --no-demuxer-thread --video-sync=audio --vd-lavc-threads=1 


  


  So it is possible to have this range of latency.
  
I'd like to have the same thing on Android.

  


  Here are my tries to do that.

  


  Exoplayer

  


  After looking at the different players available on Android studio, it seems like Exoplayer is the go-to choice.
  
I tried different options indicated in the live-streaming documentation, but I always end up with a stream taking seconds to start and with a latency of seconds.
  
I tried to add a Button to seek to the default position of the windows, but it results in a loading of several seconds.

  


  DefaultExtractorsFactory extractorsFactory =
                new DefaultExtractorsFactory()
                        .setTsExtractorFlags(DefaultTsPayloadReaderFactory.FLAG_IGNORE_AAC_STREAM);

        player = new SimpleExoPlayer.Builder(this)
                .setMediaSourceFactory(
                        new DefaultMediaSourceFactory(this, extractorsFactory))
                .setLoadControl(new DefaultLoadControl.Builder()
                        .setBufferDurationsMs(DefaultLoadControl.DEFAULT_MIN_BUFFER_MS, DefaultLoadControl.DEFAULT_MAX_BUFFER_MS, 200, 200)
                        .build())
                .build();
        MyPlayerView playerView = findViewById(R.id.player_view);
        // Bind the player to the view.
        playerView.setPlayer(player);
        // Build the media item.
        MediaItem mediaItem = new MediaItem.Builder()
                .setUri(Uri.parse("udp://0.0.0.0:5000"))
                .setLiveMaxOffsetMs(500)
                .setLiveTargetOffsetMs(0)
                .setLiveMinOffsetMs(0)
                .build();
        // Set the media item to be played.
        player.setMediaItem(mediaItem);
        // Prepare the player.
        player.setPlayWhenReady(true);
        player.prepare();
        //player.seekToDefaultPosition();


  


  This issue is about the same issue and the conclusion was that Exoplayer was not fit for this use case.

  


  


  I'll be honest, ultra low-latency like this isn't ExoPlayer's main use-case

  


  


  Vlc

  


  Another try was to use the Vlc library.
  
But I was unable to have the same low latency stream as with the two previous example with Vlc.
  
I tried changing the preferences of Vlc to stream as fast as possible.

  


  Input/Codecs -> x264 preset: ultrafast - zerolatency
Input/Codecs -> Access Module: UDP input
Input/Codecs -> Clock Jitter: 500
Audio: disable audio


  


  I also tried reducing the different buffers.
  
However, I still have a latency of more than 1 seconds with that.

  


  Gstreamer

  


  Another try was to create a react-native project to use the different players available here.
  
One player that seemed promising was react-native-gstreamer because it uses gstreamer which is able to stream with low latency (gst-launch command).
  
But the library is now outdated.

  


  Question

  


  There were other tries, but none were successful.
  
Is there a problem with one of my approaches ?
  
And if not, Is there a player on Android (that I missed) which is able to achieve low latency stream like gstream or mpv on linux ?

  


• ffplay 461 Unsupported transport

  28 octobre 2017, par Cerato

  Please help me to understand what I am doing wrong.

  I am executing the following command for making a live stream

  Executing: powershell gst-launch-1.0 filesrc location=D:/testVODs/sample.mkv ! matroskademux name=dmx dmx. ! h264parse config-interval=1 ! queue ! rtspclientsink profiles=avpf protocols=udp location="rtsp://192.168.1.1:8554/8c83bea4-b49b-4776-9c0f-d11e9e8898f0/1" name=rtsp_out dmx. ! opusparse ! queue ! rtsp_out.

  and in the result, I see that the execution ended after 00:01:21 which means that the stream is ended successfully after my video ended :

  Setting pipeline to PAUSED ...
  
  Pipeline is PREROLLING ...
  
  Progress: (open) Opening Stream
  
  Progress: (connect) Connecting to rtsp://192.168.1.1:8554/8c83bea4-b49b-4776-9c0f-d11e9e8898f0/1
  
  Progress: (open) Retrieving server options
  
  Progress: (open) Opened Stream
  
  Pipeline is PREROLLED ...
  
  Setting pipeline to PLAYING ...
  
  New clock: GstSystemClock
  
  Progress: (request) Sending RECORD request
  
  Progress: (record) Sending server stream info
  
  Progress: (request) Sending RECORD request
  
  Progress: (request) SETUP stream 1
  
  Progress: (request) SETUP stream 0
  
  Progress: (record) Starting recording
  
  Got EOS from element "pipeline0".
  
  Execution ended after 0:01:21.318315052
  
  Setting pipeline to PAUSED ...
  
  Setting pipeline to READY ...
  
  Setting pipeline to NULL ...
  
  Freeing pipeline ...

  at the same time I am trying to play the stream using the following request :

  Executing: powershell ffplay -rtsp_transport http "rtsp://192.168.1.1:8554/live/8c83bea4-b49b-4776-9c0f-d11e9e8898f0/1"

  but I have the following response :

  ffplay started on 2017-10-27 at 18:52:13
  
  Report written to "D:/testVODs/Stream1to.log"
  
  ffplay version 3.4 Copyright (c) 2003-2017 the FFmpeg developers
  
    built with gcc 7.2.0 (GCC)
  
    configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  
    libavutil      55. 78.100 / 55. 78.100
  
    libavcodec     57.107.100 / 57.107.100
  
    libavformat    57. 83.100 / 57. 83.100
  
    libavdevice    57. 10.100 / 57. 10.100
  
    libavfilter     6.107.100 /  6.107.100
  
    libswscale      4.  8.100 /  4.  8.100
  
    libswresample   2.  9.100 /  2.  9.100
  
    libpostproc    54.  7.100 / 54.  7.100
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
      nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
  
  [rtsp @ 000001930a05e500] method SETUP failed: 461 Unsupported transport
  
  rtsp://192.168.1.1:8554/live/8c83bea4-b49b-4776-9c0f-d11e9e8898f0/1: Unknown error

  why method SETUP failed as 461 Unsupported transport and how it can be solved ?

  I tried to use different ffplay, ffmpeg commands, but all of them lead to the same result.
  If I do not use rtsp_trasport command I have the error [udp @ 000001d1ab52c300] ’circular_buffer_size’ option was set but it is not supported on this build (pthread support is required)
  and the same 461 Unsupported transport

  Before I added "profiles=avpf" to my stream creation request I managed to play it successfully with "rtsp_transport tcp".
  But now profiles=avpf is required and I need to understand how to deal with it

• Decoding and playing audio with ffmpeg and XAudio2 - frequency raito wrong

  12 juillet 2016, par Brent de Carteret

  I’m using ffmpeg to decode audio and output it using the XAudio2 API, it works and plays synced with the video output using the pts. But it’s high pitched (i.e. sounds like chipmunks).

  Setting breakpoints I can see it has sets the correct sample rate from the audio codec in CreateSourceVoice. I’m stumped.

  Any help would be much appreciated.

  #include "DVDAudioDevice.h"
  
  
  
  HANDLE m_hBufferEndEvent;
  
  
  
  CDVDAudio::CDVDAudio()
  
  {
  
      m_pXAudio2 = NULL;
  
      m_pMasteringVoice = NULL;
  
      m_pSourceVoice = NULL;
  
      m_pWfx  = NULL;
  
  
  
      m_VoiceCallback = NULL;
  
  
  
      m_hBufferEndEvent = CreateEvent(NULL, false, false, "Buffer end event");
  
  }
  
  
  
  CDVDAudio::~CDVDAudio()
  
  {
  
      m_pXAudio2 = NULL;
  
      m_pMasteringVoice = NULL;
  
      m_pSourceVoice = NULL;
  
      m_pWfx  = NULL;
  
  
  
      m_VoiceCallback = NULL;
  
  
  
      CloseHandle(m_hBufferEndEvent);
  
      m_hBufferEndEvent = NULL;
  
  }
  
  
  
  bool CDVDAudio::Create(int iChannels, int iBitrate, int iBitsPerSample, bool bPasstrough)
  
  {
  
      CoInitializeEx(NULL, COINIT_MULTITHREADED);
  
  
  
      HRESULT hr = XAudio2Create( &m_pXAudio2, 0, XAUDIO2_DEFAULT_PROCESSOR);
  
  
  
      if (SUCCEEDED(hr))
  
      {
  
          m_pXAudio2->CreateMasteringVoice( &m_pMasteringVoice );
  
      }
  
  
  
      // Create source voice
  
      WAVEFORMATEXTENSIBLE wfx;
  
      memset(&wfx, 0, sizeof(WAVEFORMATEXTENSIBLE));
  
  
  
      wfx.Format.wFormatTag           = WAVE_FORMAT_PCM;
  
      wfx.Format.nSamplesPerSec       = iBitrate;//pFFMpegData->pAudioCodecCtx->sample_rate;//48000 by default
  
      wfx.Format.nChannels            = iChannels;//pFFMpegData->pAudioCodecCtx->channels;
  
      wfx.Format.wBitsPerSample       = 16;
  
      wfx.Format.nBlockAlign          = wfx.Format.nChannels*16/8;
  
      wfx.Format.nAvgBytesPerSec      = wfx.Format.nSamplesPerSec * wfx.Format.nBlockAlign;
  
      wfx.Format.cbSize               = sizeof(WAVEFORMATEXTENSIBLE)-sizeof(WAVEFORMATEX);
  
      wfx.Samples.wValidBitsPerSample = wfx.Format.wBitsPerSample;
  
  
  
      if(wfx.Format.nChannels == 1)
  
      {
  
          wfx.dwChannelMask = SPEAKER_MONO;
  
      }
  
      else if(wfx.Format.nChannels == 2)
  
      {
  
          wfx.dwChannelMask = SPEAKER_STEREO;
  
      }
  
      else if(wfx.Format.nChannels == 5)
  
      {
  
          wfx.dwChannelMask = SPEAKER_5POINT1;
  
      }
  
  
  
      wfx.SubFormat = KSDATAFORMAT_SUBTYPE_PCM;
  
  
  
      unsigned int flags = 0;//XAUDIO2_VOICE_NOSRC;// | XAUDIO2_VOICE_NOPITCH;
  
  
  
      //Source voice
  
  
  
      m_VoiceCallback = new StreamingVoiceCallback(this);
  
  
  
      hr = m_pXAudio2->CreateSourceVoice(&m_pSourceVoice,(WAVEFORMATEX*)&wfx, 0 , 1.0f, m_VoiceCallback);
  
  
  
      if(!SUCCEEDED(hr))
  
          return false;
  
  
  
      // Start sound
  
      hr = m_pSourceVoice->Start(0);
  
  
  
      if(!SUCCEEDED(hr))
  
          return false;
  
  
  
      return true;
  
  }
  
  
  
  DWORD CDVDAudio::AddPackets(unsigned char* data, DWORD len)
  
  {  
  
          memset(&m_SoundBuffer,0,sizeof(XAUDIO2_BUFFER));
  
  
  
          m_SoundBuffer.AudioBytes = len;
  
          m_SoundBuffer.pAudioData = data;
  
          m_SoundBuffer.pContext = NULL;//(VOID*)data;
  
  
  
          XAUDIO2_VOICE_STATE state;
  
  
  
          while(m_pSourceVoice->GetState( &state ), state.BuffersQueued > 60)
  
          {
  
              WaitForSingleObject( m_hBufferEndEvent, INFINITE );
  
          }
  
  
  
          m_pSourceVoice->SubmitSourceBuffer( &m_SoundBuffer );
  
  
  
      return 0;
  
  }
  
  
  
  void CDVDAudio::Destroy()
  
  {
  
      m_pMasteringVoice->DestroyVoice();
  
      m_pXAudio2->Release();
  
  
  
      m_pSourceVoice->DestroyVoice();
  
  
  
      delete m_VoiceCallback;
  
      m_VoiceCallback = NULL;
  
  }

  #include "DVDAudioCodecFFmpeg.h"
  
  #include "Log.h"
  
  
  
  CDVDAudioCodecFFmpeg::CDVDAudioCodecFFmpeg() : CDVDAudioCodec()
  
  {
  
      m_iBufferSize = 0;
  
      m_pCodecContext = NULL;
  
      m_bOpenedCodec = false;
  
  }
  
  
  
  CDVDAudioCodecFFmpeg::~CDVDAudioCodecFFmpeg()
  
  {
  
      Dispose();
  
  }
  
  
  
  bool CDVDAudioCodecFFmpeg::Open(AVCodecID codecID, int iChannels, int iSampleRate)
  
  {
  
      AVCodec* pCodec;
  
      m_bOpenedCodec = false;
  
  
  
      av_register_all();
  
  
  
      pCodec = avcodec_find_decoder(codecID);
  
  
  
      m_pCodecContext = avcodec_alloc_context3(pCodec);//avcodec_alloc_context();
  
      avcodec_get_context_defaults3(m_pCodecContext, pCodec);
  
  
  
      if (!pCodec)
  
      {
  
          CLog::Log(LOGERROR, "CDVDAudioCodecFFmpeg::Open() Unable to find codec");
  
          return false;
  
      }
  
  
  
      m_pCodecContext->debug_mv = 0;
  
      m_pCodecContext->debug = 0;
  
      m_pCodecContext->workaround_bugs = 1;
  
  
  
      if (pCodec->capabilities & CODEC_CAP_TRUNCATED)
  
          m_pCodecContext->flags |= CODEC_FLAG_TRUNCATED;
  
  
  
      m_pCodecContext->channels = iChannels;
  
      m_pCodecContext->sample_rate = iSampleRate;
  
      //m_pCodecContext->bits_per_sample = 24;
  
  
  
  /* //FIXME BRENT
  
    if( ExtraData && ExtraSize > 0 )
  
    {
  
      m_pCodecContext->extradata_size = ExtraSize;
  
      m_pCodecContext->extradata = m_dllAvCodec.av_mallocz(ExtraSize + FF_INPUT_BUFFER_PADDING_SIZE);
  
      memcpy(m_pCodecContext->extradata, ExtraData, ExtraSize);
  
    }
  
  */
  
      // set acceleration
  
      //m_pCodecContext->dsp_mask = FF_MM_FORCE | FF_MM_MMX | FF_MM_MMXEXT | FF_MM_SSE; //BRENT
  
  
  
      if (avcodec_open2(m_pCodecContext, pCodec, NULL) < 0)
  
      {
  
          CLog::Log(LOGERROR, "CDVDAudioCodecFFmpeg::Open() Unable to open codec");
  
          Dispose();
  
          return false;
  
      }
  
  
  
      m_bOpenedCodec = true;
  
      return true;
  
  }
  
  
  
  void CDVDAudioCodecFFmpeg::Dispose()
  
  {
  
      if (m_pCodecContext)
  
      {
  
          if (m_bOpenedCodec) avcodec_close(m_pCodecContext);
  
          m_bOpenedCodec = false;
  
          av_free(m_pCodecContext);
  
          m_pCodecContext = NULL;
  
      }
  
      m_iBufferSize = 0;
  
  }
  
  
  
  int CDVDAudioCodecFFmpeg::Decode(BYTE* pData, int iSize)
  
  {
  
      int iBytesUsed;
  
      if (!m_pCodecContext) return -1;
  
  
  
      //Copy into a FFMpeg AVPAcket again
  
      AVPacket packet;
  
      av_init_packet(&packet);
  
  
  
      packet.data=pData;
  
      packet.size=iSize;
  
  
  
      int iOutputSize = AVCODEC_MAX_AUDIO_FRAME_SIZE; //BRENT
  
  
  
      iBytesUsed = avcodec_decode_audio3(m_pCodecContext, (int16_t *)m_buffer, &iOutputSize/*m_iBufferSize*/, &packet);
  
  
  
      m_iBufferSize = iOutputSize;//BRENT
  
  
  
      return iBytesUsed;
  
  }
  
  
  
  int CDVDAudioCodecFFmpeg::GetData(BYTE** dst)
  
  {
  
      *dst = m_buffer;
  
      return m_iBufferSize;
  
  }
  
  
  
  void CDVDAudioCodecFFmpeg::Reset()
  
  {
  
      if (m_pCodecContext) avcodec_flush_buffers(m_pCodecContext);
  
  }
  
  
  
  int CDVDAudioCodecFFmpeg::GetChannels()
  
  {
  
      if (m_pCodecContext) return m_pCodecContext->channels;
  
      return 0;
  
  }
  
  
  
  int CDVDAudioCodecFFmpeg::GetSampleRate()
  
  {
  
      if (m_pCodecContext) return m_pCodecContext->sample_rate;
  
      return 0;
  
  }
  
  
  
  int CDVDAudioCodecFFmpeg::GetBitsPerSample()
  
  {
  
      if (m_pCodecContext) return 16;
  
      return 0;
  
  }
  #include "DVDPlayerAudio.h"
  
  #include "DVDDemuxUtils.h"
  
  #include "Log.h"
  
  
  
  #include 
  
  #include "DVDAudioCodecFFmpeg.h" //FIXME Move to a codec factory!!
  
  
  
  CDVDPlayerAudio::CDVDPlayerAudio(CDVDClock* pClock) : CThread()
  
  {
  
      m_pClock = pClock;
  
      m_pAudioCodec = NULL;
  
      m_bInitializedOutputDevice = false;
  
      m_iSourceChannels = 0;
  
      m_audioClock = 0;
  
  
  
  //  m_currentPTSItem.pts = DVD_NOPTS_VALUE;
  
  //  m_currentPTSItem.timestamp = 0;
  
  
  
      SetSpeed(DVD_PLAYSPEED_NORMAL);
  
  
  
      InitializeCriticalSection(&m_critCodecSection);
  
      m_messageQueue.SetMaxDataSize(10 * 16 * 1024);
  
  //  g_dvdPerformanceCounter.EnableAudioQueue(&m_packetQueue);
  
  }
  
  
  
  CDVDPlayerAudio::~CDVDPlayerAudio()
  
  {
  
  //  g_dvdPerformanceCounter.DisableAudioQueue();
  
  
  
      // close the stream, and don't wait for the audio to be finished
  
      CloseStream(true);
  
      DeleteCriticalSection(&m_critCodecSection);
  
  }
  
  
  
  bool CDVDPlayerAudio::OpenStream( CDemuxStreamAudio *pDemuxStream )                                 
  
  {
  
      // should alway's be NULL!!!!, it will probably crash anyway when deleting m_pAudioCodec here.
  
      if (m_pAudioCodec)
  
      {
  
          CLog::Log(LOGFATAL, "CDVDPlayerAudio::OpenStream() m_pAudioCodec != NULL");
  
          return false;
  
      }
  
  
  
      AVCodecID codecID = pDemuxStream->codec;
  
  
  
      CLog::Log(LOGNOTICE, "Finding audio codec for: %i", codecID);
  
      //m_pAudioCodec = CDVDFactoryCodec::CreateAudioCodec( pDemuxStream ); 
  
      m_pAudioCodec = new CDVDAudioCodecFFmpeg; //FIXME BRENT Codec Factory needed!
  
  
  
      if (!m_pAudioCodec->Open(pDemuxStream->codec, pDemuxStream->iChannels, pDemuxStream->iSampleRate))
  
      {
  
          m_pAudioCodec->Dispose();
  
          delete m_pAudioCodec;
  
          m_pAudioCodec = NULL;
  
          return false;
  
      }
  
  
  
      if( !m_pAudioCodec )
  
      {
  
          CLog::Log(LOGERROR, "Unsupported audio codec");
  
          return false;
  
      }
  
  
  
      m_codec = pDemuxStream->codec;
  
      m_iSourceChannels = pDemuxStream->iChannels;
  
  
  
      m_messageQueue.Init();
  
  
  
      CLog::Log(LOGNOTICE, "Creating audio thread");
  
      Create();
  
  
  
      return true;
  
  }
  
  
  
  void CDVDPlayerAudio::CloseStream(bool bWaitForBuffers)
  
  {
  
      // wait until buffers are empty
  
      if (bWaitForBuffers) m_messageQueue.WaitUntilEmpty();
  
  
  
      // send abort message to the audio queue
  
      m_messageQueue.Abort();
  
  
  
      CLog::Log(LOGNOTICE, "waiting for audio thread to exit");
  
  
  
      // shut down the adio_decode thread and wait for it
  
      StopThread(); // will set this->m_bStop to true
  
      this->WaitForThreadExit(INFINITE);
  
  
  
      // uninit queue
  
      m_messageQueue.End();
  
  
  
      CLog::Log(LOGNOTICE, "Deleting audio codec");
  
      if (m_pAudioCodec)
  
      {
  
          m_pAudioCodec->Dispose();
  
          delete m_pAudioCodec;
  
          m_pAudioCodec = NULL;
  
      }
  
  
  
      // flush any remaining pts values
  
      //FlushPTSQueue(); //FIXME BRENT
  
  }
  
  
  
  void CDVDPlayerAudio::OnStartup()
  
  {
  
      CThread::SetName("CDVDPlayerAudio");
  
      pAudioPacket = NULL;
  
      m_audioClock = 0;
  
      audio_pkt_data = NULL;
  
      audio_pkt_size = 0;
  
  
  
  //  g_dvdPerformanceCounter.EnableAudioDecodePerformance(ThreadHandle());
  
  }
  
  
  
  void CDVDPlayerAudio::Process()
  
  {
  
      CLog::Log(LOGNOTICE, "running thread: CDVDPlayerAudio::Process()");
  
  
  
      int result;
  
  
  
      // silence data
  
      BYTE silence[1024];
  
      memset(silence, 0, 1024);
  
  
  
      DVDAudioFrame audioframe;
  
  
  
      __int64 iClockDiff=0;
  
      while (!m_bStop)
  
      {
  
          //Don't let anybody mess with our global variables
  
          EnterCriticalSection(&m_critCodecSection);
  
          result = DecodeFrame(audioframe, m_speed != DVD_PLAYSPEED_NORMAL); // blocks if no audio is available, but leaves critical section before doing so
  
          LeaveCriticalSection(&m_critCodecSection);
  
  
  
          if( result & DECODE_FLAG_ERROR ) 
  
          {      
  
              CLog::Log(LOGERROR, "CDVDPlayerAudio::Process - Decode Error. Skipping audio frame");
  
              continue;
  
          }
  
  
  
          if( result & DECODE_FLAG_ABORT )
  
          {
  
              CLog::Log(LOGDEBUG, "CDVDPlayerAudio::Process - Abort recieved, exiting thread");
  
              break;
  
          }
  
  
  
          if( result & DECODE_FLAG_DROP ) //FIXME BRENT
  
          {
  
  /*          //frame should be dropped. Don't let audio move ahead of the current time thou
  
              //we need to be able to start playing at any time
  
              //when playing backwords, we try to keep as small buffers as possible
  
  
  
              // set the time at this delay
  
              AddPTSQueue(audioframe.pts, m_dvdAudio.GetDelay());
  
  */
  
              if (m_speed > 0)
  
              {
  
                  __int64 timestamp = m_pClock->GetAbsoluteClock() + (audioframe.duration * DVD_PLAYSPEED_NORMAL) / m_speed;
  
                  while( !m_bStop && timestamp > m_pClock->GetAbsoluteClock() ) Sleep(1);
  
              }
  
              continue;
  
          }
  
  
  
          if( audioframe.size > 0 ) 
  
          {
  
              // we have succesfully decoded an audio frame, openup the audio device if not already done
  
              if (!m_bInitializedOutputDevice)
  
              {
  
                  m_bInitializedOutputDevice = InitializeOutputDevice();
  
              }
  
  
  
              //Add any packets play
  
              m_dvdAudio.AddPackets(audioframe.data, audioframe.size);
  
  
  
              // store the delay for this pts value so we can calculate the current playing
  
              //AddPTSQueue(audioframe.pts, m_dvdAudio.GetDelay() - audioframe.duration);//BRENT
  
          }
  
  
  
          // if we where asked to resync on this packet, do so here
  
          if( result & DECODE_FLAG_RESYNC )
  
          {      
  
              CLog::Log(LOGDEBUG, "CDVDPlayerAudio::Process - Resync recieved.");
  
              //while (!m_bStop && (unsigned int)m_dvdAudio.GetDelay() > audioframe.duration ) Sleep(5); //BRENT
  
              m_pClock->Discontinuity(CLOCK_DISC_NORMAL, audioframe.pts);
  
          }
  
  
  
  #ifdef USEOLDSYNC
  
          //Clock should be calculated after packets have been added as m_audioClock points to the 
  
          //time after they have been played
  
  
  
          const __int64 iCurrDiff = (m_audioClock - m_dvdAudio.GetDelay()) - m_pClock->GetClock();
  
          const __int64 iAvDiff = (iClockDiff + iCurrDiff)/2;
  
  
  
          //Check for discontinuity in the stream, use a moving average to
  
          //eliminate highfreq fluctuations of large packet sizes
  
          if( ABS(iAvDiff) > 5000 ) // sync clock if average diff is bigger than 5 msec 
  
          {
  
              //Wait untill only the new audio frame wich triggered the discontinuity is left
  
              //then set disc state
  
              while (!m_bStop && (unsigned int)m_dvdAudio.GetBytesInBuffer() > audioframe.size ) Sleep(5);
  
  
  
              m_pClock->Discontinuity(CLOCK_DISC_NORMAL, m_audioClock - m_dvdAudio.GetDelay());
  
              CLog::("CDVDPlayer:: Detected Audio Discontinuity, syncing clock. diff was: %I64d, %I64d, av: %I64d", iClockDiff, iCurrDiff, iAvDiff);
  
              iClockDiff = 0;
  
          }
  
          else
  
          {      
  
              //Do gradual adjustments (not working yet)
  
              //m_pClock->AdjustSpeedToMatch(iClock + iAvDiff);
  
              iClockDiff = iCurrDiff;
  
          }
  
  #endif
  
      }
  
  }
  
  
  
  void CDVDPlayerAudio::OnExit()
  
  {
  
      //g_dvdPerformanceCounter.DisableAudioDecodePerformance();
  
  
  
      // destroy audio device
  
      CLog::Log(LOGNOTICE, "Closing audio device");
  
      m_dvdAudio.Destroy();
  
      m_bInitializedOutputDevice = false;
  
  
  
      CLog::Log(LOGNOTICE, "thread end: CDVDPlayerAudio::OnExit()");
  
  }
  
  
  
  // decode one audio frame and returns its uncompressed size
  
  int CDVDPlayerAudio::DecodeFrame(DVDAudioFrame &audioframe, bool bDropPacket)
  
  {
  
      CDVDDemux::DemuxPacket* pPacket = pAudioPacket;
  
      int n=48000*2*16/8, len;
  
  
  
      //Store amount left at this point, and what last pts was
  
      unsigned __int64 first_pkt_pts = 0;
  
      int first_pkt_size = 0; 
  
      int first_pkt_used = 0;
  
      int result = 0;
  
  
  
      // make sure the sent frame is clean
  
      memset(&audioframe, 0, sizeof(DVDAudioFrame));
  
  
  
      if (pPacket)
  
      {
  
          first_pkt_pts = pPacket->pts;
  
          first_pkt_size = pPacket->iSize;
  
          first_pkt_used = first_pkt_size - audio_pkt_size;
  
      }
  
  
  
      for (;;)
  
      {
  
          /* NOTE: the audio packet can contain several frames */
  
          while (audio_pkt_size > 0)
  
          {
  
              len = m_pAudioCodec->Decode(audio_pkt_data, audio_pkt_size);
  
              if (len < 0)
  
              {
  
                  /* if error, we skip the frame */
  
                  audio_pkt_size=0;
  
                  m_pAudioCodec->Reset();
  
                  break;
  
              }
  
  
  
              // fix for fucked up decoders //FIXME BRENT
  
              if( len > audio_pkt_size )
  
              {        
  
                  CLog::Log(LOGERROR, "CDVDPlayerAudio:DecodeFrame - Codec tried to consume more data than available. Potential memory corruption");        
  
                  audio_pkt_size=0;
  
                  m_pAudioCodec->Reset();
  
                  assert(0);
  
              }
  
  
  
              // get decoded data and the size of it
  
              audioframe.size = m_pAudioCodec->GetData(&audioframe.data);
  
  
  
              audio_pkt_data += len;
  
              audio_pkt_size -= len;
  
  
  
              if (audioframe.size <= 0) continue;
  
  
  
              audioframe.pts = m_audioClock;
  
  
  
              // compute duration.
  
              n = m_pAudioCodec->GetChannels() * m_pAudioCodec->GetBitsPerSample() / 8 * m_pAudioCodec->GetSampleRate();
  
              if (n > 0)
  
              {
  
                  // safety check, if channels == 0, n will result in 0, and that will result in a nice devide exception
  
                  audioframe.duration = (unsigned int)(((__int64)audioframe.size * DVD_TIME_BASE) / n);
  
  
  
                  // increase audioclock to after the packet
  
                  m_audioClock += audioframe.duration;
  
              }
  
  
  
              //If we are asked to drop this packet, return a size of zero. then it won't be played
  
              //we currently still decode the audio.. this is needed since we still need to know it's 
  
              //duration to make sure clock is updated correctly.
  
              if( bDropPacket )
  
              {
  
                  result |= DECODE_FLAG_DROP;
  
              }
  
              return result;
  
          }
  
  
  
          // free the current packet
  
          if (pPacket)
  
          {
  
              CDVDDemuxUtils::FreeDemuxPacket(pPacket); //BRENT FIXME
  
              pPacket = NULL;
  
              pAudioPacket = NULL;
  
          }
  
  
  
          if (m_messageQueue.RecievedAbortRequest()) return DECODE_FLAG_ABORT;
  
  
  
          // read next packet and return -1 on error
  
          LeaveCriticalSection(&m_critCodecSection); //Leave here as this might stall a while
  
  
  
          CDVDMsg* pMsg;
  
          MsgQueueReturnCode ret = m_messageQueue.Get(&pMsg, INFINITE);
  
  
  
          EnterCriticalSection(&m_critCodecSection);
  
          if (MSGQ_IS_ERROR(ret) || ret == MSGQ_ABORT) return DECODE_FLAG_ABORT;
  
  
  
          if (pMsg->IsType(CDVDMsg::DEMUXER_PACKET))
  
          {
  
              CDVDMsgDemuxerPacket* pMsgDemuxerPacket = (CDVDMsgDemuxerPacket*)pMsg;
  
              pPacket = pMsgDemuxerPacket->GetPacket();
  
              pMsgDemuxerPacket->m_pPacket = NULL; // XXX, test
  
              pAudioPacket = pPacket;
  
              audio_pkt_data = pPacket->pData;
  
              audio_pkt_size = pPacket->iSize;
  
          }
  
          else
  
          {
  
              // other data is not used here, free if
  
              // msg itself will still be available
  
              pMsg->Release();
  
          }
  
  
  
          // if update the audio clock with the pts
  
          if (pMsg->IsType(CDVDMsg::DEMUXER_PACKET) || pMsg->IsType(CDVDMsg::GENERAL_RESYNC))
  
          {
  
              if (pMsg->IsType(CDVDMsg::GENERAL_RESYNC))
  
              { 
  
                  //player asked us to sync on this package
  
                  CDVDMsgGeneralResync* pMsgGeneralResync = (CDVDMsgGeneralResync*)pMsg;
  
                  result |= DECODE_FLAG_RESYNC;
  
                  m_audioClock = pMsgGeneralResync->GetPts();
  
              }
  
              else if (pPacket->pts != DVD_NOPTS_VALUE) // CDVDMsg::DEMUXER_PACKET, pPacket is already set above
  
              {
  
                  if (first_pkt_size == 0) 
  
                  { 
  
                      //first package
  
                      m_audioClock = pPacket->pts;        
  
                  }
  
                  else if (first_pkt_pts > pPacket->pts)
  
                  { 
  
                      //okey first packet in this continous stream, make sure we use the time here        
  
                      m_audioClock = pPacket->pts;        
  
                  }
  
                  else if((unsigned __int64)m_audioClock < pPacket->pts || (unsigned __int64)m_audioClock > pPacket->pts)
  
                  {
  
                      //crap, moved outsided correct pts
  
                      //Use pts from current packet, untill we find a better value for it.
  
                      //Should be ok after a couple of frames, as soon as it starts clean on a packet
  
                      m_audioClock = pPacket->pts;
  
                  }
  
                  else if(first_pkt_size == first_pkt_used)
  
                  {
  
                      //Nice starting up freshly on the start of a packet, use pts from it
  
                      m_audioClock = pPacket->pts;
  
                  }
  
              }
  
          }
  
          pMsg->Release();
  
      }
  
  }
  
  
  
  void CDVDPlayerAudio::SetSpeed(int speed)
  
  { 
  
      m_speed = speed;
  
  
  
      //if (m_speed == DVD_PLAYSPEED_PAUSE) m_dvdAudio.Pause(); //BRENT FIXME
  
      //else m_dvdAudio.Resume();
  
  }
  
  
  
  bool CDVDPlayerAudio::InitializeOutputDevice()
  
  {
  
      int iChannels = m_pAudioCodec->GetChannels();
  
      int iSampleRate = m_pAudioCodec->GetSampleRate();
  
      int iBitsPerSample = m_pAudioCodec->GetBitsPerSample();
  
      //bool bPasstrough = m_pAudioCodec->NeedPasstrough(); //BRENT
  
  
  
      if (iChannels == 0 || iSampleRate == 0 || iBitsPerSample == 0)
  
      {
  
          CLog::Log(LOGERROR, "Unable to create audio device, (iChannels == 0 || iSampleRate == 0 || iBitsPerSample == 0)");
  
          return false;
  
      }
  
  
  
      CLog::Log(LOGNOTICE, "Creating audio device with codec id: %i, channels: %i, sample rate: %i", m_codec, iChannels, iSampleRate);
  
      if (m_dvdAudio.Create(iChannels, iSampleRate, iBitsPerSample, /*bPasstrough*/0)) // always 16 bit with ffmpeg ? //BRENT Passthrough needed?
  
      {
  
          return true;
  
      }
  
  
  
      CLog::Log(LOGERROR, "Failed Creating audio device with codec id: %i, channels: %i, sample rate: %i", m_codec, iChannels, iSampleRate);
  
      return false;
  
  }