Recherche avancée

Sur d’autres sites (4976)

• Low latency video streaming on android

  17 mai 2021, par Louis Blenner

  I'd like to be able to stream the video from my webcam to an Android app with a latency below 500ms, on my local network.

  


  To capture and send the video over the network, I use ffmpeg.

  


  ffmpeg -f v4l2 -i /dev/video0 -preset ultrafast -tune zerolatency -vcodec libx264 -an -vf format=yuv420p -f mpegts  udp://192.168.1.155:5000


  


  This command takes the webcam as an input, convert it and send it to a device using the mpegts protocol.

  


  I am able to read the video on another PC with a latency below 500 ms, using commands like

  


  gst-launch-1.0 -v udpsrc port=5000 ! video/mpegts ! tsdemux ! h264parse ! avdec_h264 ! fpsdisplaysink sync=false


  


  or

  


  mpv udp://0.0.0.0:5000 --no-cache --untimed --no-demuxer-thread --video-sync=audio --vd-lavc-threads=1 


  


  So it is possible to have this range of latency.
  
I'd like to have the same thing on Android.

  


  Here are my tries to do that.

  


  Exoplayer

  


  After looking at the different players available on Android studio, it seems like Exoplayer is the go-to choice.
  
I tried different options indicated in the live-streaming documentation, but I always end up with a stream taking seconds to start and with a latency of seconds.
  
I tried to add a Button to seek to the default position of the windows, but it results in a loading of several seconds.

  


  DefaultExtractorsFactory extractorsFactory =
                new DefaultExtractorsFactory()
                        .setTsExtractorFlags(DefaultTsPayloadReaderFactory.FLAG_IGNORE_AAC_STREAM);

        player = new SimpleExoPlayer.Builder(this)
                .setMediaSourceFactory(
                        new DefaultMediaSourceFactory(this, extractorsFactory))
                .setLoadControl(new DefaultLoadControl.Builder()
                        .setBufferDurationsMs(DefaultLoadControl.DEFAULT_MIN_BUFFER_MS, DefaultLoadControl.DEFAULT_MAX_BUFFER_MS, 200, 200)
                        .build())
                .build();
        MyPlayerView playerView = findViewById(R.id.player_view);
        // Bind the player to the view.
        playerView.setPlayer(player);
        // Build the media item.
        MediaItem mediaItem = new MediaItem.Builder()
                .setUri(Uri.parse("udp://0.0.0.0:5000"))
                .setLiveMaxOffsetMs(500)
                .setLiveTargetOffsetMs(0)
                .setLiveMinOffsetMs(0)
                .build();
        // Set the media item to be played.
        player.setMediaItem(mediaItem);
        // Prepare the player.
        player.setPlayWhenReady(true);
        player.prepare();
        //player.seekToDefaultPosition();


  


  This issue is about the same issue and the conclusion was that Exoplayer was not fit for this use case.

  


  


  I'll be honest, ultra low-latency like this isn't ExoPlayer's main use-case

  


  


  Vlc

  


  Another try was to use the Vlc library.
  
But I was unable to have the same low latency stream as with the two previous example with Vlc.
  
I tried changing the preferences of Vlc to stream as fast as possible.

  


  Input/Codecs -> x264 preset: ultrafast - zerolatency
Input/Codecs -> Access Module: UDP input
Input/Codecs -> Clock Jitter: 500
Audio: disable audio


  


  I also tried reducing the different buffers.
  
However, I still have a latency of more than 1 seconds with that.

  


  Gstreamer

  


  Another try was to create a react-native project to use the different players available here.
  
One player that seemed promising was react-native-gstreamer because it uses gstreamer which is able to stream with low latency (gst-launch command).
  
But the library is now outdated.

  


  Question

  


  There were other tries, but none were successful.
  
Is there a problem with one of my approaches ?
  
And if not, Is there a player on Android (that I missed) which is able to achieve low latency stream like gstream or mpv on linux ?

  


• How to silent the MP3 decoding process

  3 août 2019, par Golu

  I am learning ffmpeg and I made a MP3 decoder but when I am executing it , some kind of information is printing on my terminal but I don’t want it. So how to silent it ?

  Here is code (full code)

  /* FFmpeg Usage Example
  
   * Date : 28 July 2019
  
   */
  
  #include 
  
  #include <libavformat></libavformat>avformat.h>
  
  #include <libavcodec></libavcodec>avcodec.h>
  
  #include <libavutil></libavutil>avutil.h>
  
  #include 
  
  
  
  int decode_packet(AVCodecContext*, AVPacket*, AVFrame*);
  
  
  
  int main(void) {
  
          AVFormatContext *pFormatContext = avformat_alloc_context();
  
          AVCodecParameters *pCodecParameters = NULL;
  
          if(avformat_open_input(&amp;pFormatContext,"song.mp3",NULL,NULL)!=0) {
  
                  fprintf(stderr,"Could not open file\n");
  
                  return -1;
  
          }
  
          if(avformat_find_stream_info(pFormatContext,NULL)&lt;0) {
  
                  fprintf(stderr,"Could not find stream\n");
  
                  return -1;
  
          }
  
  
  
          size_t stream_index = 0;
  
          for(;stream_indexnb_streams;stream_index++) {
  
                  if(pFormatContext->streams[stream_index]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {
  
                          pCodecParameters = pFormatContext->streams[stream_index]->codecpar;
  
                  }
  
                          break;
  
          }
  
  
  
          if(stream_index == -1) {
  
                  fprintf(stderr,"could not retrive stream info from file\n");
  
                  return -1;
  
          }
  
          AVStream *stream = pFormatContext->streams[stream_index];
  
          pCodecParameters = stream->codecpar;
  
          AVCodec *cdc = avcodec_find_decoder(pCodecParameters->codec_id);
  
          AVCodecContext *cdc_ctx = avcodec_alloc_context3(cdc);
  
          assert(pCodecParameters);
  
  
  
          if(avcodec_parameters_to_context(cdc_ctx,pCodecParameters) &lt; 0) {
  
                  fprintf(stderr,"Can't copy params to codec context\n");
  
                  return -1;
  
          }
  
  
  
          if(avcodec_open2(cdc_ctx,cdc,NULL) &lt; 0) {
  
                  fprintf(stderr,"Failed to open decoder for stream\n");
  
                  return -1;
  
          }
  
  
  
          AVFrame *frame = av_frame_alloc();
  
          if(!frame) {
  
                  fprintf(stderr,"could not allocate memory for frame\n");
  
                  return -1;
  
          }
  
  
  
          AVPacket *packet = av_packet_alloc();
  
  //      av_init_packet(&amp;packet);
  
          if(!packet) {
  
                  fprintf(stderr,"could not allocate memory for packet");
  
                  return -1;
  
          }
  
  
  
          packet->data=NULL;
  
          packet->size=0;
  
          // lets read the packets
  
          while(av_read_frame(pFormatContext,packet) >= 0) {
  
                  if(packet->stream_index==stream_index) {
  
                          int response = 0 ;
  
              response = decode_packet(cdc_ctx,packet,frame);
  
  
  
                          if(response &lt; 0)
  
                                  continue;
  
                  }
  
          av_packet_unref(packet);
  
          }
  
  
  
          return 0;
  
  }
  
  
  
  int decode_packet(AVCodecContext *cdc_ctx , AVPacket *pkt, AVFrame *frm) {
  
      int response = avcodec_send_packet(cdc_ctx,pkt);
  
      if(response &lt; 0)
  
          return response;
  
      while(response >= 0) {
  
          response = avcodec_receive_frame(cdc_ctx,frm);
  
          if(response == AVERROR(EAGAIN) || response == AVERROR_EOF)
  
              return -1;
  
          else if(response &lt; 0)
  
              return response;
  
      }
  
      return 0;
  
  }

  Expected behaviour : nothing should be printed on screen

  Actual behaviour : some kind of logs are printing automatically

  Here is output logs ( some of them )

  [mp3float @ 0x75172e7400] overread, skip -6 enddists: -5 -5         
  
  [mp3float @ 0x75172e7400] overread, skip -7 enddists: -6 -6
  
  [mp3float @ 0x75172e7400] overread, skip -6 enddists: -5 -5
  
  [mp3float @ 0x75172e7400] overread, skip -6 enddists: -4 -4

• ffmpeg/libav - how to wirte video files with valid pts

  9 mai 2017, par Kai Rohmer

  I’m currently trying the write out a real time rendered video into a h264 encoded file. After reading a lot of (mostly) old samples and the few class references they call a documentation, I manager to write my video file and I’m also able to read it. Unfortunately, I need some metadata for each frame but I’m not having a constant frame rate. So my intension was to start with the presentation timestamps to "frametime" during recoding. But after all I tried I get no pts while reading the the file (pts stays -9223372036854775808). Before wring a lot of code, here are the basics steps I’m doing. I’m probably using the wrong container or I’m missing to set a flag and you will notice it right away.

  // open a AVFormatContext
  
  avformat_alloc_output_context2(&m_FormatContext, nullptr, "avi", m_FileName.c_str());
  
  
  
  // open a stream
  
  m_VideoStream = avformat_new_stream(m_FormatContext, avcodec_find_encoder(AV_CODEC_ID_H264));
  
  
  
  // setup the codec context (including bitrate, frame size, ...)
  
  m_CodecContext = m_VideoStream ->codec;
  
  m_CodecContext->coder_type = FF_CODER_TYPE_VLC;
  
  m_CodecContext->time_base = AVRational{1, 120}; // I expect 20-60 Hz
  
  m_CodecContext->pix_fmt = AV_PIX_FMT_YUV420P;
  
  m_CodecContext->color_range = AVCOL_RANGE_JPEG;
  
  ...
  
  av_opt_set(m_CodecContext->priv_data, "preset", "ultrafast", 0);
  
  av_opt_set(m_CodecContext->priv_data, "tune", "zerolatency,fastdecode", 0);
  
  
  
  
  
  // set the same time_base to the stream
  
  m_VideoStream ->time_base = m_CodecContext->time_base;
  
  
  
  // open the codec
  
  avcodec_open2(m_CodecContext, m_CodecContext->codec, nullptr);
  
  
  
  // open file and write header
  
  avio_open(&m_FormatContext->pb, m_FileName.c_str(), AVIO_FLAG_WRITE);
  
  avformat_write_header(m_FormatContext, nullptr);
  
  
  
  // then in a loop:
  
  // render frame, convert RGBA to YUV frame, set the frames pts (timestamp is double application time in seconds)
  
  frameToEncode.pts = int64_t(timestamp / av_q2d(m_VideoStream->time_base));
  
  av_init_packet(m_EncodedPacket);
  
  avcodec_encode_video2(m_CodecContext, m_EncodedPacket, frameToEncode, &got_output);
  
  
  
  // check packet infos
  
  //m_EncodedPacket->pts equals frameToEncode.pts
  
  m_EncodedPacket->dts = AV_NOPTS_VALUE; // also tried incrementing numbers, or zero
  
  m_EncodedPacket->stream_index = m_Stream->index;
  
  m_EncodedPacket->duration = 0;
  
  m_EncodedPacket->pos = -1;
  
  m_EncodedPacket->flags = 0;
  
  m_EncodedPacket->flags |= AV_PKT_FLAG_KEY; // read that somewhere
  
  
  
  // write the packet to stream
  
  av_interleaved_write_frame(m_FormatContext, m_EncodedPacket);
  
  
  
  
  
  // after the loop
  
  // I encode delayed frames and write the trailer
  
  av_write_trailer(m_FormatContext); 

  Thats pretty much it. I’m not getting what is missing. Since I have some meta data per frame I tried to add side data to each package but this data also disapered after reading from file. If decode the packets directly (instead of writing them to file, the data is there)

  I’m quite sure the problem is with the encoding. I managed to decode the big buck bunny movie in which case i got valid pts values.

  Thanks a lot for your help !