Sur d’autres sites (5536)

• lavc : use a separate field for exporting audio encoder padding

  23 août 2014, par Anton Khirnov

  lavc : use a separate field for exporting audio encoder padding
  Currently, the amount of padding inserted at the beginning by some audio
  
  encoders, is exported through AVCodecContext.delay. However
  
   the term ’delay’ is heavily overloaded and can have multiple different
  
    meanings even in the case of audio encoding.
  
   this field has entirely different meanings, depending on whether the
  
    codec context is used for encoding or decoding (and has yet another
  
    different meaning for video), preventing generic handling of the codec
  
    context.
  Therefore, add a new field — AVCodecContext.initial_padding. It could
  
  conceivably be used for decoding as well at a later point.
  

  • [DBH] doc/APIchanges
  • [DBH] libavcodec/aacenc.c
  • [DBH] libavcodec/ac3enc.c
  • [DBH] libavcodec/audio_frame_queue.c
  • [DBH] libavcodec/avcodec.h
  • [DBH] libavcodec/g722enc.c
  • [DBH] libavcodec/libfaac.c
  • [DBH] libavcodec/libfdk-aacenc.c
  • [DBH] libavcodec/libmp3lame.c
  • [DBH] libavcodec/libopencore-amr.c
  • [DBH] libavcodec/libopusenc.c
  • [DBH] libavcodec/libspeexenc.c
  • [DBH] libavcodec/libtwolame.c
  • [DBH] libavcodec/libvo-aacenc.c
  • [DBH] libavcodec/libvo-amrwbenc.c
  • [DBH] libavcodec/libvorbis.c
  • [DBH] libavcodec/mpegaudioenc.c
  • [DBH] libavcodec/nellymoserenc.c
  • [DBH] libavcodec/ra144enc.c
  • [DBH] libavcodec/utils.c
  • [DBH] libavcodec/version.h
  • [DBH] libavcodec/wmaenc.c

• ffmpeg and streaming video - frame time issue

  6 mai 2014, par 4ntoine

  I’ve compiled ffmpeg-android (https://github.com/appunite/AndroidFFmpeg) and it works for files.
  The problem is that it shows nothing for network streams (both rtmp and hls) as frame timestamp is too big and it seems to be not from video beginning (as for file). Then it waits for the first frame time which for network stream is too big :

  player_decode_video copying...
  
  05-05 18:11:26.994: INFO/player.c(16998): player_decode_video Decoded video frame: 568.233000, time_base: 51140970
  
  05-05 18:11:26.994: INFO/player.c(16998): player_wait_for_frame[0] start
  
  ...
  
  05-05 18:11:30.587: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (0.000977)
  
  05-05 18:11:30.587: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 568232023
  
  05-05 18:11:31.088: INFO/player.c(16998): player_wait_for_frame[0] timeout
  
  05-05 18:11:31.088: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (0.501542)
  
  05-05 18:11:31.088: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 567731458
  
  05-05 18:11:31.588: INFO/player.c(16998): player_wait_for_frame[0] timeout
  
  05-05 18:11:31.588: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (1.002778)
  
  05-05 18:11:31.588: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 567230222
  
  05-05 18:11:32.089: INFO/player.c(16998): player_wait_for_frame[0] timeout
  
  05-05 18:11:32.089: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (1.504563)
  
  05-05 18:11:32.089: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 566728437
  
  05-05 18:11:32.590: INFO/player.c(16998): player_wait_for_frame[0] timeout
  
  05-05 18:11:32.590: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (2.005708)
  
  05-05 18:11:32.590: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 566227292

  also i can’t figure out what timestamp for frame for network stream means (for file frames it’s a time from video beginning)

  for file it’s clear : time_base is frame number and timestamp is time from video beginning and it plays the files :

  player_decode_video Decoded video frame: 0.320000, time_base: 8
  
  05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0] start
  
  05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.320000) - (0.344337)
  
  05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -24337
  
  05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0] finish[0]
  
  05-05 18:32:42.344: INFO/player.c(16998): player_update_time: 0.346169/4.000000
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode waiting for frame[0]
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode decoding frame[0]
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode_video decoding
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode_video copy wait
  
  05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream Read frame
  
  05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream looking for stream
  
  05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream stream found [0]
  
  05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream waiting for queue
  
  05-05 18:32:42.344: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888
  
  05-05 18:32:42.344: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode_video copying...
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode_video Decoded video frame: 0.360000, time_base: 9
  
  05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0] start
  
  05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.360000) - (0.427994)
  
  05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -67994
  
  05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0] finish[0]
  
  05-05 18:32:42.425: INFO/player.c(16998): player_update_time: 0.429214/4.000000
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode waiting for frame[0]
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode decoding frame[0]
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode_video decoding
  
  05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream Read frame
  
  05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream looking for stream
  
  05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream stream found [0]
  
  05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream waiting for queue
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode_video copy wait
  
  05-05 18:32:42.435: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888
  
  05-05 18:32:42.435: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280
  
  05-05 18:32:42.435: INFO/player.c(16998): player_decode_video copying...
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode_video Decoded video frame: 0.400000, time_base: 10
  
  05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0] start
  
  05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.400000) - (0.494742)
  
  05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -94742
  
  05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0] finish[0]
  
  05-05 18:32:42.495: INFO/player.c(16998): player_update_time: 0.495993/4.000000
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode waiting for frame[0]
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode decoding frame[0]
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode_video decoding
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode_video copy wait
  
  05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream Read frame
  
  05-05 18:32:42.495: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888
  
  05-05 18:32:42.495: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode_video copying...
  
  05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream looking for stream
  
  05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream stream found [0]
  
  05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream waiting for queue
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode_video Decoded video frame: 0.440000, time_base: 11
  
  05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0] start
  
  05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.440000) - (0.556698)
  
  05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -116698
  
  05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0] finish[0]
  
  05-05 18:32:42.555: INFO/player.c(16998): player_update_time: 0.557858/4.000000
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode waiting for frame[0]
  
  05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream Read frame
  
  05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream looking for stream
  
  05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream stream found [0]
  
  05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream waiting for queue
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode decoding frame[0]
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode_video decoding
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode_video copy wait
  
  05-05 18:32:42.565: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888
  
  05-05 18:32:42.565: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280
  
  05-05 18:32:42.565: INFO/player.c(16998): player_decode_video copying...
  
  05-05 18:32:42.625: INFO/player.c(16998): player_decode_video Decoded video frame: 0.480000, time_base: 12

  So what is the meaning of timestamp for stream and what should i change to make it playing streams (rtmp/hls - its can be opened now but it wait for frame time which is far in the future) ?

  UPDATE :

      int64_t pts = av_frame_get_best_effort_timestamp(frame);
  
  if (pts == AV_NOPTS_VALUE) {
  
      pts = 0;
  
  }
  
  int64_t time = av_rescale_q(pts, stream->time_base, AV_TIME_BASE_Q);
  
  LOGI(10,
  
          "player_decode_video Decoded video frame: %f, time_base: %" SCNd64,
  
          time/1000000.0, pts);
  
  player_wait_for_frame(player, time, stream_no);

  it extracts time which is far in future :

  player_wait_for_frame[0 = Video] = (568.233000) - (0.000977)

  Is it correct ?

• On-demand and seamless transcoding of individual HLS segments

  5 janvier 2024, par Omid Ariyan

  Background

  


  I've been meaning to implement on-demand transcoding of certain video formats such as ".mkv", ".wmv", ".mov", etc. in order to serve them on a media management server using ASP.NET Core 6.0, C# and ffmpeg.

  


  My Approach

  


  The approach I've decided to use is to serve a dynamically generated .m3u8 file which is simply generated using a segment duration of choice e.g. 10s and the known video duration. Here's how I've done it. Note that the resolution is currently not implemented and discarded :

  


  public string GenerateVideoOnDemandPlaylist(double duration, int segment)
{
   double interval = (double)segment;
   var content = new StringBuilder();

   content.AppendLine("#EXTM3U");
   content.AppendLine("#EXT-X-VERSION:6");
   content.AppendLine(String.Format("#EXT-X-TARGETDURATION:{0}", segment));
   content.AppendLine("#EXT-X-MEDIA-SEQUENCE:0");
   content.AppendLine("#EXT-X-PLAYLIST-TYPE:VOD");
   content.AppendLine("#EXT-X-INDEPENDENT-SEGMENTS");

   for (double index = 0; (index * interval) < duration; index++)
   {
      content.AppendLine(String.Format("#EXTINF:{0:#.000000},", ((duration - (index * interval)) > interval) ? interval : ((duration - (index * interval)))));
      content.AppendLine(String.Format("{0:00000}.ts", index));
   }

   content.AppendLine("#EXT-X-ENDLIST");

   return content.ToString();
}

[HttpGet]
[Route("stream/{id}/{resolution}.m3u8")]
public IActionResult Stream(string id, string resolution)
{
   double duration = RetrieveVideoLengthInSeconds();
   return Content(GenerateVideoOnDemandPlaylist(duration, 10), "application/x-mpegURL", Encoding.UTF8);
}


  


  Here's an example of how the .m3u8 file looks like :

  


  #EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:10.000000,
00000.ts
#EXTINF:3.386667,
00001.ts
#EXT-X-ENDLIST


  


  So the player would ask for 00000.ts, 00001.ts, etc. and the next step is to have them generated on demand :

  


  public byte[] GenerateVideoOnDemandSegment(int index, int duration, string path)&#xA;{&#xA;   int timeout = 30000;&#xA;   int totalWaitTime = 0;&#xA;   int waitInterval = 100;&#xA;   byte[] output = Array.Empty<byte>();&#xA;   string executable = "/opt/homebrew/bin/ffmpeg";&#xA;   DirectoryInfo temp = Directory.CreateDirectory(System.IO.Path.Combine(System.IO.Path.GetTempPath(), System.IO.Path.GetRandomFileName()));&#xA;   string format = System.IO.Path.Combine(temp.FullName, "output-%05d.ts");&#xA;&#xA;   using (Process ffmpeg = new())&#xA;   {&#xA;      ffmpeg.StartInfo.FileName = executable;&#xA;&#xA;      ffmpeg.StartInfo.Arguments = String.Format("-ss {0} ", index * duration);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-y -t {0} ", duration);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-i \"{0}\" ", path);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-c:v libx264 -c:a aac ");&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-segment_time {0} -reset_timestamps 1 -break_non_keyframes 1 -map 0 ", duration);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-initial_offset {0} ", index * duration);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-f segment -segment_format mpegts {0}", format);&#xA;&#xA;      ffmpeg.StartInfo.CreateNoWindow = true;&#xA;      ffmpeg.StartInfo.UseShellExecute = false;&#xA;      ffmpeg.StartInfo.RedirectStandardError = false;&#xA;      ffmpeg.StartInfo.RedirectStandardOutput = false;&#xA;&#xA;      ffmpeg.Start();&#xA;&#xA;      do&#xA;      {&#xA;         Thread.Sleep(waitInterval);&#xA;         totalWaitTime &#x2B;= waitInterval;&#xA;      }&#xA;      while ((!ffmpeg.HasExited) &amp;&amp; (totalWaitTime &lt; timeout));&#xA;&#xA;      if (ffmpeg.HasExited)&#xA;      {&#xA;         string filename = System.IO.Path.Combine(temp.FullName, "output-00000.ts");&#xA;&#xA;         if (!File.Exists(filename))&#xA;         {&#xA;            throw new FileNotFoundException("Unable to find the generated segment: " &#x2B; filename);&#xA;         }&#xA;&#xA;         output = File.ReadAllBytes(filename);&#xA;      }&#xA;      else&#xA;      {&#xA;         // It&#x27;s been too long. Kill it!&#xA;         ffmpeg.Kill();&#xA;      }&#xA;   }&#xA;&#xA;   // Remove the temporary directory and all its contents.&#xA;   temp.Delete(true);&#xA;&#xA;   return output;&#xA;}&#xA;&#xA;[HttpGet]&#xA;[Route("stream/{id}/{index}.ts")]&#xA;public IActionResult Segment(string id, int index)&#xA;{&#xA;   string path = RetrieveVideoPath(id);&#xA;   return File(GenerateVideoOnDemandSegment(index, 10, path), "application/x-mpegURL", true);&#xA;}&#xA;</byte>

  


  So as you can see, here's the command I use to generate each segment incrementing -ss and -initial_offset by 10 for each segment :

  


  ffmpeg -ss 0 -y -t 10 -i "video.mov" -c:v libx264 -c:a aac -segment_time 10 -reset_timestamps 1 -break_non_keyframes 1 -map 0 -initial_offset 0 -f segment -segment_format mpegts /var/folders/8h/3xdhhky96b5bk2w2br6bt8n00000gn/T/4ynrwu0q.z24/output-%05d.ts


  


  The Problem

  


  Things work on a functional level, however the transition between segments is slightly glitchy and especially the audio has very short interruptions at each 10 second mark. How can I ensure the segments are seamless ? What can I improve in this process ?