Sur d’autres sites (4987)

• Winamp and the March of GUI

  1er juillet 2012, par Multimedia Mike — General, ars technica, gui, user interface, winamp

  Ars Technica recently published a 15-year retrospective on the venerable Winamp multimedia player, prompting bouts of nostalgia and revelations of "Huh ? That program is still around ?" from many readers. I was among them.

  
  
  

  I remember first using Winamp in 1997. I remember finding a few of these new files called MP3s online and being able to play the first 20 seconds using the official Fraunhofer Windows player— full playback required the fully licensed version. Then I searched for another player and came up with Winamp. The first version I ever used was v1.05 in the summer of 1997. I remember checking the website often for updates and trying out every single one. I can’t imagine doing that nowadays— programs need to auto-update themselves (which Winamp probably does now ; I can’t recall the last time I used the program).

  Video Underdog
  The last time Winamp came up on my radar was early in 2003 when a new version came with support for a custom, proprietary multimedia audio/video format called Nullsoft Video (NSV). I remember the timeframe because the date is indicated in the earliest revision of my NSV spec document (back when I was maintaining such docs in a series of plaintext files). This was cobbled together from details I and others in the open source multimedia community sorted out from sample files. It was missing quite a few details, though.

  Then, Winamp founder Justin Frankel — introduced through a colleague on the xine team — emailed me his official NSV format and told me I was free to incorporate details into my document just as long as it wasn’t obvious that I had the official spec. This put me in an obnoxious position of trying to incorporate details which would have been very difficult to reverse engineer without the official doc. I think I coped with the situation by never really getting around to updating my doc in any meaningful way. Then, one day, the official spec was released to the world anyway, and it is now mirrored here at multimedia.cx.

  I don’t think the format ever really caught on in any meaningful way, so not a big deal. (Anytime I say that about a format, I always learn it saw huge adoption is some small but vocal community.)

  What’s Wrong With This Picture ?
  What I really wanted to discuss in this post was the matter of graphical user interfaces and how they have changed in the last 15 years.
  
  I still remember when I first downloaded Winamp v1.05 and tried it on my Windows machine at the time. Indignantly, the first thought I had was, "What makes this program think it’s so special that it’s allowed to violate the user interface conventions put forth by the rest of the desktop ?" All of the Windows programs followed a standard set of user interface patterns and had a consistent look and feel... and then Winamp came along and felt it could violate all those conventions.

  I guess I let the program get away with it because it was either that or only play 20-second clips from the unregistered Fraunhofer player. Though incredibly sterile by comparison, the Fraunhofer player, it should be noted, followed Windows UI guidelines to the letter.

  As the summer of 1997 progressed and more Winamp versions were released, eventually one came out (I think it was v1.6 or so) that supported skins. I was excited because there was a skin that made the program look like a proper Windows program— at least if you used the default Windows color scheme, and had all of your fonts a certain type and size.

  Skins were implemented by packaging together a set of BMP images to overlay on various UI elements. I immediately saw a number of shortcomings with this skinning approach. A big one was UI lock-in. Ironically, if you skin an app and wish to maintain backwards compatibility with the thousands of skins selflessly authored by your vibrant community (seriously, I couldn’t believe how prolific these things were), then you were effectively locked into the primary UI. Forget about adding a new button anywhere.

  Another big problem was resolution-independence. Basing your UI on static bitmaps doesn’t scale well with various resolutions. Winamp had its normal mode and it also had double-sized mode.

  Skins proliferated among many types of programs in the late 1990s. I always treasured this Suck.com (remember them ? that’s a whole other nostalgia trip) essay from April, 2000 entitled Skin Cancer. Still, Winamp was basically the standard, and the best, and I put away my righteous nerd rage and even dug through the vast troves of skins. I remember settling on Swankamp for a good part of 1998, probably due to the neo-swing revival at the time.

  
  
  

  Then again, if Winamp irked me, imagine my reaction when I was first exposed to the Sonique Music Player in 1998 :

  
  
  

  The New UI Order
  Upon reflection, I realize now that I had a really myopic view of what a computer GUI should be. I thought the GUIs were necessarily supposed to follow the WIMP (windows, icons, mouse, pointer) paradigm and couldn’t conceive of anything different. For a long time, I couldn’t envision a useful GUI on a small device (like a phone) because WIMP didn’t fit well on such a small interface (even though I saw various ill-fated attempts to make it work). This thinking seriously crippled me when I was trying to craft a GUI for a custom console media player I was developing as a hobby many years ago.

  I’m looking around at what I have open on my Windows 7 desktop right now. Google Chrome browser, Apple iTunes, Adobe Photoshop Elements, and VMware Player are 4 programs which all seem to have their own skins. Maybe Winamp doesn’t look so out of place these days.

• On-demand and seamless transcoding of individual HLS segments

  5 janvier 2024, par Omid Ariyan

  Background

  


  I've been meaning to implement on-demand transcoding of certain video formats such as ".mkv", ".wmv", ".mov", etc. in order to serve them on a media management server using ASP.NET Core 6.0, C# and ffmpeg.

  


  My Approach

  


  The approach I've decided to use is to serve a dynamically generated .m3u8 file which is simply generated using a segment duration of choice e.g. 10s and the known video duration. Here's how I've done it. Note that the resolution is currently not implemented and discarded :

  


  public string GenerateVideoOnDemandPlaylist(double duration, int segment)
{
   double interval = (double)segment;
   var content = new StringBuilder();

   content.AppendLine("#EXTM3U");
   content.AppendLine("#EXT-X-VERSION:6");
   content.AppendLine(String.Format("#EXT-X-TARGETDURATION:{0}", segment));
   content.AppendLine("#EXT-X-MEDIA-SEQUENCE:0");
   content.AppendLine("#EXT-X-PLAYLIST-TYPE:VOD");
   content.AppendLine("#EXT-X-INDEPENDENT-SEGMENTS");

   for (double index = 0; (index * interval) < duration; index++)
   {
      content.AppendLine(String.Format("#EXTINF:{0:#.000000},", ((duration - (index * interval)) > interval) ? interval : ((duration - (index * interval)))));
      content.AppendLine(String.Format("{0:00000}.ts", index));
   }

   content.AppendLine("#EXT-X-ENDLIST");

   return content.ToString();
}

[HttpGet]
[Route("stream/{id}/{resolution}.m3u8")]
public IActionResult Stream(string id, string resolution)
{
   double duration = RetrieveVideoLengthInSeconds();
   return Content(GenerateVideoOnDemandPlaylist(duration, 10), "application/x-mpegURL", Encoding.UTF8);
}


  


  Here's an example of how the .m3u8 file looks like :

  


  #EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:10.000000,
00000.ts
#EXTINF:3.386667,
00001.ts
#EXT-X-ENDLIST


  


  So the player would ask for 00000.ts, 00001.ts, etc. and the next step is to have them generated on demand :

  


  public byte[] GenerateVideoOnDemandSegment(int index, int duration, string path)&#xA;{&#xA;   int timeout = 30000;&#xA;   int totalWaitTime = 0;&#xA;   int waitInterval = 100;&#xA;   byte[] output = Array.Empty<byte>();&#xA;   string executable = "/opt/homebrew/bin/ffmpeg";&#xA;   DirectoryInfo temp = Directory.CreateDirectory(System.IO.Path.Combine(System.IO.Path.GetTempPath(), System.IO.Path.GetRandomFileName()));&#xA;   string format = System.IO.Path.Combine(temp.FullName, "output-%05d.ts");&#xA;&#xA;   using (Process ffmpeg = new())&#xA;   {&#xA;      ffmpeg.StartInfo.FileName = executable;&#xA;&#xA;      ffmpeg.StartInfo.Arguments = String.Format("-ss {0} ", index * duration);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-y -t {0} ", duration);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-i \"{0}\" ", path);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-c:v libx264 -c:a aac ");&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-segment_time {0} -reset_timestamps 1 -break_non_keyframes 1 -map 0 ", duration);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-initial_offset {0} ", index * duration);&#xA;      ffmpeg.StartInfo.Arguments &#x2B;= String.Format("-f segment -segment_format mpegts {0}", format);&#xA;&#xA;      ffmpeg.StartInfo.CreateNoWindow = true;&#xA;      ffmpeg.StartInfo.UseShellExecute = false;&#xA;      ffmpeg.StartInfo.RedirectStandardError = false;&#xA;      ffmpeg.StartInfo.RedirectStandardOutput = false;&#xA;&#xA;      ffmpeg.Start();&#xA;&#xA;      do&#xA;      {&#xA;         Thread.Sleep(waitInterval);&#xA;         totalWaitTime &#x2B;= waitInterval;&#xA;      }&#xA;      while ((!ffmpeg.HasExited) &amp;&amp; (totalWaitTime &lt; timeout));&#xA;&#xA;      if (ffmpeg.HasExited)&#xA;      {&#xA;         string filename = System.IO.Path.Combine(temp.FullName, "output-00000.ts");&#xA;&#xA;         if (!File.Exists(filename))&#xA;         {&#xA;            throw new FileNotFoundException("Unable to find the generated segment: " &#x2B; filename);&#xA;         }&#xA;&#xA;         output = File.ReadAllBytes(filename);&#xA;      }&#xA;      else&#xA;      {&#xA;         // It&#x27;s been too long. Kill it!&#xA;         ffmpeg.Kill();&#xA;      }&#xA;   }&#xA;&#xA;   // Remove the temporary directory and all its contents.&#xA;   temp.Delete(true);&#xA;&#xA;   return output;&#xA;}&#xA;&#xA;[HttpGet]&#xA;[Route("stream/{id}/{index}.ts")]&#xA;public IActionResult Segment(string id, int index)&#xA;{&#xA;   string path = RetrieveVideoPath(id);&#xA;   return File(GenerateVideoOnDemandSegment(index, 10, path), "application/x-mpegURL", true);&#xA;}&#xA;</byte>

  


  So as you can see, here's the command I use to generate each segment incrementing -ss and -initial_offset by 10 for each segment :

  


  ffmpeg -ss 0 -y -t 10 -i "video.mov" -c:v libx264 -c:a aac -segment_time 10 -reset_timestamps 1 -break_non_keyframes 1 -map 0 -initial_offset 0 -f segment -segment_format mpegts /var/folders/8h/3xdhhky96b5bk2w2br6bt8n00000gn/T/4ynrwu0q.z24/output-%05d.ts


  


  The Problem

  


  Things work on a functional level, however the transition between segments is slightly glitchy and especially the audio has very short interruptions at each 10 second mark. How can I ensure the segments are seamless ? What can I improve in this process ?

  


• ac3enc_fixed : convert to 32-bit sample format

  9 janvier 2021, par Lynne

  ac3enc_fixed : convert to 32-bit sample format
  The AC3 encoder used to be a separate library called "Aften", which
  
  got merged into libavcodec (literally, SVN commits and all).
  
  The merge preserved as much features from the library as possible.
  The code had two versions - a fixed point version and a floating
  
  point version. FFmpeg had floating point DSP code used by other
  
  codecs, the AC3 decoder including, so the floating-point DSP was
  
  simply replaced with FFmpeg's own functions.
  
  However, FFmpeg had no fixed-point audio code at that point. So
  
  the encoder brought along its own fixed-point DSP functions,
  
  including a fixed-point MDCT.
  The fixed-point MDCT itself is trivially just a float MDCT with a
  
  different type and each multiply being a fixed-point multiply.
  
  So over time, it got refactored, and the FFT used for all other codecs
  
  was templated.
  Due to design decisions at the time, the fixed-point version of the
  
  encoder operates at 16-bits of precision. Although convenient, this,
  
  even at the time, was inadequate and inefficient. The encoder is noisy,
  
  does not produce output comparable to the float encoder, and even
  
  rings at higher frequencies due to the badly approximated winow function.
  Enter MIPS (owned by Imagination Technologies at the time). They wanted
  
  quick fixed-point decoding on their FPUless cores. So they contributed
  
  patches to template the AC3 decoder so it had both a fixed-point
  
  and a floating-point version. They also did the same for the AAC decoder.
  
  They however, used 32-bit samples. Not 16-bits. And we did not have
  
  32-bit fixed-point DSP functions, including an MDCT. But instead of
  
  templating our MDCT to output 3 versions (float, 32-bit fixed and 16-bit fixed),
  
  they simply copy-pasted their own MDCT into ours, and completely
  
  ifdeffed our own MDCT code out if a 32-bit fixed point MDCT was selected.
  This is also the status quo nowadays - 2 separate MDCTs, one which
  
  produces floating point and 16-bit fixed point versions, and one
  
  sort-of integrated which produces 32-bit MDCT.
  MIPS weren't all that interested in encoding, so they left the encoder
  
  as-is, and they didn't care much about the ifdeffery, mess or quality - it's
  
  not their problem.
  So the MDCT/FFT code has always been a thorn in anyone looking to clean up
  
  code's eye.
  Backstory over. Internally AC3 operates on 25-bit fixed-point coefficients.
  
  So for the floating point version, the encoder simply runs the float MDCT,
  
  and converts the resulting coefficients to 25-bit fixed-point, as AC3 is inherently
  
  a fixed-point codec. For the fixed-point version, the input is 16-bit samples,
  
  so to maximize precision the frame samples are analyzed and the highest set
  
  bit is detected via ac3_max_msb_abs_int16(), and the coefficients are then
  
  scaled up via ac3_lshift_int16(), so the input for the FFT is always at least 14 bits,
  
  computed in normalize_samples(). After FFT, the coefficients are scaled up to 25 bits.
  This patch simply changes the encoder to accept 32-bit samples, reusing
  
  the already well-optimized 32-bit MDCT code, allowing us to clean up and drop
  
  a large part of a very messy code of ours, as well as prepare for the future lavu/tx
  
  conversion. The coefficients are simply scaled down to 25 bits during windowing,
  
  skipping 2 separate scalings, as the hacks to extend precision are simply no longer
  
  necessary. There's no point in running the MDCT always at 32 bits when you're
  
  going to drop 6 bits off anyway, the headroom is plenty, and the MDCT rounds
  
  properly.
  This also makes the encoder even slightly more accurate over the float version,
  
  as there's no coefficient conversion step necessary.
  SIZE SAVINGS :
  
  ARM32 :
  
  HARDCODED TABLES :
  
  BASE           - 10709590
  
  DROP  DSP      - 10702872 - diff :   -6.56KiB
  
  DROP  MDCT     - 10667932 - diff :  -34.12KiB - both :   -40.68KiB
  
  DROP  FFT      - 10336652 - diff : -323.52KiB - all :   -364.20KiB
  
  SOFTCODED TABLES :
  
  BASE           -  9685096
  
  DROP  DSP      -  9678378 - diff :   -6.56KiB
  
  DROP  MDCT     -  9643466 - diff :  -34.09KiB - both :   -40.65KiB
  
  DROP  FFT      -  9573918 - diff :  -67.92KiB - all :   -108.57KiB
  ARM64 :
  
  HARDCODED TABLES :
  
  BASE           - 14641112
  
  DROP  DSP      - 14633806 - diff :   -7.13KiB
  
  DROP  MDCT     - 14604812 - diff :  -28.31KiB - both :   -35.45KiB
  
  DROP  FFT      - 14286826 - diff : -310.53KiB - all :   -345.98KiB
  
  SOFTCODED TABLES :
  
  BASE           - 13636238
  
  DROP  DSP      - 13628932 - diff :   -7.13KiB
  
  DROP  MDCT     - 13599866 - diff :  -28.38KiB - both :   -35.52KiB
  
  DROP  FFT      - 13542080 - diff :  -56.43KiB - all :    -91.95KiB
  x86 :
  
  HARDCODED TABLES :
  
  BASE           - 12367336
  
  DROP  DSP      - 12354698 - diff :  -12.34KiB
  
  DROP  MDCT     - 12331024 - diff :  -23.12KiB - both :   -35.46KiB
  
  DROP  FFT      - 12029788 - diff : -294.18KiB - all :   -329.64KiB
  
  SOFTCODED TABLES :
  
  BASE           - 11358094
  
  DROP  DSP      - 11345456 - diff :  -12.34KiB
  
  DROP  MDCT     - 11321742 - diff :  -23.16KiB - both :   -35.50KiB
  
  DROP  FFT      - 11276946 - diff :  -43.75KiB - all :    -79.25KiB
  PERFORMANCE (10min random s32le) :
  
  ARM32 - before -  39.9x - 0m15.046s
  
  ARM32 - after  -  28.2x - 0m21.525s
  
                         Speed :  -30%
  ARM64 - before -  36.1x - 0m16.637s
  
  ARM64 - after  -  36.0x - 0m16.727s
  
                         Speed : -0.5%
  x86   - before - 184x -    0m3.277s
  
  x86   - after  - 190x -    0m3.187s
  
                         Speed :   +3%
  

  • [DH] doc/encoders.texi
  • [DH] libavcodec/Makefile
  • [DH] libavcodec/ac3enc.c
  • [DH] libavcodec/ac3enc.h
  • [DH] libavcodec/ac3enc_fixed.c
  • [DH] libavcodec/ac3enc_float.c
  • [DH] libavcodec/ac3enc_template.c
  • [DH] libavcodec/version.h
  • [DH] tests/fate/ac3.mak
  • [DH] tests/fate/ffmpeg.mak
  • [DH] tests/ref/fate/unknown_layout-ac3
  • [DH] tests/ref/lavf/rm