Recherche avancée

Sur d’autres sites (6237)

• Minimal Understanding of VP8′s Forward Transform

  16 novembre 2010, par Multimedia Mike — VP8

  Regarding my toy VP8 encoder, Pengvado mentioned in the comments of my last post, “x264 looks perfect using only i16x16 DC mode. You must be doing something wrong in computing residual or fdct or quantization.” This makes a lot of sense. The encoder generates a series of elements which describe how to reconstruct the original image. Intra block reconstruction takes into consideration the following elements :

  
  
  

  I have already verified that both my encoder and FFmpeg’s VP8 decoder agree precisely on how to reconstruct blocks based on the predictors, coefficients, and quantizers. Thus, if the decoded image still looks crazy, the elements the encoder is generating to describe the image must be wrong.

  So I started studying the forward DCT, which I had cribbed wholesale from the original libvpx 0.9.0 source code. It should be noted that the formal VP8 spec only defines the inverse transform process, not the forward process. I was using a version designated as the “short” version, vs. the “fast” version. Then I looked at the 0.9.5 FDCT. Then I got the idea of comparing the results of each.

  input:   92 91 89 86 91 90 88 86 89 89 89 88 89 87 88 93

  • libvpx 0.9.0 “short” :

    forward : -314 5 1 5 4 5 -2 0 0 1 -1 -1 1 11 -3 -4
    inverse : 92 91 89 86 89 86 91 90 91 90 88 86 88 86 89 89
    

  • libvpx 0.9.0 “fast” :

    forward : -314 4 0 5 4 4 -2 0 0 1 0 -1 1 11 -2 -5
    inverse : 91 91 89 86 88 86 91 90 91 90 88 86 88 86 89 89
    

  • libvpx 0.9.5 “short” :

    forward : -312 7 1 0 1 12 -5 2 2 -3 3 -1 1 0 -2 1
    inverse : 92 91 89 86 91 90 88 86 89 89 89 88 89 87 88 93
    

  I was surprised when I noticed that input[] != idct(fdct(input[])) in some of the above cases. Then I remembered that the aforementioned property isn’t what is meant by a “bit-exact” transform– only that all implementations of the inverse transform are supposed to produce bit-exact output for a given vector of input coefficients.

  Anyway, I tried applying each of these forward transforms. I got slightly differing results, with the latest one I tried (the fdct from libvpx 0.9.5) producing the best results (to my eye). At least the trees look better in the Big Buck Bunny logo image :

  
  
  The dense trees of the Big Buck Bunny logo using one of the libvpx 0.9.0 forward transforms
  
  
  The same segment of the image using the libvpx 0.9.5 forward transform
  

  Then again, it could be that the different numbers generated by the newer forward transform triggered different prediction modes to be chosen. Overall, adapting the newer FDCT did not dramatically improve the encoding quality.

  Working on the intra 4×4 mode encoding is generating some rather more accurate blocks than my intra 16×16 encoder. Pengvado indicated that x264 generates perfectly legible results when forcing the encoder to only use intra 16×16 mode. To be honest, I’m having trouble understanding how that can possibly occur thanks to the Walsh-Hadamard transform (WHT). I think that’s where a lot of the error is creeping in with my intra 16×16 encoder. Then again, FFmpeg implements an inverse WHT function that bears ‘vp8′ in its name. This implies that it’s custom to the algorithm and not exactly shared with H.264.

• Why is ffmpeg linking with too many libraries ?

  2 décembre 2020, par Thomas

  I'm trying to build ffmpeg shared libraries for decoding video on Linux systems. The build is done on Arch Linux but the resulting shared libraries need to be as portable as possible.

  


  Fortunately, the functionality built into ffmpeg is sufficient, so I don't want to link with any libraries that might or might not be present on the target system. I'm taking libvdpau as an example of an unwanted dependency here, but there are many more.

  


  Here's what I tried :

  


  $ git clone https://git.ffmpeg.org/ffmpeg.git --branch n4.3.1 --depth 1
$ cd ffmpeg
$ mkdir build
$ cd build
$ ../configure \
    --enable-shared \
    --disable-programs --disable-doc \
    --disable-avdevice --disable-postproc --disable-avfilter \
    --disable-autodetect
$ make -j9


  


  Notice --disable-autodetect which should prevent autodetection of which dependencies happen to be present on the build system.

  


  And yet, when I check the resulting libraries, all of them have dependencies that I didn't ask for. For example :

  


  $ ldd libavcodec/libavcodec.so
    linux-vdso.so.1 (0x00007ffcd73cd000)
    libswresample.so.3 => /usr/lib/libswresample.so.3 (0x00007fba1e45c000)
    libavutil.so.56 => /usr/lib/libavutil.so.56 (0x00007fba1e1a7000)
    libm.so.6 => /usr/lib/libm.so.6 (0x00007fba1e061000)
    libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fba1e03f000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007fba1de76000)
    libsoxr.so.0 => /usr/lib/libsoxr.so.0 (0x00007fba1ddf4000)
    libva-drm.so.2 => /usr/lib/libva-drm.so.2 (0x00007fba1dded000)
    libva.so.2 => /usr/lib/libva.so.2 (0x00007fba1ddc0000)
    libva-x11.so.2 => /usr/lib/libva-x11.so.2 (0x00007fba1ddb8000)
    libvdpau.so.1 => /usr/lib/libvdpau.so.1 (0x00007fba1ddb3000)     <- Why is this here?
    libX11.so.6 => /usr/lib/libX11.so.6 (0x00007fba1dc72000)
    libdrm.so.2 => /usr/lib/libdrm.so.2 (0x00007fba1dc5d000)
    libmfx.so.1 => /usr/lib/libmfx.so.1 (0x00007fba1dc4d000)
    libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fba1dc47000)
    /usr/lib64/ld-linux-x86-64.so.2 (0x00007fba1f971000)
    libgomp.so.1 => /usr/lib/libgomp.so.1 (0x00007fba1dc06000)
    libXext.so.6 => /usr/lib/libXext.so.6 (0x00007fba1dbf1000)
    libXfixes.so.3 => /usr/lib/libXfixes.so.3 (0x00007fba1dbe8000)
    libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007fba1dbbc000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fba1d9df000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fba1d9c5000)
    libXau.so.6 => /usr/lib/libXau.so.6 (0x00007fba1d9c0000)
    libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007fba1d9b8000)


  


  I also tried listing all disable flags explicitly, such as --disable-vdpau, but this didn't change anything ; libvdpau.so.1 is still listed in ldd output.

  


  Maybe they are just being linked, but not actually used ? readelf -d seems to point in that direction :

  


  $ ls */*.so
libavcodec/libavcodec.so    libavutil/libavutil.so          libswscale/libswscale.so
libavformat/libavformat.so  libswresample/libswresample.so
$ readelf -d */*.so | grep 'Shared library:' | sort -u
 0x0000000000000001 (NEEDED)             Shared library: [libavcodec.so.58]
 0x0000000000000001 (NEEDED)             Shared library: [libavutil.so.56]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libswresample.so.3]


  


  This is a nice and tight set of dependencies, the way I actually want them to be.

  


  So maybe the build system is stupid and always adds all detected libraries to the linker command line anyway ? To suppress this, I tried adding --extra-ldflags=-Wl,--as-needed, but that doesn't seem to have any effect either.

  


  The resulting linker command line looks like this (abridged) :

  


  $ make -n libavcodec/libavcodec.so.58
gcc \
    -shared \
    -Wl,-soname,libavcodec.so.58 \
    -Wl,-Bsymbolic \
    -Wl,--version-script,libavcodec/libavcodec.ver \
    -Llibavcodec -Llibavdevice -Llibavfilter -Llibavformat -Llibavresample -Llibavutil -Llibpostproc -Llibswscale -Llibswresample \
    -Wl,--as-needed \
    -Wl,-z,noexecstack \
    -Wl,--warn-common \
    -Wl,-rpath-link=:libpostproc:libswresample:libswscale:libavfilter:libavdevice:libavformat:libavcodec:libavutil:libavresample \
    -Wl,--as-needed \
    -o libavcodec/libavcodec.so.58 \
    [...all .o files...] \
    -lswresample -lavutil -pthread -lm -lm -pthread -lm


  


  It looks like -Wl,--as-needed was already added anyway.

  


  Interestingly, -Llibavdevice is also listed, even though I passed --disable-avdevice --disable-postproc --disable-avfilter to the configure script. That might be a clue : it would pick up my system's libavdevice, which of course comes with far more dependencies. Still I'd expect -Wl,--as-needed to trim these unbuilt dependencies out as well.

  


  Where do these dependencies come from, and how do I get rid of them ?

  


• Send MP3 audio extracted from m3u8 stream to IBM Watson Speech To Text

  20 novembre 2018, par Link69

  I’m extracting audio in MP3 format from a M3U8 live url and the final goal is to send the live audio stream to IBM Watson Speech To Text. The m3u8 is obtained by calling an external script via a Process. Then I use FFMPEG script to get the audio in stdout. It works if I save the audio in a file but I don’t want to save the extracted audio, I need to send the datas directly to the STT service. So far I proceeded like this :

  SpeechToTextService speechToTextService = new SpeechToTextService(sttUsername, sttPassword);
  
  string m3u8Url = "https://something.m3u8";
  
  char[] buffer = new char[48000];
  
  Process ffmpeg = new ProcessHelper(@"ffmpeg\ffmpeg.exe", $"-v 0 -i {m3u8Url} -acodec mp3 -ac 2 -ar 48000 -f mp3 -");
  
  
  
  ffmpeg.Start();
  
  int count;
  
  while ((count = ffmpeg.StandardOutput.Read(buffer, 0, 48000)) > 0)
  
  {
  
      ffmpeg.StandardOutput.Read(buffer, 0, 48000);
  
      var answer = speechToTextService.RecognizeSessionless(
  
          audio: buffer.Select(c => (byte)c).ToArray(),
  
          contentType: "audio/mpeg",
  
          smartFormatting: true,
  
          speakerLabels: false,
  
          model: "en-US_BroadbandModel"
  
      );
  
      // Get answer.ResponseJson, deserializing, clean buffer, etc...
  
  }

  When requesting the transcribed audio I’m getting this error :

  An unhandled exception of type 'System.AggregateException' occurred in IBM.WatsonDeveloperCloud.SpeechToText.v1.dll: 'One or more errors occurred. (The API query failed with status code BadRequest: Bad Request | x-global-transaction-id: bd6cd203720a70d83b9a03451fe28973 | X-DP-Watson-Tran-ID: bd6cd203720a70d83b9a03451fe28973)'
  
   Inner exceptions found, see $exception in variables window for more details.
  
   Innermost exception     IBM.WatsonDeveloperCloud.Http.Exceptions.ServiceResponseException : The API query failed with status code BadRequest: Bad Request | x-global-transaction-id: bd6cd203720a70d83b9a03451fe28973 | X-DP-Watson-Tran-ID: bd6cd203720a70d83b9a03451fe28973
  
     at IBM.WatsonDeveloperCloud.Http.Filters.ErrorFilter.OnResponse(IResponse response, HttpResponseMessage responseMessage)
  
     at IBM.WatsonDeveloperCloud.Http.Request.<getresponse>d__30.MoveNext()
  
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
  
     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
  
     at IBM.WatsonDeveloperCloud.Http.Request.<asmessage>d__23.MoveNext()
  
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
  
     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
  
     at IBM.WatsonDeveloperCloud.Http.Request.<as>d__24`1.MoveNext()
  
  </as></asmessage></getresponse>

  ProcessHelper is just for convenience :

  class ProcessHelper : Process
  
  {
  
      private string command;
  
      private string arguments;
  
      public ProcessHelper(string command, string arguments, bool redirectStandardOutput = true)
  
      {
  
          this.command = command;
  
          this.arguments = arguments;
  
          StartInfo = new ProcessStartInfo()
  
          {
  
              FileName = this.command,
  
              Arguments = this.arguments,
  
              UseShellExecute = false,
  
              RedirectStandardOutput = redirectStandardOutput,
  
              CreateNoWindow = true
  
          };
  
      }
  
  }

  Pretty sure I’m doing it wrong, I’d love someone to shine a light on this. Thanks.