Recherche avancée

Sur d’autres sites (2919)

• avcodec, avformat : deprecate anything related to side data merging

  16 mars 2017, par wm4

  avcodec, avformat : deprecate anything related to side data merging
  This patch deprecates anything that has to do with merging/splitting
  
  side data. Automatic side data merging (and splitting), as well as all
  
  API symbols involved in it, are removed completely.
  Two FF_API_ defines are dedicated to deprecating API symbols related to
  
  this : FF_API_MERGE_SD_API removes av_packet_split/merge_side_data in
  
  libavcodec, and FF_API_LAVF_KEEPSIDE_FLAG deprecates
  
  AVFMT_FLAG_KEEP_SIDE_DATA in libavformat.
  Since it was claimed that changing the default from merging side data to
  
  not doing it is an ABI change, there are two additional FF_API_ defines,
  
  which stop using the side data merging/splitting by default (and remove
  
  any code in avformat/avcodec doing this) : FF_API_MERGE_SD in libavcodec,
  
  and FF_API_LAVF_MERGE_SD in libavformat.
  It is very much intended that FF_API_MERGE_SD and FF_API_LAVF_MERGE_SD
  
  are quickly defined to 0 in the next ABI bump, while the API symbols are
  
  retained for a longer time for the sake of compatibility.
  
  AVFMT_FLAG_KEEP_SIDE_DATA will (very much intentionally) do nothing for
  
  most of the time it will still be defined. Keep in mind that no code
  
  exists that actually tries to unset this flag for any reason, nor does
  
  such code need to exist. Code setting this flag explicitly will work as
  
  before. Thus it’s ok for AVFMT_FLAG_KEEP_SIDE_DATA to do nothing once
  
  side data merging has been removed from libavformat.
  In order to avoid that anyone in the future does this incorrectly, here
  
  is a small guide how to update the internal code on bumps :
   next ABI bump (probably soon) :
  
    - define FF_API_LAVF_MERGE_SD to 0, and remove all code covered by it
  
    - define FF_API_MERGE_SD to 0, and remove all code covered by it
  
   next API bump (typically two years in the future or so) :
  
    - define FF_API_LAVF_KEEPSIDE_FLAG to 0, and remove all code covered
  
      by it
  
    - define FF_API_MERGE_SD_API to 0, and remove all code covered by it
  This forces anyone who actually wants packet side data to temporarily
  
  use deprecated API to get it all. If you ask me, this is batshit fucked
  
  up crazy, but it’s how we roll. Making AVFMT_FLAG_KEEP_SIDE_DATA to be
  
  set by default was rejected as an ABI change, so I’m going all the way
  
  to get rid of this once and for all.
  Reviewed-by : James Almer <jamrial@gmail.com>
  
  Reviewed-by : Rostislav Pehlivanov <atomnuker@gmail.com>
  
  Reviewed-by : Michael Niedermayer <michael@niedermayer.cc>
  

  • [DH] doc/APIchanges
  • [DH] libavcodec/avcodec.h
  • [DH] libavcodec/avpacket.c
  • [DH] libavcodec/utils.c
  • [DH] libavcodec/version.h
  • [DH] libavformat/avformat.h
  • [DH] libavformat/mux.c
  • [DH] libavformat/options_table.h
  • [DH] libavformat/utils.c
  • [DH] libavformat/version.h

• Reverse Engineering Radius VideoVision

  3 avril 2011, par Multimedia Mike — Reverse Engineering

  I was called upon to help reverse engineer an old video codec called VideoVision (FourCC : PGVV), ostensibly from a company named Radius. I’m not sure of the details exactly but I think a game developer has a bunch of original FMV data from an old game locked up in this format. The name of the codec sounded familiar. Indeed, we have had a sample in the repository since 2002. Alex B. did some wiki work on the codec some years ago. The wiki mentions that there existed a tool to transcode PGVV data into MJPEG-B data, which is already known and supported by FFmpeg.

  The Software
  My contacts were able to point me to some software, now safely archived in the PGVV samples directory. There is StudioPlayer2.6.2.sit.hqx which is supposed to be a QuickTime component for working with PGVV data. I can’t even remember how to deal with .sit or .hqx data. Then there is RadiusVVTranscoder101.zip which is the tool that transcodes to MJPEG-B.

  Disassembling for Reverse Engineering
  Since I could actually unpack the transcoder, I set my sights on that. Unpacking the archive sets up a directory structure for a component. There is a binary called RadiusVVTranscoder under RadiusVVTranscoder.component/Contents/MacOS/. Basic deadlisting disassembly is performed via ’otool’ as shown :

    otool -tV RadiusVVTranscoder | c++filt
  

  This results in a deadlisting of both PowerPC and 32-bit x86 code, as the binary is a "fat" Mac OS X binary designed to run on both architectures. The command line also demangles C++ function signatures which gives useful insight into the parameters passed to a function.

  Pretty Pictures
  The binary had a lot of descriptive symbols. As a basis for reverse engineering, I constructed call graphs using these symbols. Here are the 2 most relevant portions (click for larger images).

  The codec initialization generates Huffman tables relevant to the codec :

  
  
  

  The main decode function calls AddMJPGFrame which apparently does the heavy lifting for the transcode process :

  
  
  

  Based on this tree, I’m guessing that luma blocks can be losslessly transcoded (perhaps with different Huffman tables) which chroma blocks may rely on a different quantization method.

  Assembly Constructs
  I started looking at the instructions (the x86 ones, of course). The binary uses a calling convention I haven’t seen before, at least not for the x86 : Rather than pushing function arguments onto the stack, the code manually subtracts, e.g., 12 from the ESP register, loads 3 32-bit arguments into memory relative to ESP, and then proceeds with the function call.

  I’m also a little unclear on constructs such as "call ___i686.get_pc_thunk.bx" seen throughout relevant functions such as MakeRadiusQuantizationTables().

  I’m just presenting what I have so far in case anyone else wants to try their hand.

• Playing With File

  8 septembre 2011, par Multimedia Mike — General

  I played with the ‘file’ utility a long time ago because I wanted to make it recognize a large number of multimedia formats. I had trouble getting my changes to take. But I’m prepared to try again after many years.

  Aiming at the Corpus
  In my local mirror of the MPlayerHQ samples archive, I find 9853 unique files. So I run all of them through the ‘file’ command :

    ’find /path/to/samples -type f -print0 | xargs -0 file —no-pad’
  

  My Ubuntu installation has file v5.04. I also tested against 5.07 and the latest, 5.08. Here is the number of files each version was unable to identify (generically marking as ‘data’) :

  5.04  1521
  5.07  1405
  5.08  1501
  

  That seems like a regression for v5.08 until I dug into the details and saw quite a few items like this, indicating that the MPEG detection could use some work :

  
   mov/mov-demux-infinite-loop.mpg : DOS-executable (
  +mov/mg-‹demux-infinite-loop.mpg : data
  
   image-samples/UNeedQT4.pntg : DOS-executable (
  +imY- samples/UNeedQT4.pntg : data
  

  Workflow
  These are just notes to myself and perhaps anyone else who wants to add new file formats to be identified by the ‘file’ command.

  First, download either the latest release from the FTP or clone from Github. Do the usual unpack, ‘./configure’, ‘make’ routine. To use this newly-built version and its associated magic file :

    ./src/file —magic-file magic/magic.mgc <file>
  

  To add a new format for ID, first, run the foregoing command to ensure that it’s not already identified. Then, check over the files in magic/Magdir and see which one might pertain to what you’re doing (it’s unlikely that your format will merit a new file in this directory). For example, for this round, I modified animation, audio, iff, and riff. Add or modify existing specs based on the copious examples in the directory and by consulting the appropriate man page (‘man 5 magic’).

  Finally, run ‘make’ again which will regenerate the magic file. Invoke the above command again to use the modified magic file.

  Before and After
  On a selection of formats taken from the samples archive (renamed and cut down to a kilobyte because detection typically only relies on the first few bytes), here is the “before” :

  amv :            RIFF (little-endian) data
  armovie :        data
  bbc-dirac :      data
  interplay-mve :  data
  mtv :            data
  nintendo-thp :   data
  nullsoft-video : data
  redcode :        data
  sega-film :      data
  smacker :        data
  trueaudio :      data
  vqa :            IFF data
  wavpack :        data
  wc3-mve :        IFF data
  wtv :            data
  

  And the “after” :

  amv :            RIFF (little-endian) data, AMV
  armovie :        ARMovie
  bbc-dirac :      BBC Dirac Video
  interplay-mve :  Interplay MVE Movie
  mtv :            MTV Multimedia File
  nintendo-thp :   Nintendo THP Multimedia
  nullsoft-video : Nullsoft Video
  redcode :        REDCode Video
  sega-film :      Sega FILM/CPK Multimedia, 320 x 224
  smacker :        RAD Game Tools Smacker Multimedia version 2, 320 x 200, 100 frames
  trueaudio :      True Audio Lossless Audio
  vqa :            IFF data, Westwood Studios VQA Multimedia, 418 video frames, 320 x 200
  wavpack :        WavPack Lossless Audio
  wc3-mve :        IFF data, Wing Commander III Video, PC version
  wtv :            Windows Television DVR Media
  

  After rerunning ‘file’ on the mphq corpus using the modified magic file, only 1329 files remain unidentified (down from 1501).

  Going Forward
  As mentioned, MPEG detection could probably be strengthened. However, a major weakness is QuickTime/MP4. Many files are not detected, probably owing to the many ways that QuickTime files can begin.