Recherche avancée

Sur d’autres sites (3536)

• Reverse Engineering Clue Chronicles Compression

  15 janvier 2019, par Multimedia Mike — Game Hacking

  My last post described my exploration into the 1999 computer game Clue Chronicles : Fatal Illusion. Some readers expressed interest in the details so I thought I would post a bit more about how I have investigated and what I have learned.

  It’s frustrating to need to reverse engineer a compression algorithm that is only applied to a total of 8 files (out of a total set of 140), but here we are. Still, I’m glad some others expressed interest in this challenge as it motivated me to author this post, which in turn prompted me to test and challenge some of my assumptions.

  Spoiler : Commenter ‘m’ gave me the clue I needed : PKWare Data Compression Library used the implode algorithm rather than deflate. I was able to run this .ini data through an open source explode algorithm found in libmpq and got the correct data out.

  Files To Study
  I uploaded a selection of files for others to study, should they feel so inclined. These include the main game binary (if anyone has ideas about how to isolate the decompression algorithm from the deadlisting) ; compressed and uncompressed examples from 2 files (newspaper.ini and Drink.ini) ; and the compressed version of Clue.ini, which I suspect is the root of the game’s script.

  The Story So Far
  This ad-hoc scripting language found in the Clue Chronicles game is driven by a series of .ini files that are available in both compressed and uncompressed forms, save for a handful of them which only come in compressed flavor. I have figured out a few obvious details of the compressed file format :

  bytes 0-3 "COMP"
  bytes 4-11 unknown
  bytes 12-15 size of uncompressed data
  bytes 16-19 size of compressed data (filesize - 20 bytes)
  bytes 20- compressed payload
  

  The average compression ratio is on the same order as what could be achieved by running ‘gzip’ against the uncompressed files and using one of the lower number settings (i.e., favor speed vs. compression size, e.g., ‘gzip -2’ or ‘gzip -3’). Since the zlib/DEFLATE algorithm is quite widespread on every known computing platform, I thought that this would be a good candidate to test.

  Exploration
  My thinking was that I could load the bytes in the compressed ini file and feed it into Python’s zlib library, sliding through the first 100 bytes to see if any of them “catch” on the zlib decompression algorithm.

  Here is the exploration script :
  

  <script src="https://gist.github.com/multimediamike/c95f1a9cc58b959f4d8b2a299927d35e.js"></script>
  

  It didn’t work, i.e., the script did not find any valid zlib data. A commentor on my last post suggested trying bzip2, so I tried the same script but with the bzip2 decompressor library. Still no luck.

  Wrong Approach
  I realized I had not tested to make sure that this exploratory script would work on known zlib data. So I ran it on a .gz file and it failed to find zlib data. So it looks like my assumptions were wrong. Meanwhile, I can instruct Python to compress data with zlib and dump the data to a file, and then run the script against that raw zlib output and the script recognizes the data.

  I spent some time examining how zlib and gzip interact at the format level. It looks like the zlib data doesn’t actually begin on byte boundaries within a gzip container. So this approach was doomed to failure.

  A Closer Look At The Executable
  Installation of Clue Chronicles results in a main Windows executable named Fatal_Illusion.exe. It occurred to me to examine this again, specifically for references to something like zlib.dll. Nothing like that. However, a search for ‘compr’ shows various error messages which imply that there is PNG-related code inside (referencing IHDR and zTXt data types), even though PNG files are not present in the game’s asset mix.

  But there are also strings like “PKWARE Data Compression Library for Win32”. So I have started going down the rabbit hole of determining whether the compression is part of a ZIP format file. After all, a ZIP local file header data structure has 4-byte compressed and uncompressed sizes, as seen in this format.

  Binary Reverse Engineering
  At one point, I took the approach of attempting to reverse engineer the binary. When studying a deadlisting of the code, it’s easy to search for the string “COMP” and find some code that cares about these compressed files. Unfortunately, the code quickly follows an indirect jump instruction which makes it intractable to track the algorithm from a simple deadlisting.

  I also tried installing some old Microsoft dev tools on my old Windows XP box and setting some breakpoints while the game was running and do some old-fashioned step debugging. That was a total non-starter. According to my notes :

  Address 0x004A3C32 is the setup to the strncmp(“COMP”, ini_data, 4) function call. Start there.

  Problem : The game forces 640x480x256 mode and that makes debugging very difficult.

  Just For One Game ?
  I keep wondering if this engine was used for any other games. Clue Chronicles was created by EAI Interactive. As I review the list of games they are known to have created (ranging between 1997 and 2000), a few of them jump out at me as possibly being able to leverage the same engine. I have a few of them, so I checked those… nothing. Then I scrubbed some YouTube videos showing gameplay of other suspects. None of those strike me as having similar engine characteristics to Clue Chronicles. So this remains a mystery : did they really craft this engine with its own scripting language just for one game ?

  The post Reverse Engineering Clue Chronicles Compression first appeared on Breaking Eggs And Making Omelettes.

• ffmpeg nvenc_h264 streaming uhd2160 rawvideo. Youtube buffering

  23 janvier 2019, par Alex Deroza

  I use this batch file for starting my stream :

  echo starting transocding from native uhd2160 to fhd format.. Host: YouTube
  
  ffmpeg ^
  
      -loglevel -8 ^
  
      -f rawvideo -s:v 1920x1080 -r 60 -pix_fmt nv12 ^
  
      -f dshow -i video="Game Capture 4K60 Pro Video 01":audio="Game Capture 4K60 Pro Audio 01" ^
  
      -c:v h264_nvenc ^
  
      -level:v 4.2 ^
  
      -profile:v high ^
  
      -preset:v hq ^
  
      -b:v 8.8M ^
  
      -color_range 2 ^
  
      -colorspace bt709 ^
  
      -bf 0 ^
  
      -g 60 ^
  
      -2pass 1 ^
  
      -rc:v cbr ^
  
      -coder cabac ^
  
      -acodec aac ^
  
      -ab 128k ^
  
      -movflags +faststart ^
  
      -f flv rtmp://a.rtmp.youtube.com/live2/xxxx-xxxx-xxxx-xxxx && pause

  Stream on YouTube starts fine, and plays with good quality. But stream stops after when "Buffer Health" emptied. And in this situation Youtube says that "Video output low", "YouTube is not receiving enough video to maintain smooth streaming. As such, viewers will experience buffering."

  I can’t understand why it happens. Because I’ve a good ISP, and over 30 mbps inet to YT services within 90ms latency to them.

  YouTube is not receiving enough video to maintain smooth streaming. As
  such, viewers will experience buffering.

  Your encoder is sending data faster than realtime (multipleseconds of
  video each second). You must rate limit your livevideo upload to
  approximately 1 second of video each second.

  The stream’s current bitrate (5730.00 Kbps) is lower than the
  recommended bitrate. We recommend that you use a stream bitrate of
  4500 Kbps.

• Pass ID3D11Texture2D back buffer to libx264 encoder

  17 janvier 2019, par Chris Sixsmith

  I’m writing a C++ program to encode frames from a DirectX game to the H.264/MPEG-4 AVC format. I am using libx264 alone with no other dependencies at the moment.

  I have a ID3D11Texture2D* resolved back buffer of the next game frame. I need to somehow copy this into the x264_picture input (apparently YUV420P format according to limited help I’ve found) but I cannot find any way to do so online.

  Here is my code at the moment :

  void Fx264VideoEncoder::Fx264VideoEncoderImpl::InitFrameInputBuffer(const FTexture2DRHIRef& BackBuffer, FFrame& Frame)
  
  {
  
      x264_picture_alloc(Frame.InputPicture, X264_CSP_I420, x264Parameters.i_width, x264Parameters.i_height);
  
  
  
      // We need to take the back buffer and convert it to an input format that libx264 can understand
  
      {
  
          ID3D11Texture2D* ResolvedBackBufferDX11 = (ID3D11Texture2D*)(GetD3D11TextureFromRHITexture(Frame.ResolvedBackBuffer)->GetResource());
  
          EPixelFormat PixelFormat = Frame.ResolvedBackBuffer->GetFormat();
  
  
  
          // ...?
  
      }
  
  }