Recherche avancée

Sur d’autres sites (8924)

• Understanding The Dreamcast GD-ROM Layout

  24 mars 2022, par Multimedia Mike — Sega Dreamcast

  I’m finally completing something I set out to comprehend over a decade ago. I wanted to understand how data is actually laid out on a Sega Dreamcast GD-ROM drive. I’m trying to remember why I even still care. There was something about how I wanted to make sure the contents of a set of Dreamcast demo discs was archived for study.

  
  
  

  I eventually figured it out. Read on, if you are interested in the technical details. Or, if you would like to examine the fruits of this effort, check out the Dreamcast demo discs that I took apart and uploaded to the Internet Archive.

  If you care to read some geeky technical details of some of the artifacts on these sampler discs, check out this followup post on Dreamcast Finds.

  Motivation
  Why do I still care about this ? Well, see the original charter of this blog above. It’s mostly about studying multimedia formats, as well as the general operation of games and their non-multimedia data formats. It’s also something that has nagged at me ever since I extracted a bunch of Dreamcast discs years ago and tried to understand why the tracks were arranged the way they were, and how I could systematically split the files out of the filesystem. This turns out not to be as easy as it might sound, even if you can get past the obstacle of getting at the raw data.
  

  CD/CD-ROM Refresher
  As I laid out in my Grand Unified Theory of Compact Disc, every compact disc can be viewed conceptually as a string of sectors, where each sector is 2352 bytes long. The difference among the various CD types (audio CDs, various CD-ROM types) boils down to the format of contents of the 2352-byte sectors. For an audio CD, every sector’s 2352 bytes represents 1/75 of a second of CD-quality audio samples.

  Meanwhile, there are various sector layouts for different CD-ROM modes, useful for storing computer data. This post is most interested in “mode 1/form 1”, which uses 2048 of the 2352 bytes for data, while using the remaining bytes for error detection and correction codes. A filesystem (usually ISO-9660) is overlaid on these 2048-byte sectors in order to create data structures for organizing strings of sectors into files.

  A CD has between 1 and 99 tracks. A pure CD-ROM will have a single data track. Pure audio CDs tend to have numerous audio tracks, usually 1 per song. Mixed CDs are common. For software, this usually manifests as the first track being data and containing an ISO-9660 filesystem, followed by a series of audio tracks, sometimes for in-game music. For audio CDs, there is occasionally a data track at the end of the disc with some extra media types.

  GD-ROM Refresher
  The Dreamcast used optical discs called GD-ROMs, where the GD stands for “gigadisc”. These discs were designed to hold about 1 gigabyte of data, vs. the usual 650-700MB offered by standard CD solutions, while using the same laser unit as is used for CDs. I’m not sure how it achieved this exactly. I always assumed it was some sort of “double density” sector scheme. According to Wikipedia, the drive read the disc at a slower rate which allowed it to read more data (presumably the “pits” vs. “lands” which comprise the surface of an optical disc). This might be equivalent to my theory.

  The GD-ROM discs cannot be read in a standard optical drive. It is necessary to get custom software onto the Dreamcast which will ask the optical hardware to extract the sectors and exfiltrate them off of the unit somehow. There are numerous methods for this. Alternatively, just find rips that are increasingly plentiful around the internet. However, just because you might be able to find the data for a given disc does not mean that you can easily explore the contents.

  Typical Layout Patterns
  Going back to my study of the GD-ROM track layouts, 2 clear patterns emerge :

  All of the game data is packed into track 3 :

  
  
  

  Track 3 has data, the last track has data, and the tracks in between contain standard CD audio :

  
  
  

  Also, the disc is always, always 100% utilized.

  Track 1 always contains an ISO-9660 filesystem and can be read by any standard CD-ROM drive. And it usually has nothing interesting. Track 3 also contains what appears to be an ISO-9660 filesystem. However, if you have a rip of the track and try to mount the image with standard tools, it will not work. In the second layout, the data follows no obvious format.

  Cracking The Filesystem Code
  I figured out quite a few years ago that in the case of the consolidated data track 3, that’s simply a standard ISO-9660 filesystem that would work fine with standard ISO-9660 reading software… if the data track were located beginning at sector 45000. The filesystem data structures contain references to absolute sector numbers. Thus, if it were possible to modify some ISO-9660 software to assume the first sector is 45000, it ought to have no trouble interpreting the data.

  
  
  

  How about the split data track format ? Actually, it works the same way. If all the data were sitting on its original disc, track 3 would have data structures pointing to strings of contiguous sectors (extents) in the final track, and those are the files.

  To express more succinctly : track 3 contains the filesystem root structure and the directory structures, while the final track contains the actual file data. How is the filesystem always 100% full ? Track 3 gets padded out with 0-sectors until the beginning of any audio sectors.

  
  
  

  Why Lay Things Out Like This ?
  Why push the data as far out on the disc as possible ? A reasonable explanation for this would be for read performance. Compact discs operate on Constant Linear Velocity (CLV), vs. Constant Angular Velocity (CAV). The implication of this is that data on the outside of the disc is read faster than data on the inside. I once profiled this characteristic in order to prove it to myself, using both PC CD drives as well as a Dreamcast. By pushing the data to the outer sectors, graphical data gets loaded into RAM faster, and full motion videos, which require a certain minimum bitrate for a good experience, have a better guarantee that playback will be smooth.

  Implications For Repacking
  Once people figured out how to boot burned CDs in the Dreamcast, they had a new problem : Squeeze as much as 1 gigabyte down to around 650 megabytes at the most. It looks like the most straightforward strategy was to simply rework the filesystem to remove the often enormous amount of empty space in track 3.

  My understanding is that another major strategy is to re-encode certain large assets. Full motion video (FMV) assets are a good target here since the prevailing FMV middleware format used on Sega Dreamcast games was Sofdec, which is basically just MPEG-1 video. There is ample opportunity to transcode these files to lower bitrate settings to squeeze some bits (and a lot of visual quality) out of them.

  Further, if you don’t really care about the audio tracks, you could just replace them with brief spurts of silence.

  Making A Tool
  So I could make a tool that would process these collections of files representing a disc. I could also adapt it for various forms that a Dreamcast rip might take (I have found at least 3 so far). I could eventually expand it to handle lots of other disc formats (you know, something like Aaru does these days). And that would have been my modus operandi perhaps 10 or more years ago. And of course, the ambitious tool would have never seen daylight as I got distracted by other ideas.

  I wanted to get a solution up and running as quickly as possible this time. Here was my initial brainstorm : assemble all the tracks into a single, large disc while pretending the audio tracks consist of 2048-byte sectors. In doing so, I ought to be able to use fuseiso to mount the giant image, with a modification to look for the starting sector at a somewhat nonstandard location.

  To achieve the first part I wrote a quick Python script that processed the contents of a GDI file, which was stored alongside the ISO (data) and RAW (audio) track track rips from when I extracted the disc. The GDI is a very matter-of-fact listing of the tracks and their properties, e.g. :

  5
  1 0 4 2048 track01.iso 0
  2 721 0 2352 track02.raw 0
  3 45000 4 2048 track03.iso 0
  4 338449 0 2352 track04.raw 0
  5 349096 4 2048 track05.iso 0
  

  track number / starting sector / track type (4=data, 0=audio) / bytes per sector / filename / ??

  The script skips the first 2 filenames, instead writing 45000 zero sectors in order to simulate the CD-compatible area. Then, for each file, if it’s an ISO, append the data to the final data file ; if it’s audio, compute the number of sectors occupied, and then append that number of 2048-byte zero sectors to the final data file.

  Finally, to interpret the filesystem, I used an old tool that I’ve relied upon for a long time– fuseiso. This is a program that leverages Filesystem in Userspace (FUSE) to mount ISO-9660 filesystems as part of the local filesystem, without needing root privileges. The original source hasn’t been updated for 15 years, but I found a repo that attempts to modernize it slightly. I forked a version which fixes a few build issues.

  Anyway, I just had to update a table to ask it to start looking for the root ISO-9660 filesystem at a different location than normal. Suddenly, after so many years, I was able to freely browse a GD-ROM filesystem directly under Linux !

  Conclusion And Next Steps
  I had to hack the fuseiso3 tool a bit in order to make this work. I don’t think it’s especially valuable to make sure anyone can run with the same modifications since the tool assumes that a GD-ROM rip has been processed through the exact pipeline I described above.

  I have uploaded all of the North American Dreamcast demo discs to archive.org. See this post for a more granular breakdown of what this entails. In the course of this exercise, I also found some European demo discs that could use the same extraction.

  What else ? Should I perform the same extraction experiment for all known Dreamcast games ? Would anyone care ? Maybe if there’s a demand for it.

  Here is a followup on the interesting and weird things I have found on these discs so far.

  The post Understanding The Dreamcast GD-ROM Layout first appeared on Breaking Eggs And Making Omelettes.

• FFMPEG Api conversion from YUV420P to RGB produces strange output

  20 novembre 2024, par fasc8

  I'm using the FFMPEG Api in Rust to get RGB images from video files.

  


  While some videos work correct and I get the frames back as expected, some work not. Or at least the result is not the way I expected it to be.

  


  The code I use in Rust :

  


  ffmpeg::init().unwrap();

let in_ctx = input(&Path::new(source)).unwrap();
let input = in_ctx
    .streams()
    .best(Type::Video)
    .ok_or(ffmpeg::Error::StreamNotFound)?;

let decoder = input.codec().decoder().video()?;

let scaler = Context::get(
    decoder.format(),
    decoder.width(),
    decoder.height(),
    Pixel::RGB24,
    decoder.width(),
    decoder.height(),
    Flags::FULL_CHR_H_INT | Flags::ACCURATE_RND,
)?; // <--- Is basically sws_getContext

// later to get the actual frame
let mut decoded = Video::empty();
if self.decoder.receive_frame(&mut decoded).is_ok() {
    let mut rgb_frame = Video::empty();
    self.scaler.run(&decoded, &mut rgb_frame)?; // <--- Does sws_scale
    println!("Converted Pixel Format: {}", rgb_frame.format() as i32);
    Ok(Some(rgb_frame))
}


  


  Which should roughly translate to C like so :

  


  // Get the context and video stream
SwsContext * ctx = sws_getContext(imgWidth, imgHeight,
                              imgFormat, imgWidth, imgHeight,
                              AV_PIX_FMT_RGB24, 0, 0, 0, 0);
sws_scale(ctx, decoded.data, decoded.linesize, 0, decoded.height, rgb_frame.data, rbg_frame.linesize);


  


  And like I said earlier, sometimes it works fine and I get the expected frame back. But sometimes I get something like this :
Weird result image

  


  I saved the images as .ppm files for quick visual comparison. I used this method, which basically writes the bytes to a file with a simple .ppm header :

  


  fn save_file(frame: &Video, index: usize) -> std::result::Result<(), std::io::Error> 
{
    let mut file = File::create(format!("frame{}.ppm", index))?;
    file.write_all(format!("P6\n{} {}\n255\n", frame.width(), frame.height()).as_bytes())?;
    file.write_all(frame.data(0))?;
    Ok(())
}


  


  Here you can see that on the left side there is a good image result vs. on the right side there is a bad image result.
Comparison of the .ppm files

  


  To come to the question now :

  


  Why is this happening. I tested everything on my side and the only thing left is ffmpeg conversion. FFMPEG seems to convert these two test files differently even though it reports YUV420P as format for both. I cannot figure out what the difference may be...

  


  Here the info for the two video files i used :

  


  Good video file :

  


  General
Complete name                            : /mnt/smb/Snapchat-174933781.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/mp42)
File size                                : 1.90 MiB
Duration                                 : 9 s 612 ms
Overall bit rate                         : 1 661 kb/s
Encoded date                             : UTC 2021-07-28 22:09:36
Tagged date                              : UTC 2021-07-28 22:09:36
eng                                      : -180.00

Video
ID                                       : 512
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3.1
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 1 frame
Format settings, GOP                     : M=1, N=30
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 9 s 598 ms
Bit rate                                 : 1 597 kb/s
Width                                    : 480 pixels
Height                                   : 944 pixels
Display aspect ratio                     : 0.508
Frame rate mode                          : Variable
Frame rate                               : 29.797 FPS
Minimum frame rate                       : 15.000 FPS
Maximum frame rate                       : 30.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.118
Stream size                              : 1.83 MiB (96%)
Title                                    : Snap Video
Language                                 : English
Encoded date                             : UTC 2021-07-28 22:09:36
Tagged date                              : UTC 2021-07-28 22:09:36
Color range                              : Full
colour_range_Original                    : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.601
transfer_characteristics_Original        : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC

Audio
ID                                       : 256
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 9 s 612 ms
Bit rate mode                            : Constant
Bit rate                                 : 62.0 kb/s
Channel(s)                               : 1 channel
Channel layout                           : C
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 73.3 KiB (4%)
Title                                    : Snap Audio
Language                                 : English
Encoded date                             : UTC 2021-07-28 22:09:36
Tagged date                              : UTC 2021-07-28 22:09:36


  


  Bad video file :

  


  General
Complete name                            : /mnt/smb/Snapchat-1989594918.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/mp42)
File size                                : 2.97 MiB
Duration                                 : 6 s 313 ms
Overall bit rate                         : 3 948 kb/s
Encoded date                             : UTC 2019-07-11 06:43:04
Tagged date                              : UTC 2019-07-11 06:43:04
com.android.version                      : 9

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Baseline@L3.1
Format settings                          : 1 Ref Frames
Format settings, CABAC                   : No
Format settings, Reference frames        : 1 frame
Format settings, GOP                     : M=1, N=30
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 6 s 313 ms
Bit rate                                 : 3 945 kb/s
Width                                    : 496 pixels
Height                                   : 960 pixels
Display aspect ratio                     : 0.517
Frame rate mode                          : Variable
Frame rate                               : 29.306 FPS
Minimum frame rate                       : 19.767 FPS
Maximum frame rate                       : 39.508 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.283
Stream size                              : 2.97 MiB (100%)
Title                                    : VideoHandle
Language                                 : English
Encoded date                             : UTC 2019-07-11 06:43:04
Tagged date                              : UTC 2019-07-11 06:43:04
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC


  


  Or as a diff image :

  


  The problem is that I am not that familiar with ffmpeg yet I don't know all the quirks it has.

  


  I hope someone can point me in the right direction.

  


• Evolution of Multimedia Fiefdoms

  1er octobre 2014, par Multimedia Mike — General

  I want to examine how multimedia fiefdoms have risen and fallen through the years.

  
  
  

  Back in the day, the multimedia fiefdoms were built around the formats put forth by competing companies : there was Microsoft/WMV, Apple/MOV, and Real/RM as the big contenders. On2 always wanted to be a player in this arena but could never quite catch a break. A few brave contenders held the line for open source and also for the power users who desired one application that could handle everything (my original motivation for wanting to get into multimedia hacking).

  The computer desktop was the battleground for internet-based media stream. Whatever happened to those days ? Actually, if memory serves, Flash-based video streaming stepped on all of them.

  Over the last 6-7 years, the battleground has expanded to cover mobile devices, where Flash’s impact has… lessened. During this time, multimedia technology pretty well standardized on a particular stack, namely, the MPEG (MP4/H.264/AAC) stack.

  The belligerents in this war tried for years to effectively penetrate new territory, namely, the living room where the television lived. This had been slowgoing for years due to various user interface and content issues, but steadily improved.

  Last April, Amazon announced their entry into the set-top box market with the Fire TV. That was when it suddenly crystallized for me that the multimedia ecosystem has radically shifted. Now, the multimedia fiefdoms revolve around access to content via streaming services.

  Off the top of my head, here are some of the fiefdoms these days (fiefdoms I have experience using) :

  • Netflix (subscription streaming)
  • Amazon (subscription, rental, and purchased streaming)
  • Hulu Plus (subscription streaming)
  • Apple (rental and purchased media)

  I checked some results on Can I Stream.It ? (which I refer to often) and found a bunch more streaming fiefdoms such as Google (both Play and YouTube, which are separate services), Sony, Xbox 360, Crackle, Redbox Instant, Vudu, Target Ticket, Epix, Sony, SnagFilms, and XFINITY StreamPix. And surely, these are probably just services available in the United States ; I know other geographical regions have their own fiefdoms.

  What happened ?

  When I got into multimedia hacking, there were all these disparate, competing ecosystems. As a consumer, I didn’t care where the media came from, I just wanted to play it. That’s what inspired me to work on open source multimedia projects. Now I realize that I have the same problem 10-15 years later : there are multiple competing ecosystems. I might subscribe to fiefdoms X and Y, but am frustrated to learn that something I’d like to watch is only available through fiefdom Z. Very few of these fiefdoms can be penetrated using open source technology.

  I’m not really sure about the point about this whole post. Multimedia technology seems really standardized these days. But that’s probably just my perspective because I have spent way too long focusing on a few areas of multimedia technology such as audio and video coding. It’s interesting that all these services probably leverage the same limited number of codecs. Their differentiation comes from the catalog of content that each is able to license for streaming. There are different problems to solve in the multimedia arena now.