USF Frequently Asked Questions

I never really understood what "frequently asked" meant until I started getting the same email, phrased slightly differently, again and again... Here's a "best of hcs tech support" listing.
Q: Why do I get "what's in code" errors?
Short Answer: Get a new version of the USF set from a reputable mirror (such as one listed on USF Central).

Long Answer: there was a change in the USF format around the time I started numbering the versions 0.xx instead of build xx, and this reflected what I though of as the transition into the final stage of the USF format. The change was made to reflect a complaint by Neill Corlett that the "code" section of a PSF should be valid zlib compressed data, and my noncompliance might cause inoperability with some PSF tools, which are supposed to be (and are, thanks to Neill's foresight) universal. As I genuinely didn't want to incorporate zlib compression into USF I used the fact that the "reserved" section has no restrictions on the format of its data, so I put all of the data in this section (formerly in a USF the ROM was in code and the save state was in the reserved section).

By this time, unfortunately, I had released early versions of my USF sets and 64th Note, so the obsolete player and sets were distributed to some extent. Now I could have easily added support to new versions of 64th Note to support the "old" format, but I preferred to strictly support the new, "official" version. My mistake was to provide a useless error message when an old-style file was encountered, "What's in code" is generated when the USF parser finds any data in the "code" section. The error really should read "You may be using an ancient USF, get a newer version."

Q: Why don't you release USFs anymore?
A: I simply can't get any more USFs done. A few times I've sat down and tried to work on a game but I've never gotten far enough without becoming frustrated. Initially I was driven by my love of the music and the hacker ethic of wanting to see if I could accomplish something, but that apparently isn't a strong enough drive for me anymore.
Q: How can I rip USFs?
Short answer: I really, really hate to discourage anyone, but the truth is that I can barely understand the process myself, much less explain it. All the other rippers you've seen already had experience with PSF rippping or N64 hacking. I really can't even point you to anything to learn short of saying "study MIPS assembly and start hacking games".

Long answer: Here's what I wrote to someone to try to describe the ripping process that I use:

I look for anything that I know might be related to music. For example in Majora's Mask I knew that there had to be a routine in the intro (where it tells the backstory) to start playing that music, so I looked for the into text, then looked for the routine that read the intro text, then worked backwards until I found the intro function, then looked through that for something that looked like a music select function. Music modifiers are useful, too, but they'll still only give you a place to start looking. When a game uses the standard audio library you can start by looking for references to al functions that have the word "Seq" (for sequence) in them. Another method involves searching for a thread that handles audio (Mario 64, for example, had only two main threads, audio and video (and a "gameplay" thread), most games are much more complicated) and looking for stuff that affects that, in Mario 64's case I made a modification to Project 64 to record all variables that the audio thread read and that helped me find what I was looking for. Threads can be found by looking for calls to osCreateThread, the a2 argument is the address of the body of the thread.

One obstacle I encounter frequently is that games generally don't have a direct connection between the song select function and the functions that actually load the music. There is generally a queue where commands to play sound effects, change volume, and play music are written to. There will be a bunch of functions that write to this, usually through a single main function that actually does the writing, and a single big function to read it, which generally uses a jump table (which has a way of screwing up automated analysis such as that done by IDA).

In my experience every game has had a single function to start playing a song, it generally takes the song number as a0 or a1. If I locate a function that I think is this I modify Project 64 so that when it hits that function it will display the arguments with a line like the following in ExecuteInterpreterOpcode:

if (PROGRAM_COUNTER==whateveraddress) DisplayError("a0: %08x a1: %08x",GPR[4].UW[0],GPR[5].UW[0]);

I then run the game and see if there is any correspondence between when songs change, what they change to, and the value of the arguments. If there seems to be a pattern I'd force the argument to a certain value and see if by doing so I can fix the song being played.

Once I've located a song select function I pretty much look for the very first time it is called, usually in a game's intro, and I try to disable everything else from running. This takes various forms: I can try to disable non-audio threads by removing their loop commands (usually an unconditional branch), change the audio thread to use a nonblocking osRecvMesg (a2 parameter is 1, I believe, either that or 0) so that it won't yield control to other threads, or I can go about removing function calls that don't affect the music.

From then on it's easy :)

For most of this analysis it will be inifnitely helpful to have IDA Pro, if you've looked through someone42's document you'll know how to find it. The .sig files for the N64 libs go in the sigs/mips subdirectory.

Maybe that'll give you someplace to start...

Q: Is an automated USF ripper or logger possible?
I don't think it could ever be fully automated (like Saptapper), but as a good many games use the standard Smashy driver it may be possible to isolate that driver and insert the data from individual games without having to go through all the deprogramming for each one.

As for a logger I don't know, its hard enough to tell what's needed when you're looking at the code, just logging what's relied on for music would be very difficult if you don't want monstrously huge files (think GYM). I delibrately didn't go that direction with USF.

Q: Why are the links down?
A: I have a rather small upload limit on freeshell.org, not nearly enough to handle the amount of traffic USF Central demands (as I quickly learned on the first day I released anything), so instead I hosted the larger files on my Rutgers University web space. Recently I've gotten into some trouble and my account has been disabled, so that won't be back again until at least Summer 2005.
Q: Why does 64th Note crash at random?
A: If I knew the answer to that I'd have fixed it by now. Clearly there is a problem somewhere in the recompiler CPU (both the main CPU and RSP) with initialization, as it shouldn't matter at all what you were playing before, but I've been unable to locate it.