kong BIGFILE

[player98198]

heres a pic of the app for those of u who can't run it
i know the files are in incorrect order, i didn't have much time to fix that bug. but if anyone is still interested i could fix it up some more, with added features such as
right click extract
indivual file extract
custom "@extracted" folder rebuilder
sorting

[Hoss]

Permanent link here:
http://www.dslinux.org/bf_unpack_v11.rar

[xor37h]

Hey guys (and gals?)

I took at little look at this, and whipped up a tool (with full source as always) that you might have fun with

You can download it here: http://hitmen.c02.at/files/releases/xbox360/unpack02.zip

JADE Game Engine .bin file unpacker v0.2 by xor37h/hitmen
using LZO real-time data compression library (v2.02, Oct 17 2005).

Usage: unpack.exe infile.bin outfile.bin

ok, the first version was really too quick a hack, it didnt work with all the .bin files
and the code was ugly as hell

Here we go with an updated version that supports ALL the .bin files, also the big ones!

---

Just a quick hack to unpack the .bin files from the extracted kingkong .bf

Q: JADE Game Engine ?
A: Yep thats the name, check http://www.mobygames.com/game_group/sheet/gameGroupId,4464/

Q: whats minilzo ?
A: A nice little opensource compression library avaliable at http://www.oberhumer.com/opensource/lzo/

Q: How did you figure out that it was using minilzo ?
A: I downloaded the release called: Peter.Jacksons.King.Kong.DEMO.XBOX-XD, and reversed the .xbe

Q: I see that the retail releases come with a file named "BUG" in its magic, whats that?
A: Its ubisofts retail bigfile format, its the "BIG" format with some xor scrambling, you can
   reverse the XBOX-XD .xbe for more information, they were nice enough to link in support for
   both formats.

Nothing more to say, the code is not amazing, it was hacked together in a few minutes, but it works
and this will maybe lead the way to even more nice things

Have fun!

 ... xor37h signing off

Yep you can decompress the .bin files with this, shouldnt be much of a problem to do a recompressor either.. have fun!

[th0mas]

nice work xor37h!

[xor37h]

ok, the first version was really too quick a hack, it didnt work with all the .bin files
and the code was ugly as hell

Here we go with an updated version that supports ALL the .bin files, also the big ones!

get it from http://hitmen.c02.at/files/releases/xbox360/unpack02.zip

Enjoy, xor37h

[th0mas]

For anyone interested in code related to how the directory structure in the BF files works (hi DarkNeo ) take a look at:

http://th0mas.sixbit.org/dumpdirs.c

takes as arguments the filename of a .bf file, and the number of directory entries to process - while looking I never found this number hardcoded in any header and didn't write a check to see if it read a blank entry - there is around 8000 directory entries so feel free to use a number around that size.  If you use a number too big the trailing entries will be obviously garbage.

[Dark_Neo]

Got a link to bfcommon.h?

Btw the number of directories is at 0xC and 0x30. Not sure why they repeat the file + directory count twice in the header...

[th0mas]

http://th0mas.sixbit.org/bfcommon.h - sorry about that.

[xor37h]

I had some freetime today, and hacked this together

You can grab it from:
   http://hitmen.c02.at/files/releases/xbox360/jadepack.zip

JADE Game Engine .bin file pack-tool v1.0 by xor37h/hitmen
using LZO real-time data compression library (v2.02, Oct 17 2005).

    Usage: jadepack [-option] infile outfile

 Examples: jadepack -c ff801b17.dec ff801b17.bin
           jadepack -d ff801b17.bin ff801b17.dec

 Switches:
    -c           compress
    -d           decompress

With this tool you can decompress and recompress the extracted .bin files
from the extracted KingKongTheGame_clean.bf bigfile on the 360 kiosk disc.

It should work on other packed .bin files using the JADE game engine.

depacking -> repacking will not produce a 1:1 identical file due to the
fact that the lzo algorithm gives different compressed output depending
on library version used aswell as the machine it is running on.

depacking -> repacking -> depacking will produce a 1:1 identical file.

Also have no fear as the lzo manual states that even the latest miniLZO
is backwards compatible.

I have been unable to find the exact version of minilzo the Ubisoft guys
used, since they removed version and date strings from the library they
used, allthough I suspect they might be using an old version 1.06.

I decided to use version 2.02 as its backwards compatible and works on
64-bit processors.

The file size of the repacked file will often (but not always) be the same.

I have allready tested the following:

 1. extract KingKongTheGame_clean.bf
 2. unpack a .bin file.
 3. change some strings
 4. repack the .binfile
 5. rebuild KingKongTheGame_clean.bf
 6. load it up on a 360 and see the changes without any problems.

Special thanks to groepaz/hitmen for porting and testing on linux.

notice:
  jadepack uses miniLZO v2.02 (c) Markus Franz Xaver Johannes Oberhumer

Enjoy guys!

[th0mas]

So has anyone tried to gently break the BF file format and then test the results?

For example, try going to the file structure offset at 0x138c0 and make a filename be 65 bytes long (or 64 followed by a null), then burn and run the disk.  You could also try the same thing with a directory from the structure at 0xea644.

Another idea would be to attempt an array read over bounds by changing the parentDir entry for a directory entry (see it's position in the struct by looking at dumpdirs.c and bfcommon.h)

Tom

[Dark_Neo]

One thing that should be pointed out, which is quite confusing. th0mas's tool (and my plugin for my app I'm just finishing up, will post it soon) extract the files without the four bytes at the start (the size), player98198's tool however does extract these bytes, which is what xor37h's BIN unpacker is based on. Now looking at the way the BF file is structured I agree with th0mas that these four bytes shouldn't be extracted, because the size is mentioned twice (like the number of files and directories is mentioned twice) and these sizes both indicate that these four bytes shouldn't be extracted. However the BIN file is structured with multiple blocks, each starting with:

struct {
   uint32 fileSize,
   uint32 unpackedSize,
   uint32 packedSize
}

But each block after the first (assuming the file size bytes are extracted) contains 0x00000000 as the file size. If the four bytes aren't supposed to be extracted this could just be a dword padding, which is my current theory.

Hope I made sense, in a bit of a rush. What does everyone else think?

[xor37h]

Hi Dark_Neo,

I belive you are wrong.. let me explain...

In my code I do stuff like:

Code:

            // pad to blocksize 2048
            header.filesize = ftell(fout);
            i = 2048 - (header.filesize % 2048);
            memset(out,0,i);
            fwrite(out,1,i,fout);

            // update header.filesize in the start
            header.filesize += i-4;           // YES i know it says -4, thats the remaining file size _after_ the dword is read.
            fseek(fout, 0, SEEK_SET);
            fwrite(&header.filesize,1,4,fout);

The .bin files are always a multitude of 2048 (0x800) when it includes that filesize dword, this is due to the fact that we load from disc, and the sector size of such a disc is 2048..

Well actually a dvd sector size if 2064 bytes [ID.4][IED.2][CPR_MAI.6][DATA.2048][ECC.4] but out of that only 2048 is data, and reading a full sector is nearly as fast as say, reading half a sector, simply due to the drives mechanies.

The reason the filesize is there is for the decompression routine, I know this for a fact, since I reversed on the xbox1 .xbe to figure out the compression format.

Anyways, youre natururally welcome to use my code to integrate .bin de/compression it into your bigfile manipulation tools, as the .bin format is a intermediate chunked and padded format, the file wont actually be truely extracted before its unpacked.

[Hoss]

if you guys got together and made one package that does it all, that would be great.  I will host if needed.

[th0mas]

Hi xor37h,

I can see how it might make sense with the .bin, but it makes no sense for other files, such as the wave files inside the Sound_Common.bf..  If we don't include that first dword then we have a properly formed WAVE file.  Otherwise we don't.  I'm assuming that the .bin unpacker in the xbox1 XBE that you looked at was unpacking the bin file at the same time as reading it from the .bf?  Or was it a two step process - extract the .bin from the .bf (including the starting filesize dword), then decompress.  Because if it's doing it all at once then there is no real "proper" way to do it so to speak, except for someone to include .bin de/compressing as a part of their bf unpacker.

Cheers,
Tom

[player98198]

that would be some tedious work to do.
im thinking it would require a few different steps

extract file
->check header to see if correct file
->if correct run decompress tool
->go to next file

its easier said than done. but it would help for multple files

[Assassin X]

heres a pic of the app for those of u who can't run it
i know the files are in incorrect order, i didn't have much time to fix that bug. but if anyone is still interested i could fix it up some more, with added features such as
right click extract
indivual file extract
custom "@extracted" folder rebuilder
sorting

Sorry to bump this after so long, heck I am not sure what this forum is under besides "xbox". I found this after hours of searching! Maybe you "1337" people can help me out.

I bumped this because I am trying to extract/access...etc "bigfiles" from a pc version of "Tomb Raider: Legend". Any game that comes out I try to get its music because I just love game music! But I can't figure out how to get the files out! I seen this extractor but it doesn't work, obviously I realized its for XBOX related stuff. Anyone know a BIGFILE extractor for PC items?

[Assassin X]

Seems at a Russian site someone figured out how to extract the music for this game only. http://www.ag.ru/files/software/373/16521/agc

Its ashame there are so many diffrent file types, game music is so awsome.

[annerajb]

i know this tread is old but would like to contribute something here the sounds reference u found in that file or the whole file could be a debug dump left in the game and those could be the things that while the game loaded spitted out to the debug log. not sure about this also the xbox360 not sure if always do but i thing yes uses xact xbox auditioning content tool for sounds so the sounds need to first be put in a xgs and xws xsb those files can be generated in the pc using the xact tool included in directxsdk so if u wanna take a try on recreating that file and trying to load a custom sound go ahead. hope that helped