BUU has asked for the wisdom of the Perl Monks concerning the following question:

Heres my problem, i have a file thats in, what i assume, to be a binary format. Basically the program takes a file with a series of plain alphanumerics seperated by new lines and then translates it into this binary format. The reason im thinking its binary is b/c its got all these funky characters that definately dont appear on the keyboard. Anyways, my problem is that i need to figure out how to translate to, and back from, the binary format, using perl perferably. I know what the series of characters it takes to produce the ouput, but i cant figure out how to produce the output. Anyone out there have some idea of how i could easily figure this out? Im assuming i would use pack/unpack some how?

Heres an exmaple (i hope) of a text list and what it translates into binarily.
List:
//NAME: Untitled Deck
        4 Black Lotus

[download]
And heres the contents of the binary file that produces:
Actually this is not the list, because it "Cannot cut, copy, or drag and drop text containing null (code = 0) characters".
So, here is a link to where i uploaded the file.

Also if anyone is curious, this is indeed for Magic: The Gathering (a card game), specifically for Apprentice

Replies are listed 'Best First'.
Re: 'parsing' a binary file?
by jmcnamara (Monsignor) on May 30, 2002 at 23:51 UTC

    Here is what the binary information in your file looks like: 
     00: 04 00 FF FF 0D 55 6E 74 69 74 6C 65 64 20 44 65 ..ÿÿ.Untitled De
 10: 63 6B 6C 8A 76 3D DF 16 04 00 00 00 00 00 00 00 cklŠv=ß.........
 20: A8 07 27 01 00 00 00 00 00 00 00 00 15 00 02 00 ¨.'.............
 30: 04 00 14 3C 6A 66 45 59 6F 3F 56 51 5F 52 57 56 ...<jfEYo?VQ_RWV
 40: 4F 43 43 4C 3E 72 5A 00 00 00 00 00 00 00 A8 07 OCCL>rZ.......¨.
 50: 27 01 00 00 00 00 00 00 00 00 15 00             '...........

    [download]

    Apart from NAME it is hard to see any correlation with the input text.

    If you can get some documentation about the binary file format then you will be able to read and write similar files using unpack and pack.

    Otherwise, you will spend long lonely nights, like me, staring into a hex editor, wondering what this has to do with programming in Perl. ;-)

    --
    John.

[reply]
[d/l]
      So basically, give up before i go insane?
[reply]
        No, close :)

        Your best bet is probably to try to contact the author of the original program, explain to them what you're trying to do, and ask for their data structure.

        Failing that, I remember seeing an article on cracking Turbo Pascal data structures years ago. The jist of it was to know how the data structures were stored. Different variables had different byte representations.

        So, what I'd do if all else failed would be to run the hex editor on the original program, see what language it's written in, and then go from there and see if that gets you anywhere.

        Hope that helps!

        Couple of links from a google search that may help as well:

        REC Decompiler Home Page
        <a href="http://tsehp.cjb.net/>Last Fravia's mirror of Reverse code engineering

        Some people fall from grace. I prefer a running start...

[reply]

        In short, yes. :-)

        Seriously, it is possible to reverse engineering a binary file format but it is a long slow process.

        On the other hand, if there is documentation available then dealing with binary files can be quite easy thanks to the venerable pack and unpack.

        --
        John.

[reply]
        A reply falls below the community's threshold of quality. You may see it by logging in.
Re: 'parsing' a binary file?
by graff (Chancellor) on May 30, 2002 at 22:57 UTC
    Basically the program takes a file with a series of plain alphanumerics seperated by new lines and then translates it into this binary format.

    Um, which program is it that does this? (Does it matter?)

    i need to figure out how to translate to, and back from, the binary format

    Do you mean you need to reverse-engineer whatever program it is that you're talking about?

    Alas, it seems that the link to the binary data does not work as intended -- I get a blank page, and "view page source" also yields a blank page.

    How about starting with a simple octal or hex dump of the file? The unix "od" command is probably best for this, since it gives you the flexibility of treating binary data as bytes, 16-bit words, etc. But of course, you want to know how to do it in Perl, so start with this quick-and-dirty method for a hex dump of byte codes: 

    $/=undef;
open(I,"your_binary.file");
$d=<I>;
$o=join("\n",map{sprintf("%x",ord)} split(//,$d))."\n";
print $o;

    [download]
    Using unpack is fun too, but you really want to try that out for yourself -- experimentation is the best teacher...

    update: removed pointless, stupid remark

[reply]
[d/l]
    A reply falls below the community's threshold of quality. You may see it by logging in.

Back to Seekers of Perl Wisdom