elbow has asked for the wisdom of the Perl Monks concerning the following question:

Hi all

I've got a text file that has been created by a (argh!) mainframe process. Unfortunately the program has failed to remove all packed decimals fields, so some of the characters shown on the text file are invalid/unreadable/wrong e.g.

ABBE02`pr%90073462

I have no idea what you'd call these characters (are they hex?) but I'm trying to put together something that could convert them back to readable values.

Has anybody any ideas/suggestions? (before anybody says anything - I've amended the mainframe job to stop it reoccurring!)

Thanks

elbow

Replies are listed 'Best First'.
Re: Conversion of strange characters
by Skeeve (Parson) on Feb 17, 2004 at 11:39 UTC
    We can't help if you don't tell us, what ABBE02`pr%90073462 should really look like.
      That's a really good point! Sorry.

      The line should look like

      ABBE02799799690073462

      the 7997996 is the packed decimal field that is converting to `pr%. Unfortunately, the packed fields vary on each line so I can't do a straight regex of those characters to the correctly numerical form.

      elbow
Re: Conversion of strange characters
by Abigail-II (Bishop) on Feb 17, 2004 at 11:46 UTC
    I have no idea what you'd call these characters (are they hex?)
    If you don't know whether those characters are hex (well, `pr% certainly isn't), how would you or we know how to translate them? For all we know, ABBE02`pr%90073462 is some encoding of The Atlanta Braves will win the Worldseries 2004.

    I've amended the mainframe job to stop it reoccurring!)
    The world is bigger than OSses run on x86 architectures!

    Abigail

      If you don't know whether those characters are hex (well, `pr% certainly isn't), how would you or we know how to translate them?

      sorry - I just thought someone with more experience might have more of an idea what these characters represent...and come across a similar problem with a text file that they'd managed to convert

      elbow
        Here are a couple of articles that might have some helpful information.
        from the pen
        perl discuss

        This mod tests for a valid packed decimal. Might have some clues for you as well.
        cpan mod

        More information would definately help. Is it IBM-370 style? COBOL? BCD?

        Or is it just the country code of Puerto rico...

        Kerry
        "Yet what are all such gaieties to me
        Whose thoughts are full of indices and surds?"
        quotes the Lama
        I have converted many files in the past. There are billions and billions of ways to encode files. If you don't know how it's encoded, you might want to try applying some decryption techniques, but with the handful of bytes you've given, that's a hopeless task.

        Abigail

Re: Conversion of strange characters
by tbone1 (Monsignor) on Feb 17, 2004 at 15:41 UTC

    I've done a few of these conversions, and they are unpretty, to say the least. Part of it is that, as others have noted, we don't know the encryption/decryption is. Also, you hint that there has been a partial decryption; that might make certain canned solutions invalid, since they might do something you don't wish to have done again.

    Also, I've seen that some mainframes don't always use the same values to mean the same characters. That is frightening. I ran across this when one of our customers was having trouble translating a file. Apparently, their computer was using "incomplete UTF-EBCDIC", which as you might guess wasn't precisely the same as the EBCDIC others were using.

    Just FYI.

    --
    tbone1, YAPS (Yet Another Perl Schlub)
    And remember, if he succeeds, so what.
    - Chick McGee

Re: Conversion of strange characters
by iburrell (Chaplain) on Feb 17, 2004 at 21:46 UTC
    I did a brief search for "packed decimal field". I found a message talking about the Convert::IBM390 module that handles translating between ASCII and EBCDIC and packed decimal fields.

    If the packed decimal fields in random places, you may have a hard time pulling them out. In your example, it is obvious that "`pr%` isn't like the rest. But I don't know how a parser would recognize the field. But there are no separators between fields.