Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: Thoughts on designing a file format.

by gargle (Chaplain)
on Sep 12, 2005 at 19:41 UTC ( #491349=note: print w/replies, xml ) Need Help??


in reply to Thoughts on designing a file format.

Hi,

A lovely node! My years as a cobol programmer can come in handy for treating batch programs :) Just a few remarks:

header
put the name of the program creating the file, a date and a time of creation and a date and time of modification. Also put the filename in the header! The point is that the header identifies the file. So you need info about which program created/modified and when. You can even put extra info about the programs that are expected to modify next or treated the file before the current program had a go at it!
trailer
put the total number of records (header and trailer inclusive). Put a total (or a md5 total) for the most import fields in your data. Include the number of data records. If you decide to keep creation data/time in the header put your date and time of modification here.
data
Just as you use a header and trailer for the complete file use seperate headers and trailers for blocks of data in your file. Your data records should be identifiable by 0 for a header, 1 for real data and 9 for a trailer of a data block.

You'll end up with:

H20050912211900,MYPROGRAMNAME,FILENAME,NEXT_PROGRAM,PREVIOUS_PROGRAM< 0ACCOUNT001DATABLOCKHEADER< 1MY_FIRST_TRANSACTION OF 500 EUR< 1MY_SECOND_TRANSACTION OF 100O EUR< 9,NUMBER_OF_TRANSACTIONS,TOTAL_AMOUNT< (2 transactions, 1500 eur total +) 0ACCOUNT002DATABLOCKHEADER< 9,0_TRANSACTIONS,0 AMOUNT< (0 transactions, 0 eur total) 0ACCOUNT003DATABLOCKHEADER< 1MY_FIRST_TRANSACTION OF 25 EUR< 2MY_FIRST_TRANSACTION OF 25 EUR< 3MY_FIRST_TRANSACTION OF 25 EUR< 9,TOTAL_NUMBER_OF_RECORDS,NUMBER_OF_TRANSACTIONS,TOTAL_AMOUNT< (3 tran +saction, 75 eur total) T,TOTAL_ACCOUNTS,TOTAL_TRANSACTIONS,TOTAL_AMOUNT_ALL_TRANSACTIONS< (13 + records,3 accounts (data blocks), 5 transactions, 1575 eur total)

Some comments:

name program creating the file
This allows you to check if the next program treating the file is the correct one. You can controle the sequence of treatment by isuing a die if the correct sequence is out of order. To make this even better put a second field to name the program that will process the file next.
trailer info
Reading the complete file and adding subtotals allows you to check the trailer for modifications in the file.
data block trailer
Reading the datablock and adding subtotals allows you to check the data block for integrity

Of course, all other notes of the OP count as well (however, thinking in cobol makes me prefer fixed record lengths)

More info: Jackson

--
if ( 1 ) { $postman->ring() for (1..2); }

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://491349]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (8)
As of 2023-02-06 11:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I prefer not to run the latest version of Perl because:







    Results (34 votes). Check out past polls.

    Notices?