Something I have found useful for flat files, not hierarchical files, is to have the names of the columns as the first line.

This helps avoid hardcoding the column names in your parsing/construction code - I'd expect the names will embedded in the code that utilises the constructed structure, after all, its hard to escape the need to type "$row->{address}" when you need to access the address field (or $row->address() if you've built objects to give yourself one degreee of separation)

With the columns names as the first row, you are insulated from the introduction of new columns until you really need them. For example, say you have columns 'name','street','town','country'. You have code to parse this line and create the appropriate accessors, and more code that reads a line and, for each line, returns an object that has these accessor methods. You then have code that uses these objects to build address labels or populate a database. Then one day the client who suplies these files adds a new field, say zip code, and it is in the data between the town and country columns. NONE of your code has to change until your ready to use the zip code field. Your objects have an extra accessor, which is automatically created by the code that parses the column-description line, but nothing else is affected.

It could be you want to add the zip code to the address labels, but you dont want to change the DB schema to capture zip codes. You change the label generation code, and leave the DB insert code alone. So your DB inserts continue to work, and your address labels now have zip codes.

All because you were able to have the client give you one extra line in the supplied flat files.

...reality must take precedence over public relations, for nature cannot be fooled. - R P Feynmann


In reply to Re: Thoughts on designing a file format. by leriksen
in thread Thoughts on designing a file format. by demerphq

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.