in reply to how to identify a fixed width file

The reason why XML, CSV, and other similar file formats were created was to address the inherent problems with fixed with formats. THe first formats were fixed width because they are very simple to work with. In essence, they are the serialization of an array of structs in C. So, marshalling one of those in C is really simple. Finding a given record when you know its index (10th, 1024th, etc) is very simple. Overwriting a given record is very simple. It's the ultimate in RAM-backed-to-disk. The only problem is that you have to know the mapping. If you don't know what a fixed-width format means, you're out of luck.

And, furthermore, many fixed-width files have a header and, possibly, a footer. DBM::Deep's file format is a record-based format with a two headers (first is fixed, second is variable). Good luck detecting that it's a DBM::Deep file without recognizing the first four bytes.

Frankly, I'd do the following:

  1. Is it XML, CSV, HTML, etc?
  2. Is it a fixed-width format I recognize (PNG, JPG, DOC, XLS, etc)?
  3. Punt.
Which, essentially, is what the file utility does.
My criteria for good software:
  1. Does it work?
  2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Replies are listed 'Best First'.
Re^2: how to identify a fixed width file
by ftumsh (Scribe) on May 14, 2008 at 14:17 UTC 
    1) I do recognise XML, CSV etc already, the problem is with fixed width.

2) The formats mentioned are not text files so are of no relevance.

3) I'd rather not punt if possible, tho it seems I may have to.

My code atm uses File::MMagic to get the mime type. If it's a text file I then work out what sort of text file it is ie

1) XML - uses mmagic and XML::LibXML
2) SAIFFE - regex
3) EDIFACT - regex
4) Tradacoms - regex
5) CSV - Text::CSV_XS
6) Fixed width - foobar
[reply]

In Section Seekers of Perl Wisdom