I appreciate what both of you have said in your posts back to me.
The first post about variable name length, I tend to keep variable names in a short hand style instead of spelling them out (just one of my quirks). The strict and the warn are in the code and I have done alot of clean-up with these 2 turned on of course, but that does not eliminate the problem that I have run into. I am on a Windows XP machine, lets not hear the grief over that. I found that with the eof from the first loop the file that I am parsing has some violations in it of its own accord that really sets the parser off into la-la land. Getting the record count fixes most of the problems but I still have a few things to fix in the data record. The checking of the values is in a later part of the routine that goes against the ARCGIS shape files, and boy is that fun as again the data is not as well adjusted as it should be.

Long story short I am a Software Tester and I have been given the dubious task of going through Company data and suggesting where to fix it. So far I have fixed 10 of 50 data streams in under 4 months saving the Company 6 million a month in lost revenue, but who cares back to task.

The data files are not really that big, about a meg for each file and the number of files varies between 8 and 300. Slow for me tends to be breaking one file in over a minute, I should be able to break a file every few seconds and have an error file gen'ed in about a minute for all of the files that are in the dataset.

All in all I again appreciate what both of you have said and I will continue to work over the code base. From what has been posted so far I may have more of an issue with the data being presented in the files than with the poor code I have written to parse said files.



Thanks, Mike

In reply to Re^2: File Parsing by Anonymous Monk
in thread File Parsing by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.