At my job, I have to write parsers for data that comes from online information repositories. Each repository provides pretty much the same information, but each has its own quirky format. Until recently, I was writing parsers based on the domain name of the repository. The problem here is that sometimes a repository will start using a new format. Once this is detected (and it might take a while for us to realize that) I have to examine this new format and write a new parser.

Then, this morning, I noticed two repositories in a row in my queue that had the same IP address. D'oh! It turns out I've written a lot of duplicate parsers. Things that appeared different were not distinct, but only variable (e.g., an address that requires two lines instead of one). These differences between repositories made me write seperate parsers, but this IP realization has shed light on the issue.

As of tomorrow (when I should have the redundant parsers done away with and the core parsers centralized) there will be far less code to maintain, and it will be easier to deal with a change of formatting. See, if repositories A, B, and C are all the same IP address, then if that IP address's format changes, I only need to update its format. If just A changes its IP address, I can take care of A's format without worrying about B's and C's. Even better, if these repositories new IP address happens to already be covered, I don't need to anything.

Laziness, Impatience, Hubris. Amen.


Jeff japhy Pinyan, P.L., P.M., P.O.D, X.S.: Perl, regex, and perl hacker
How can we ever be the sold short or the cheated, we who for every service have long ago been overpaid? ~~ Meister Eckhart

Replies are listed 'Best First'.
Re: Perl Virtue: Laziness
by artist (Parson) on Jul 11, 2005 at 14:24 UTC
    Proper insight in your data can help to code less and get more done. Unfortunately I don't see much literature about it for programmers, and many times it come down to Aha! moments individually.

    --Artist
Re: Perl Virtue: Laziness
by tbone1 (Monsignor) on Jul 19, 2005 at 12:11 UTC
    I think this might be why I consider those of us with degrees in the physical/mathematical sciences to be good programmers. Admittedly, I'm biased; my degree is in astrophysics. However, I've worked with people who studied physics, chemistry, astronomy, geology, etc, and we had to spend a good bit of our undergraduate and graduate days analyzing big masses of data, trying to find out why, say, these four stars have more ultraviolet emission lines than expected, or whatever. That makes excellent training for programming against data that comes from a variety of sources, or when specs aren't 100% complete, or having to do a little quick improvisational programming, and so on.

    Just my $.02.

    --
    tbone1, YAPS (Yet Another Perl Schlub)
    And remember, if he succeeds, so what.
    - Chick McGee