swngnmonk,
I have to vehemently disagree with you. The one person who suggested utils was only trying to offload the sorting and left the rest of the "actual file-processing" to Perl. graff realized one limitation - imbedded newlines. I went on to list others.

Use sed to clean up any potential issues
What are you talking about?
CSV is like HTML - it should be parsed not have a single regex applied to it. While it is possible to build a sed parsing script - it is a nightmare to maintain and does not come close to having the logic functionality of Perl.

Use sort to re-order the file based on the unique-key column
How exactly do you propose to do that? CSV is not fixed width and with imbedded quote characters being escaped it doesn't sound too easy. While I think awk might be up to the challenge, it seems that each set of CSV files would require a unique solution.

diff the newly-generated file against the last one we processed
I guess you didn't read the requirements section of my post. Being able to have the user specify columns to be ignored in the comparison, being able to select which columns are case sensitive, being able to get which individual fields are different and not just the entire record.

Finally, I indicated that this had to be "user friendly" and never mentioned speed as a bonus let alone a requirement. I am quite familiar with *nix utilities and don't hesitate to use them when they are the "right tool" for the job. They can't make my "life MUCH easier here" if they fail to meet the requirements.

L~R

In reply to Re^2: CSV Diff Utility by Limbic~Region
in thread CSV Diff Utility by Limbic~Region

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.