It is possible to do what you want with Perl, but it will not be easy (believe me I had to do it myself and it took me a lot of time and sweat and cursing and ...), so perhaps one should consider a different approach.

I read "I have a flat file exported from a database" and I wonder whether you can directly access this database.

As you know Perl has the wonderful DBI /DBD family of modules which allow you to work with most types of databases (including MS Access).

So if you need to transfer data from one database to another, DBI/DBD is the obvious way to go: write some SQL to select the data you need, execute this SQL, save the results in an array or hash-variable, then insert the data in the other database.

If you cannot speak to the original database, perhaps you can change the export format. A real "CSV" format seems not possible (due to the repeating fields at the end), but an 'XML' format is ideal. XML is very well suited for data-interchange.

One last comment: Access is perhaps not the best choice when you have a lot of data to transfer. After a few thousand records the performance really goes down (if not worse and it crashes on you). Before that happens you should investigate into using a real database server (MySQL, Postgres, to name but a few).

CountZero

"If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

In reply to Re: Extracting data from a flat file and inserting it into a databas by CountZero
in thread Extracting data from a flat file and inserting it into a databas by scribefaith

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.