IMO, the best way to avoid the problem is to not allow it to be a problem in the first place.

You can do this by generating two files, one where all of the rows are guaranteed to unique, and then everything else would be treated as an exception.

Your system will need to intelligently create your bulk insert files to provide the appropriate indices (if it's not able to do that inherently), and to detect for unique key collisions.

I wouldn't think that time would be any consideration for just 200k records.

For the exceptions file, you would then have your preprocessor generate the appropriate SQL script to facilitate the updates.


Your wish is my commandline.

In reply to Re: DBI Question: bulk insertion with Unique-key constraint by thezip
in thread DBI Question: bulk insertion with Unique-key constraint by lihao

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.