Yoy want two opposites:
Speed and
Integrity checks. I don't think they can easily be combined. How about the following:
- Bulk load the data with indices disabled (alter table t disable keys).
- Then enable the keys without enforcing uniqueness.
- Create queries to identify your duplicates, and deal with them (i.e. delete).
- Then enable the keys enforcing uniqueness
I agree with
perrin, use the database for what it's worth.
--
No matter how great and destructive your problems may seem now, remember, you've probably only seen the tip of them. [1]