in reply to Reading 39GB Flat File?
Your database (which one you don't mention) probably has a way to import bulk data. For postgres, there is the COPY command which will read from STDIN (or directly from file):
cat file.txt | psql -d dbname -c "copy mytable from stdin csv"
'mytable' must have the appropriate columns, of course.
( A site-search on 'Bulk import' will probably yield bulk import methods for each RDBMS: bcp for sybase/mssql, imp for oracle, etc. )
update:
ok, ok... To appease the venerable and dodge the infamous award, this should be written as:
< file.txt psql -d dbname -c "copy mytable from stdin csv"
Although a process isn't as expensive as it used to be :)
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: Reading 39GB Flat File?
by merlyn (Sage) on Nov 12, 2009 at 17:13 UTC |