in reply to selecting columns from a tab-separated-values file

You might be better off tolerating the expense of loading your data into a database of some sort once, perhaps SQLite, and thereafter take advantage of the speed of the database which is optimised for data look-ups, using SQL queries to access your columns. DBI and DBD::SQLite would facilitate this.

I hope this is helpful.

Cheers,

JohnGG

  • Comment on Re: selecting columns from a tab-separated-values file

Replies are listed 'Best First'.
Re^2: selecting columns from a tab-separated-values file
by ibm1620 (Hermit) on Jan 22, 2013 at 04:15 UTC
    Interesting idea. Can SQLite handle that capacity? 10^9 rows and about 80GB of data total?
[reply]
      give a go and you'll find out ;) or you can look here: http://sqlite.org/limits.html
[reply]
[reply]

In Section Seekers of Perl Wisdom