in reply to Re: Re: Re: Re: reading columns from a flat file
in thread reading columns from a flat file

There could be 10,000 ids each having up to 200,000 elements

Egads! Is there a possibility of just writing a script that takes these unwieldy inverted flat-files extracts the data in correct order once, and inserts it into a proper database for greater scalability? DBI with DBD::SQLite might be one (of several) good choices for the database stuff. Do the conversion once (or whenever you get a new inverted file), and then just be done with it. ...and enjoy the flexibility and scalability of a database.


Dave

  • Comment on Re: Re: Re: Re: Re: reading columns from a flat file