Re^10: search a large text file

Thanks, made it work with some small modifications! However the file still goes out of memory? which is strange! although I read the file line by line I get out of memory message using 10 gb of ram!

Comment on Re^10: search a large text file

Replies are listed 'Best First'.
Re^11: search a large text file by jethro (Monsignor) on Feb 10, 2011 at 15:37 UTC
Possibly this is because of too small pointers in the file (strangely the default is to limit the database size to 4GB). Read about that in section "LARGEFILE SUPPORT" of the DBM::Deep documentation. You also need a 64bit operating system to circumvent that limit If you don't have a 64bit system or still get out-of-memory errors it might be better to use a "real" database engine, like DBD::SQLite (which has the advantage that you don't have to install a separate database engine), mysql or postgreSQL (see erix answer).	[reply]

Replies are listed 'Best First'.

Re^11: search a large text file
by jethro (Monsignor) on Feb 10, 2011 at 15:37 UTC

Possibly this is because of too small pointers in the file (strangely the default is to limit the database size to 4GB). Read about that in section "LARGEFILE SUPPORT" of the DBM::Deep documentation. You also need a 64bit operating system to circumvent that limit

If you don't have a 64bit system or still get out-of-memory errors it might be better to use a "real" database engine, like DBD::SQLite (which has the advantage that you don't have to install a separate database engine), mysql or postgreSQL (see erix answer).

[reply]