http://qs1969.pair.com?node_id=11140380


in reply to Re^2: [OT] Re: Get unique fields from file
in thread Get unique fields from file

Well I guess there is also DBD::CSV but once I got onto SQLite, I haven't looked back. SQLite is the most used DB in the world (geez, it is on every cell phone and in almost all browsers...), well debugged and very efficient for what it does.

Let's not quibble too much about the details.

The main point is that using SQL especially in conjunction with a real DB is definitely "on the table" as a reasonable solution for the OP's problem. So I don't think that is OT at all. This approach will scale well into much, much larger files. Functionality like "build histogram" and "print only the first time" are heavily optimized and a real DB is pretty smart about how it uses the memory available to it.

I don't think we have heard back from the OP about requirement questions, so this is really just an exercise for our own edification and enjoyment.