in reply to Re: Netflix (or on handling large amounts of data efficiently in perl)
in thread Netflix (or on handling large amounts of data efficiently in perl)

It really depends what processing needs to be done on the data. You are trading space for speed. Here if you need to get all the data for a movie, you will need to go through all of the users. So before choosing a format to store the data, it might be useful to know what you want to do with it first.

  • Comment on Re^2: Netflix (or on handling large amounts of data efficiently in perl)

Replies are listed 'Best First'.
Re^3: Netflix (or on handling large amounts of data efficiently in perl)
by matrixmadhan (Beadle) on Dec 25, 2008 at 05:05 UTC
    I perfectly agree that space is being traded for speed. But as per the OP it seems that the storage is much more important than the retrieval, so I think my approach might prove well to be a fit.

    As an improved version of storing them as a map, an auto-generator can be applied where based on the index retrieval even contiguous storage can be used without even having to create lookup maps and retrieving from them.
[reply]

In Section Seekers of Perl Wisdom