Space filling means that there is no portion of the map which does not have a block associated with it (no gaps). In Geo/ShapeFile, it says it only loads data as it needs it, so it's possible that only some fraction of the 6GB need be loaded for your problem - might be worth running a test script to check.

Knowing how G-men think, the QuadTree approach Joost suggests is definitely worth trying - IIRC you essentially build a tree of hierarchical rectangles. You can then use this minimal information (which should certainly be less than 6GB) to compare instead of reloading point data in your loop.