The only way I can think of doing this *scalably* wrt memory seems pretty boring (i.e. you have to read in fixed sized blocks or something similar).

Could you just use sysopen/sysread to read blocks of data in and then remember the offsets of the 4-byte sequences? You could then go back and seek directly to the remembered offset and sysread the data in, repeating as needed.

You do have to worry about the case where the 4-bytes overlapped the end of a block. You could handle this by always reading ahead 1 block and then copying 3 bytes from the beginning of the next block to the end of the one you are working on before you check for the 4-byte sequence. (I've written too much C to be able to come up with anything more elegant.)

bluto