$ perl -lwe '$/ = \128; ++$hist[tr/\0//c] while <>; print "$_: $hist[$ +_]" for 0..$#hist;' lookzpM.byte-* 0: 12003438 1: 148200 2: 147874 ... 42: 330481 43: 33028
This means you have plenty of room to pack the lines as: {N} ({i}{x})* {pad}.
Compute q8 = (i ^ m7) & 127; q9 = (x ^ HASH_LEN);
Average 15.984 elements per line. Lots of looping and branches can be avoided this way.
Update. One more thing occurs to me. Indeed, those files compress rather well: lzma packs them 1:2500!
In reply to Re^2: The 10**21 Problem (Part 2)
by oiskuu
in thread The 10**21 Problem (Part 2)
by eyepopslikeamosquito
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |