llil4vec start use OpenMP ( old ) ( next ) ( buf ) (2024-07) memory consumption 62.4 GB 33.0 GB 29.2 GB 26.8 GB use boost sort get properties 32.659s 12.993s 12.186s 7.182s sort properties 39.399s 33.421s 22.713s 20.518s vector reduce 75.644s 35.568s 33.045s 23.868s vector stable sort 31.133s 21.987s 19.845s 14.556s write stdout 3.206s 3.693s 1.423s 3.108s total time 182.042s 107.664s 89.214s 69.235s count lines 970195200 count unique 295755152 29612263 5038456270 real time 3m38.118s 1m50.092s 1m30.368s 1m09.791s user 70m13.776s 45m32.094s 47m28.779s 38m18.296s sys 2m29.584s 2m20.917s 1m10.917s 0m59.318s