in reply to Is 100% CPU utilisation during a procees is aproblem?

The biggest file I had handy is 1GB, and it took 15 seconds to process using wc -l big.dat and uses ~5% cpu.

Based on that, your 17GB should take ~4 1/2 mins and the small one ~3/4 min, and the cpu usage of 5%-10% is typical for an IO bound program.

My guess is that you're using more memory than your machine has available and have therefore moved into thrashing.

Whether this is a problem depends upon your urgency and expectations.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

In Section Seekers of Perl Wisdom