MIME::QuotedPrint poor performance

RenalPete has asked for the wisdom of the Perl Monks concerning the following question:

Hello, monks.

I have a strange performance problem using MIME::QuotedPrint.

I have uploaded a file called bad.txt - it is 3054 lines and 116k and is full of 'X's and whitespace.

If I generate a sample twice the size by perl -e 'print ( (("x"x100)."\n") x 3000)' > /tmp/good.txt

then run time perl -MMIME::QuotedPrint -e 'print encode_qp(`cat /tmp/good.txt`)' it runs in 0.7 seconds on my machine.

If I do the same for bad.txt, it takes 14 minutes.

Debugging via printf leads me to a regex in QuotedPrint.pm with a comment This makes things complicated and a regex which is too much for me! Can anyone see what's going on? Many thanks....

Comment on MIME::QuotedPrint poor performance Select or Download Code

Replies are listed 'Best First'.
Re: MIME::QuotedPrint poor performance by mr_mischief (Monsignor) on Apr 04, 2008 at 15:48 UTC
Update:Looks like the problem was solved while I was researching it and typing up my reply. The general idea of the first paragraph still stands for any benchmarks involving a filesystem, and I'll put the rest in a readmore in case anyone's interested. Well, for one thing, make sure you're writing the smaller file out immediately before reading it back in if you want to make sure your testing is accurate. There's a good chance your OS is buffering the write and reading the file back in from the buffer in the case of good.txt so make sure it does the same for bad.txt too. Read more... (766 Bytes)	[reply]
Re: MIME::QuotedPrint poor performance by Anonymous Monk on Apr 04, 2008 at 15:13 UTC
If i use $_ = join '', encode_qp(`cat good.txt` ); [download] I get `0 wallclock secs ( 0.05 usr + 0.00 sys = 0.05 CPU) 0 wallclock secs ( 0.01 usr + 0.03 sys = 0.05 CPU)` [download] 3 seconds if I print to my shell.	[reply] [d/l] [select]
Re^2: MIME::QuotedPrint poor performance by Anonymous Monk on Apr 04, 2008 at 15:14 UTC
This with MIME::QuotedPrint 3.03, perl v5.8.7 built for MSWin32-x86-multi-thread	[reply]
Re^2: MIME::QuotedPrint poor performance by RenalPete (Acolyte) on Apr 04, 2008 at 15:15 UTC
How long does it take with bad.txt? For me it is a looong time...	[reply]
Re^3: MIME::QuotedPrint poor performance by Anonymous Monk on Apr 04, 2008 at 15:18 UTC
one time is from bad, one from good, makes very very little different	[reply]
Re^4: MIME::QuotedPrint poor performance by RenalPete (Acolyte) on Apr 04, 2008 at 15:21 UTC