Re: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!

Your screenshot contains lines mentioning a path that ends in \cgi\forum\test2.txt. Are you trying to write to write a forum script that stores posts in a flat textfile? That would be a bad idea, and no amount of microoptimising Perl’s I/O can make that fast.

Makeshifts last the longest.

Comment on Re: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!

Replies are listed 'Best First'.
Re^2: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO! by NeilF (Sexton) on Jan 01, 2006 at 19:48 UTC
Aristotle, yes it is for a forum... I wrote it 4-5 years ago and its been chugging along very nicely for all those years. It's not heavily used, ie: typically only 0-12 people using it at any one time... However, the io processing has been noticed by my ISP hence my attempts to reduce it. I've made a number of mods already that have generally improved/reduced io processes, but one of the biggest hogs is when postting a message. It seems at the least with writing:- a) Just by adding "binmode" prior to the write that will more than half the io. And this can be applied to all such writes to improve across the board! b) By consolidating an array into a variable that will further reduce the io. However, I don't like the idea of having a loop which goes through X thousand records, and having the data held twice (once in the array and once in a variable) at the same time! So my two questions are currently:- a) Is there a simple means of improving the reading, as has seemingly been found for the write. Reading in 4K chunks, blah! b) Is there a better means of printing the array as a consolidated variable, so the data is not held twice in memory etc.	[reply]
Re^3: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO! by merlyn (Sage) on Jan 01, 2006 at 22:02 UTC
However, the io processing has been noticed by my ISP hence my attempts to reduce it. If your ISP is poking into it that much, it's either time to renegotiate with your ISP to make that a permitted activity, or get any one of the four million ISPs that won't care. Most ISPs charge for network bandwidth and disk storage, and would never care how many times you call read() or write(). -- Randal L. Schwartz, Perl hacker Be sure to read my standard disclaimer if this is a reply.	[reply]
Re^4: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO! by NeilF (Sexton) on Jan 01, 2006 at 23:01 UTC
They have a prefered limit of 20,000,000 IO processes a day...	[reply]