Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello.
I want to create a txt file that is 1 bytes or several MB.
The script is doing what I want to but its slow.

How can make it faster?.

CODE:
### Give the number in bytes that should be created:
print ("Number in bytes:\n");

### Read in the choice.
$choice = <STDIN>;
chomp ($choice);

### Divide the choice.
$divide = $choice / $choice;

@x = $divide..$choice;

foreach $x (@x)
{
open(LOG, ">> file.txt");
print LOG "x";
close LOG;
}

[download]
//Anders Andersson

Edited by BazB added code tags

Replies are listed 'Best First'.
Re: Slow file creator
by exussum0 (Vicar) on Feb 07, 2004 at 22:50 UTC
    Lets see where to start. 1. first you are creating a "big ass array". you throw away th edata. No reason to do that. 2. You are opening/closing the file over and over again. You can write more than 8 bits to a file at once. Why not do something simple like... 
    open( ... )
for( 1 .. $choice )
{
  print LOG "x";
}
close( ... )

    [download]
    3rdly, why divide choice by itself to get 1? What if the user types in 0? you dont' exit gracefully, but kill your own program.

    Play that funky music white boy..
[reply]
[d/l]
      sporty++, you've nailed it on the head! That's the way I would have done it too. To answer the anony-monk's question though, it is the opening and closing of the file that is taking so long.

      Alter your script following sporty's code snippet and your script should complete in less than a second for even a few megabytes!

      - - Arden.

[reply]
Re: Slow file creator
by tachyon (Chancellor) on Feb 07, 2004 at 23:52 UTC

    You do realise that $divide will always be 1 don't you as anything/anything is always going to be 1..... If you want a short fast solution just do: 

    perl -e 'print "x" x 1000' > outfile.txt

    [download]

    Substitute the 1000 with whatever number of bytes you want.

    cheers

    tachyon

[reply]
[d/l]
Re: Slow file creator
by guha (Priest) on Feb 07, 2004 at 23:45 UTC

    It's not clear if you don't care about the file's content as long it has the right size, if so the code below will be quite quick.

    my $req_len = shift || 100;

open FH, "+>", 'test.txt';
seek FH, $req_len - 1,0;
print FH 'X';
close FH;

    [download]

    or simpler ? 

    my $req_len = shift || 100;
truncate 'test.txt', $req_len;

    [download]
    HTH
[reply]
[d/l]
[select]
Re: Slow file creator
by Berik (Sexton) on Feb 07, 2004 at 23:16 UTC
    You could create the file content in advance,
    and save (possible) system calls.

    my $fill_char = " ";
my $string = $fill_char x $nr_of_bytes;

open FW, ... ;
print FW $string;
close FW;

    [download]
    With this approach you do only one print,
    open and close systemcall. As systemcalls can
    take up much time, this would be lot faster.
    You also give the perl compiler more chance of
    optimizing by not forcing it to use a loop.

    Also, if you are on a machine with a gigabyte
    of ram, who cares about your script taking one
    MB for it's job. If you're planning to use more
    than 50 MB or so.. You should check out the tools
    that have been build for this kind of job. See the
    'dd' manpage if you are on a *NIX box.
[reply]
[d/l]
      The printing is buffered anyhow, so it's not like a loop would kill perormance.

      Play that funky music white boy..
[reply]
        But still doing a print call gives overhead,
        because the print buffer will be flushed much
        more often.
        use Benchmark::Timer;

my $t = Benchmark::Timer->new();
my $size = 1000000;

$t->start( 'string' );
open FW, "> temp.out";
print FW " " x $size;
close FW;
$t->stop( 'string' );

$t->start( 'loop' );
open FW, "> temp.out";
for ( 0..$size ) { print FW " " }
close FW;
$t->stop( 'loop' );

$t->report;
__END__
Reports:
1 trial of string (17.594ms total)
1 trial of loop (733.321ms total)

        [download]
        This clearly shows the loop decreases performance.

        Update: I figured that perl could create the string
        at compile time, so I put the size in a variable. But
        this yields thesame results. Perl still optimizes
        the string version in this exapmle. But that was
        also the point I made in the first reply.
[reply]
[d/l]
Re: Slow file creator
by flyingmoose (Priest) on Feb 07, 2004 at 23:50 UTC

    Lots of fine comments already, but... 

    ### Read in the choice.
$choice = <STDIN>;
chomp ($choice);

### Divide the choice.
$divide = $choice / $choice;

    [download]

    Just to add...."$divide=$choice/$choice" is always going to be either 1 or "0/0". You don't want to compute 0/0, especially!

[reply]
[d/l]

Back to Seekers of Perl Wisdom