in reply to Re: Re: Re: Re: opening a file in a subroutine
in thread opening a file in a subroutine

*blushes*

The first and the second times are to be expected to be different. The third should be the same as the second. The reason for the difference is that on the first time all it has to do is allocate the space for the array and the strings. The second and third trys have to deallocate first and then suffer the same overhead.

However, I believe you have stumbled on a bug. When I change the context to scalar the times are predicatable. For instance your code on an 8.5MB file it takes 13 seconds for the first try and more minutes than I am willing to wait on the next two. When i replace it with a slurp and a plit it becomes 2 seconds and 5 seconds. When i replace it with a while (<$fh>) it goes to 4 and 10. Heres the code:

use strict; use warnings; $|++; my @lines; for my $try (1..3) { print "Try $try\n"; my $start=time; @lines=(); # 6 million ways to die, choose one: # my_slurp(\@lines); # my_while(\@lines); # my_file(\@lines); print "Took ".(time-$start)." seconds.\n"; } sub my_file { my ($line) = @_; open my $fh, "D:\\Perl\\DevLib\\Scrabble\\dict.txt"; @{$line}= <$fh>; close($fh); } sub my_slurp { my ($line) = @_; local $/; open my $fh, "D:\\Perl\\DevLib\\Scrabble\\dict.txt"; @{$line}=split /\n/,<$fh>; close($fh); } sub my_while { my ($line) = @_; open my $fh, "D:\\Perl\\DevLib\\Scrabble\\dict.txt"; push @$line,$_ while <$fh>; close($fh); } __END__ my_file: Try 1 Took 13 seconds. Try 2 ^C (I killed it after 5 minutes.) my_while: Try 1 Took 4 seconds. Try 2 Took 10 seconds. Try 3 Took 9 seconds. my_slurp: Try 1 Took 2 seconds. Try 2 Took 6 seconds. Try 3 Took 5 seconds.
Hopefully someone with 5.8 or bleadperl can check if this is a resolved bug, if not it definately should be reported.

Sorry I didnt try your code properly on my first reply. :-)

Dont forget the resolution on the timings is at least +-1 second.

Also, have you considered that maybe there is a better strategy than reading the whole file in three times? Do you have to read it all in at once? Why cant you just copy the data once its read? This doesnt resolve the bug, but it sounds like design improvements are not out of order.

--- demerphq
my friends call me, usually because I'm late....

Replies are listed 'Best First'.
Re: Re: Re: Re: Re: Re: opening a file in a subroutine
by abhishes (Friar) on Feb 09, 2003 at 17:11 UTC
    WOOOOOOWWWW !!!!!

    the call local $/; did the magic!!! as soon as I put that into my function my code is flying now.

    Thank you so much for the slurp function .... Now its really really fast!!!

    BTW, what did it do? I don't understand the meaining of local $/?

    regards,
    Abhishek.
      Er, hold on there... Dont be too hasty.

      $/ is the end of record seperator. It defaults to "\n".

      What my code, which is in essence

      my @list=split /\n/,do {local $/; <$fh>};
      does is to read the whole file into a buffer then split it up by newlines and then return them, which should be more or less the same as
      my @list=<$fh>;
      (at least under default conditions) but obviously isn't on AS 5.6.

      I hope you didnt do it naively as otherwise youll only have one entry in the array, which will contain the whole file.

      --- demerphq
      my friends call me, usually because I'm late....

        Thanks again,
        I modified my code to
        local $/; open my $fh, "xml.log"; my $line = <$fh>; close ($fh);

        Now I have the whole file in $line and I extract all the data which I want from regular expressions. My code is working as expected and is very very fast.

        regards,
        Abhishek.
Re ^5: opening a file in a subroutine
by BazB (Priest) on Feb 09, 2003 at 17:13 UTC

    OK, I tested this with v5.8.0 built for i386-linux-thread-multi.

    Test data was a 32M, 2.5 million line file.
    Machine is a 1740Mhz Althon, 512Mb real memory, 768Mb swap.

    Results:

    my_slurp: Try 1 Took 11 seconds. Try 2 Took 16 seconds. Try 3 Took 15 seconds. my_while: Try 1 Took 4 seconds. Try 2 Took 5 seconds. Try 3 Took 4 seconds. my_file: Try 1 Took 33 seconds. Try 2 Took 19 seconds. Try 3 Took 14 seconds.

    If the information in this post is inaccurate, or just plain wrong, don't just downvote - please post explaining what's wrong.
    That way everyone learns.