thanos1983 has asked for the wisdom of the Perl Monks concerning the following question:
Hello Monks,
Recently I was introduced to diamond operator from I/O Operators, through the thread (How to amend character count.).
Well based on the answers I started wondering what is the best and fastest way to extract data from files with thousands of lines. So I was trying to Benchmark the two two fastest from my opinion ways (while() and grep) in iterating files.
I did not think it would be a problem to replicate the behavior but I end up with the script hanging and doing nothing. I was not able to understand why, until I created separate sub routines.
When each sub routine is executed individually and the script exits everything works fine, but when the subroutines are executed sequentially the script stays hanged.
I can understand that is based on the @ARG but I do not know how to tell to the script that that there is no more files to process by using explicitly the diamond operator <> and not individually the @ARG
Sample of code that can replicate my problem:
#!/usr/bin/perl use strict; use warnings; use Benchmark::Forking qw( timethese cmpthese ); # UnixOS # use Benchmark qw(:all) ; # WindowsOS sub test_grep { my @matched = grep { $_ =~ /AGGT/ } <>; if (@matched) { for(@matched) { chomp $_; print "Grep: " . length($_) . "\n";} } close ARGV if eof; } sub test_while { my $count; while (<>) { if ($_ =~ /^>hsa/){ chomp (my $line = <>); $count .= length($line); $count .= " "; } } continue { close ARGV if eof; # reset $. each file } print "While: " . $count . "\n"; } # test_grep(); # Works fine separate but not combined test_while(); # Works fine separate but not combined __DATA__ $ perl test.pl test.txt Grep: 107 Grep: 251 ^C $ perl test.pl test.txt While: 107 251 ^C __END__ my $results = timethese( 2, { 'Grep' => sub { }, 'While' => sub { }, }, 'none' ); cmpthese( $results );
Sample of test.txt file with data for replication purposes:
>hsa_circ_0000001|chr1:1080738-1080845-|None|None ATGGGGTTGGGTCAGCCGTGCGGTCAGGTCAGGTCGGCCATGAGGTCAGGTGGGGTCGGCCATGAAGGTG +GTGGGGGTCATGAGGTCACAAGGGGGTCGGCCATGTG >hsa_circ_0000002|chr1:1158623-1159348-|NM_016176|SDF4 GGTGGATGTGAACACTGACCGGAAGATCAGTGCCAAGGAGATGCAGCGCTGGATCATGGAGAAGACGGCC +GAGCACTTCCAGGAGGCCATGGAGGAGAGCAAGACACACTTCCGCGCCGTGGACCCTGACGGGGACGGT +CACGTGTCTTGGGACGAGTATAAGGTGAAGTTTTTGGCGAGTAAAGGCCATAGCGAGAAGGAGGTTGCC +GACGCCATCAGGCTCAACGAGGAACTCAAAGTGGATGAGGAAA
Does any one know how to approach this minor problem?
Update: With the help of tobyink the following code demonstrates the solution to my problem but also that while is faster than grep on my loop because of the for loop that I am using as extra. Thanks everyone again for their time and effort.
#!/usr/bin/perl use strict; use warnings; use Benchmark::Forking qw( timethese cmpthese ); # UnixOS # use Benchmark qw(:all) ; # WindowsOS my @preserved = @ARGV; sub test_grep { @ARGV = @preserved; # restore original @ARGV my @matched = grep { $_ =~ /AGGT/ } <>; my $count; if (@matched) { for(@matched) { chomp $_; $count .= length($_) . " ";} } close ARGV if eof; } sub test_while { @ARGV = @preserved; # restore original @ARGV my $count; while (<>) { if ($_ =~ /AGGT/){ chomp; $count .= length($_) . " "; } } continue { close ARGV if eof; # reset $. each file } } my $results = timethese(10000000, { Grep => \&test_grep, While => \&test_while }, 'none'); cmpthese( $results ); __DATA__ perl test.pl test.txt Rate Grep While Grep 144175/s -- -21% While 182882/s 27% --
Thanks in advance for your time and effort.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re: Benchmark diamond operator
by tobyink (Canon) on May 09, 2017 at 14:46 UTC | |
by thanos1983 (Parson) on May 09, 2017 at 14:55 UTC | |
by tobyink (Canon) on May 09, 2017 at 18:55 UTC | |
by thanos1983 (Parson) on May 10, 2017 at 08:50 UTC | |
by Discipulus (Canon) on May 09, 2017 at 20:07 UTC | |
by thanos1983 (Parson) on May 10, 2017 at 08:58 UTC | |
|
Re: Benchmark diamond operator
by davido (Cardinal) on May 09, 2017 at 14:47 UTC | |
by thanos1983 (Parson) on May 09, 2017 at 14:59 UTC | |
by davido (Cardinal) on May 10, 2017 at 15:56 UTC | |
|
Re: Benchmark diamond operator
by 1nickt (Canon) on May 09, 2017 at 14:28 UTC | |
by thanos1983 (Parson) on May 09, 2017 at 14:47 UTC | |
|
Re: Benchmark diamond operator
by AnomalousMonk (Archbishop) on May 09, 2017 at 14:28 UTC | |
by thanos1983 (Parson) on May 09, 2017 at 15:13 UTC |