OK, here is a test. I started with a word.txt file having 113809 unique entries: 
$ wc words.txt
 113809  113809 1016714 words.txt

[download]
From there, I created a test_file.txt file, in which one 1200-character string was added to each entry of the word.txt file, and then copied 26 times, giving me a 3-million-line and 3.5 GB test_file.txt test file: 
$ perl -ne 'chomp; for my $let (A..Z) { print  "$_ ". "foobar" x 200, 
+" $let\n" }' words.txt > test_file.txt
$ wc test_file.txt
   2959034    8877102 3586152466 test_file.txt

[download]
Then, I ran a program counting for each word the number of occurrence of a 'Z': 
use strict;
use warnings;
use feature qw/say/;

my %histogram;
open my $IN, "<", "test_file.txt" or die "could not open input file $!
+";
while (<$IN>) {
    chomp;
    my ($key, undef, $val) = split / /, $_;
    $histogram{$key}++ if defined $val and $val eq 'Z';
}
print scalar keys %histogram;

[download]
Running it takes less than 14 seconds: 
$ time perl test_Z_bigfile.pl
113809
real    0m13.973s
user    0m11.734s
sys     0m2.140s

[download]
My input data file certainly does not look like yours, but I could count all entries ending with a "Z" from a 3-million-line and 3.5 GB test file in 14 seconds (on a simple laptop). I fail to see why it should take centuries on your side.

So, again, you're probably not telling everything or not doing it the right way.

In reply to Re^5: Nested greps w/ Perl by Laurent_R
in thread Nested greps w/ Perl by wackattack

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.