Hi Monks! I have a script where I'm writing a lot of files. It slows my computer up quite a bit and I've done some reading on it and it looks like making the file handle hot might be the answer - as far as freeing up memory and speeding things up. I can see that $| looks to be the key. Any help on modifying the code below would to make the file handle hot so I'm not killing my computer would be greatly appreciated 

my $calls_dir3 = "Bing/1Parsed/Html2/";
    opendir( my $search_dir2, $calls_dir3 ) or die "$!\n";
    my @files = grep /\.txt$/i, readdir $search_dir2;
    closedir $search_dir2;
    print "Got ", scalar @files, " files\n";

    # proxies
    open my $fh2, '<', 'proxies.txt' or die $!;
    chomp( my @proxies = <$fh2> );
    close $fh2;

    foreach my $file (@files) {
        my %seen         = ();
        my $current_file = $calls_dir3 . $file;
        my $proxy        = shift @proxies;
        $proxy;
        print "Current proxy:$proxy\n";
        open my $FILE, '<', $current_file or die "$file: $!\n";


        my $x=1;
        make_path('Bing/1Parsed/Html3/');
        while ( my $row = <$FILE> ) {
            open my $fh1, ">", "Bing/1Parsed/Html3/$file.$x.html"
              or die("Could not open file. $!");

            #my $rnumber2  = rand(1999);
            $x = $x + 1;
            chomp $row;
            print "$row\n";
            my $xml1 = $row;

            $fh1->print ("<meta name=" . chr(34) . "keywords" . chr(34
+) . "content=" . chr(34) . ($row) . chr(34) .">");

            # create useragent
            my $ua = LWP::UserAgent->new;
            $ua->agent('Mozilla/5.0');
            $ua->timeout('180');

            # Use this UA/Proxy to fetch something....
            $ua->proxy( ['http'], 'http://' . $proxy );

            my $xml2 = get $xml1;
            $xml2;
            
            #add pause
            #my ( $x, $y ) = ( 1, 2 );
            #my $result = int( rand( $y - $x + 1 ) ) + $x;

            #print "pausing ";
            #print $result;
            #print "seconds";
            #sleep ($result);

            $fh1->print("\n");

            print $xml2;
            $fh1->print($xml2);

            close $fh1;
            $xml2 = 1;
        }
    }

[download]

In reply to Make a file handle hot by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.