At it again...

sdyates has asked for the wisdom of the Perl Monks concerning the following question:

I feel the problem has to do with either buffering or messed up file issues. I rewrote the enter script. The output of the file A2052.php gives the following: . . .

<UL><I>Processor</I>
<LI>> 68EC030 @ 40 MHz Or 68030 @ 33 / 50 MHz, PGA
<LI>> Optional 68882 PGA FPU
<UL><I>Memory</I>
<LI>> One 72 Pin SIMM Socket Accepts 32 MB RAM
<LI>> Supports 4, 8, 16, 32 MB SIMMs, 60-70 Ns
<LI>> Burst RAM Access
<UL><I>Optional Modules</I>
<LI>> SCSI Controller
<LI>> NCR 53C80 Controller IC
<LI>> Does Not Use DMA Transfer
<LI>> Autoboot ROM (Csascsi.Device)
<LI>> DB25 External SCSI Connector
<LI>> Supported By NetBSD And OpenBSD
<LI>> Networking Controller


<LI>> Combination Of The Catweasel Mk2 Floppy Controller And The Buddh
+a Flash IDE Controller Built Into One Device
<LI>> Features All Buddha Flash And Catweasel Z-II Mk2 Functions
<LI>> Works With All A1200 Zorro Busboards


<LI>> 64 DIP Sockets Accept 2 MB RAM
<LI>> Supports 0.5, 1 Or 2 MB Configurations
<LI>> Accepts 256k&Times;1 DIPs Only
[download]

When it should only give:

<LI>> 64 DIP Sockets Accept 2 MB RAM
<LI>> Supports 0.5, 1 Or 2 MB Configurations
<LI>> Accepts 256k&Times;1 DIPs Only
[download]

At line 48, I add two returns so you can see how the script keeps appending to the array. If I clear the array at each time around, nothing of the list gets written to the OUTPUT. When I added print OUTPUT in lines 286 and 298, it would not print to the array at all, yet lines 284 and 299 print to the screen absolutely fine. This is completely bizarre... HELP PLEASE

use strict;
use warnings;

my $DeviceFileName="";
my $device="";
my $company="";
my $autoconfig="";
my $interface="";
my $amiga="";
my $date="";
my $title="";
my $point="";
my $format="";
my $SFS="";
my $FileInDirectory=0;
my $z=0;
my @dump;
my $directory="test";

###########################
### DirectoryFileSearch ###
###########################


if (substr($directory, -1,1) ne "\\") {
    $directory = "$directory" . "\\";
}
# Instead of using open, opendir takes all files with in a 
# directory and places them inside an array: @FileInDirectory
opendir(DIR, $directory) or die "Cannot open $directory";
my @FileInDirectory;
@FileInDirectory = readdir DIR;
close(DIR);
# removes '.' and '..' from the array. For file systems without
# these objects, comment out the following line.
splice(@FileInDirectory,0,2);
print @FileInDirectory;

#####################################################
### Reads files one file at a time into the array ###
### Each line is processed ne at a time.          ###
#####################################################

foreach $FileInDirectory(@FileInDirectory) {
    my $FilePath="";
    $FilePath = "$directory" . "$FileInDirectory[$z]";
    print "\n\nFile path: $FilePath  \n";
    push (@dump, "\n\n");
    open (SINGLE, $FilePath) or die "Cannot open '$FilePath'";
    while (<SINGLE>) {
        $SFS = $_;
        
        ##########################################################
        ## Process each each line. Parse required information   ##
        ## and place into scalars used to create new HTML file. ##
        ##########################################################
        
        ##########################################################
        ## Captures device name and creates filename for output ##
        ##########################################################
        if ( $SFS =~ m:<big>(.*?)</big>: ) {            # looks for <b
+ig></big and capturew text inetween
                $device = $1;
                $device =~ s/<[^>]*>//g;        # removes all characte
+rs between <> including <>
                $device =~ tr/""//d;            # removes double quote
+s
                print $device;
                $DeviceFileName = $device;
                $DeviceFileName =~ s/[^a-zA-Z0-9]//g;     #removes all
+ spaces & special characters
                print $DeviceFileName;
        } 


        ###########################
        ## Captures company name ##
        ###########################
        if ( $SFS =~ m:</font>(.*?)</small>: ) {    # looks for <big><
+/big and capturew text inetween
            $company = $1;
            $company =~ s/<[^>]*>//g;        # removes all characters 
+between <> including <>
            $company =~ tr/""//d;            # removes double quotes
        }

        ###################
        ## Captures Date ##
        ###################
        if ( $SFS =~ m:Date(.*?)</small>: ) {        # looks for <big>
+</big and capturew text inetween
            $date = $1;
            $date =~ s/<[^>]*>//g;            # removes all characters
+ between <> including <>
            $date =~ tr/""//d;            # removes double quotes
        }

        ###########################
        ## Captures amiga models ##
        ###########################
        if ( $SFS =~ m:Amiga(.*?)</small>: ) {        # looks for <big
+></big and capturew text inetween
            $amiga = $1;
            $amiga =~ s/<[^>]*>//g;            # removes all character
+s between <> including <>
            $amiga =~ tr/""//d;            # removes double quotes
        }

        ########################
        ## Captures interface ##
        ########################
        if ( $SFS =~ m:Interface(.*?)</small>: ) {    # looks for <big
+></big and capturew text inetween
            $interface = $1;
            $interface =~ s/<[^>]*>//g;        # removes all character
+s between <> including <>
            $interface =~ tr/""//d;            # removes double quotes
        }

        #########################
        ## Captures autoconfig ##
        #########################
        if ( $SFS =~ m:Autoconfig(.*?)</small>: ) {    # looks for <bi
+g></big and capturew text inetween
            $autoconfig = $1;
            $autoconfig =~ s/<[^>]*>//g;        # removes all characte
+rs between <> including <>
            $autoconfig =~ tr/""//d;        # removes double quotes
        }

        my $word =0; my $line =0; my $char = 0;
        
        #####################################################
        ## HTML main script informaiton with some scalars. ##
        ## Date, company and other info in $HTML2          ##
        #####################################################

        my $HTML1='<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transi
+tional//EN">
        <html>
        <head>
        <title>'. $device .' :: RetroGameAndComputer.com</title>
        <link rel="Shortcut Icon" href="favicon.ico">
        <meta property="fb:admins" content="654435532" />
        <META name="Abstract" content="Retro Amiga Storage Peripherals
+ '. $device .'">
        <META name="Description" content="The '. $device .' with speci
+fications, interface slot and front and back images.">
        <META name="KeyWords" content="'. $device .', '. $company .', 
+Amiga Storage Peripherals, expansion cards">
        <META name="Copyright" content="Copyright &copy; 2011 | RetroG
+ameAndComputer.com || All RightsReserved">
        <META http-equiv="content-language" content="en">
        <META http-equiv="Pragma" content="no-cache">
        <META http-equiv="Content-Type" content="text/html; charset=is
+o-8859-1">
        <META name="Robots" content="index,follow">
        <META name="Robots" content="noarchive">
        <META name="Googlebot" content="noarchive">
        <META name="MSNbot" content="noarchive">
        <META name="Security" content="Public">
        <META name="Distribution" content="Global">
        <link rel="StyleSheet" href="css/common-style.css" type="text/
+css">
        <link rel="StyleSheet" href="css/gray.css" type="text/css">
        <link rel="StyleSheet" href="css/gray-menu.css" type="text/css
+">
        <script language="JavaScript" type="text/javascript" src="css/
+css.js"></script>
        <script language="JavaScript" type="text/javascript" src="java
+scripts.js"></script>
        <script language="JavaScript" type="text/javascript" src="pop-
+closeup.js"></script>
        <script src="Scripts/AC_RunActiveContent.js" type="text/javasc
+ript"></script>
        <!-- Start Google Analytics -->
        <script type="text/javascript">
        
          var _gaq = _gaq || [];
          _gaq.push([\'_setAccount\', \'UA-21373706-3\']);
          _gaq.push([\'_trackPageview\']);
        
          (function() {
            var ga = document.createElement(\'script\'); ga.type = \'t
+ext/javascript\'; ga.async = true;
            ga.src = (\'https:\' == document.location.protocol ? \'htt
+ps://ssl\' : \'http://www\') + \'.google-analytics.com/ga.js\';
            var s = document.getElementsByTagName(\'script\')[0]; s.pa
+rentNode.insertBefore(ga, s);
          })();
        
        </script>
        <!-- End Google Analytics -->
        </head>
        <BODY BGCOLOR="#FFFFFF" TEXT="#000000">
        <!-- Start Outer Page Table -->
        <table bgcolor="#FFFFFF" align="center" cellpadding="0" cellsp
+acing="0" border="1" width="994">
          <tr>
            <td align="center" valign="top"><script language="JavaScri
+pt" type="text/javascript" src="header.js"></script>
              <!-- Start Picture Bar Table -->
              <table cellpadding="0" cellspacing="0" border="0" width=
+"100%">
                <tr>
                  <td background="picts/home.jpg" class="pictbackgroun
+d"><img src="img/headers/retrogameandcomputer.jpg" border="0" width="
+100%" height="80" alt="Image"><br></td>
                </tr>
                <tr class="printhide">
                  <td class="pagebars"><img src="picts/spacer.gif" wid
+th="10" height="1" alt="image"><br></td>
                </tr>
              </table>
              <!-- End Picture Bar Table -->
              <script language="JavaScript" type="text/javascript" src
+="menu.js"></script>
              <!-- Start Split Table -->
              <table cellpadding="0" cellspacing="0" border="0" width=
+"100%">
                <tr>
                  <td align="left" valign="top" class="sidebar-backgro
+und"><iframe name="Sidebar" src="sidebar.htm" width="187" height="600
+" frameborder="0" marginheight="0" marginwidth="0" scrolling="no" cla
+ss="sidebar-frame"> </iframe>
                    <!-- Start Left Sidebar Content Table -->
                    <TABLE cellpadding="10" cellspacing="0" border="0"
+ width="187" class="sidebartext">
                      <tr>
                        <td align="left" valign="top">
        
        <?php include("inc/google.search.inc.php"); ?>
        
        <br>
        <!-- Start Facebook iFrame-->                  
        <iframe src="http://www.facebook.com/plugins/like.php?href=htt
+p%3A%2F%2Fretrogameandcomputer.com&amp;layout=button_count&amp;show_f
+aces=true&amp;width=450&amp;action=like&amp;font&amp;colorscheme=ligh
+t&amp;height=21" scrolling="no" frameborder="0" style="border:none; o
+verflow:hidden; width:167px; height:21px;" allowTransparency="true"><
+/iframe>
        <!-- End Facebook iFrame-->                  
        <br><br><br>
        
        <!-- Place this tag in your head or just before your close bod
+y tag -->
        
        
        <!-- Place this tag where you want the +1 button to render -->
        <script type="text/javascript" src="https://apis.google.com/js
+/plusone.js"></script>
        <g:plusone></g:plusone><br>
                          <br>
                          
                          <p class="sidebartext" align="justify">Retro
+ Game And Computer covers retro game consoles, home computers, classi
+c video games and hardware peripherals.</p>
                          <p class="sidebartext" align="justify">We co
+ver all computers systems and game forms from the 1970s including Ata
+ri, Commodore, ZX Spectrum, Sega, Sony, Nintendo and more.</p>
                          
                          <br><div align="center"><a class="sidelink" 
+href="link-to-us.php"><img border="0" src="img/linktous.png" alt="Lin
+k To Us"></a></div>
                          
                          <br><p class="sidebartitle">Advertise With U
+s</p>
                          <p class="sidebartext" align="justify">Find 
+out how to advertise with RetroGameAndComputer.com. It\'s quick and e
+asy and we we have advertising solutions for any budget.</p>
                          <p class="sidebartext">:: <a class="sidelink
+" href="advertising.php">Advertise With Us</a></p>
                          <br>
        
        <br>
        <br>
        <script type="text/javascript"><!--
        google_ad_client = "ca-pub-1786059499637793";
        /* wide game site */
        google_ad_slot = "1105492661";
        google_ad_width = 160;
        google_ad_height = 600;
        //-->
        </script>
        <script type="text/javascript"
        src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
        </script>                
                          <p class="sidebartitle">Reciprocal Links</p>
                          <p class="sidebartext">We encourage a recipr
+ocal link policy. If you have a web site that relates to bass fishing
+ please feel free to link to us and let us know. We\'ll link back in 
+return. Visit our <a class="sidelink" href="link-to-us.php">Link To U
+s</a> page for details on how to link to us.</p>
                          <p class="sidebartext">:: <a class="sidelink
+" href="link-to-us.php">Link To Us</a></p></td>
                      </tr>
                    </table>
                    <!-- End Left Sidebar Content Table --></td>
                  <td align="left" valign="top" width="40" class="page
+height"><div id="cornerimage"> <img src="picts/spacer.gif" height="40
+" width="40" alt="image"><br>
                    </div>
                    <img src="picts/spacer.gif" height="200" width="10
+" border="0" alt="image"><br></td>
                  <td align="center" valign="top" class="shadow-horizo
+ntal"><br>
                    <br>
                    <!-- End Left Sidebar Content Table -->
        ';
        
        ##############################################
        ## All date header information starts here. ##
        ## List information is part of $HTML3.      ##
        ##############################################
        
        my $HTML2 ='<!--------------------------------------------->
        <!--          Start Main Secrtion            -->
        <!--------------------------------------------->
        
                    <table cellpadding=\"0\" cellspacing=\"0\" border=
+\"0\" width=\"100%\">
                      <tr>
                        <td align=\"left\" valign=\"top\" class=\"just
+\"><!-- Start Main Content Table -->
                          <p class=\"smalltext\"><a href=\"index.php\"
+>HOME</a>&gt;&gt; <a href=\"retro-peripherals.php\">Peripherals</a> &
+gt;&gt; <a href=\"amiga-peripherals.php\">Amiga</a> &gt;&gt; <a href=
+\"'. $device .'">Storage</a> &gt;&gt; </a> '. $device .'</p>
                          <h1>Amiga Storage Peripherals</h1>
                          <hr class=\"page-splits\">
                          <!-- Start Main Column Content - Single Colu
+mn -->
                          <table width=\"100%\" border=\"0\" cellpaddi
+ng=\"7\">
                            <tr>
                              <td>
        
        <h2><strong>'. $device .'</strong></h2>
        
        <p align=\"justify\">> Company: '. $company .'<br>
        > Date: '. $date .'<br>
        > Amiga: '. $amiga .'<br>
        > Interface: '. $interface .'<br>
        > Autoconfig ID: '. $autoconfig .'<br>
        </p>
        <br>
        <br>
        ';

        
        if ($SFS =~ m:<UL>(.*?)</I>:g) {
            $title = $1;
            $title = $title . "\n";
            $title =~ s/<[^>]*>//g;            # removes all character
+s between <> including <>
            $title =~ s/\b(\w)/\u$1/g;
            chomp $title;
            $format = "<UL><I>".$title."</I>\n"; $title = $format;
            print $title;
            push (@dump, $title);
#            print OUTPUT $title;
        }

        if ( $SFS =~ m:<LI>(.*?)$:g ) {            # looks for <big></
+big and capturew text inetween
            $point = $1;
            $point = "> ". $point . "\n";
            $point =~ s/<[^>]*>//g;            # removes all character
+s between <> including <>
            $point =~ s/\b(\w)/\u$1/g;        # Capitalises first word
            $format = "<LI>".$point."\n"; 
            $point = $format;
            chomp $point;
            push (@dump, $point);
#            print OUTPUT $point;
            print $point;
        }
            
        open OUTPUT, ">$DeviceFileName.php" or die "Cannot open $Devic
+eFileName";
        print OUTPUT $HTML1.$HTML2;
        print OUTPUT @dump;
        print OUTPUT "</UL>";
        close OUTPUT;

    }

    close SINGLE;
    $z = $z +1;
}
[download]

Comment on At it again... Select or Download Code

Replies are listed 'Best First'.
Re: At it again... by keszler (Priest) on Dec 06, 2011 at 01:35 UTC
First, kudos for using strict and warnings! Addressing only the `@dump` issue mentioned... You define `@dump` as a global variable. In the `foreach $FileInDirectory(@FileInDirectory) {` loop you add `"\n\n"` to `@dump` - that would be once per file. Then in the `while (<SINGLE>) {` loop you add `$title` and/or `$point` to `@dump` each time a line in the file matches the regex. When you start processing the 2nd file, `@dump` still contains whatever it got from the 1st. 3rd file, 1st and 2nd, etc. You need to clear out `@dump` between files. You stated: If I clear the array at each time around, nothing of the list gets written to the OUTPUT. I suspect you added something like `@dump = ();` after the `while (<SINGLE>) {` line, clearing it before processing each line of the file. Then, since the last line of the file didn't match, `@dump` was empty. The fix is to clear the array once per file, not once per line. . . . OK, I can't resist this one. You open OUTPUT, print to it, and close it in the while loop. If the file being processed has 172 lines, you're creating that file 172 times.	[reply] [d/l] [select]
Re: At it again... by Anonymous Monk on Dec 06, 2011 at 04:11 UTC
I feel the problem has to do with either buffering or messed up file issues ... That is kind of optimistic :) Going from Can't Write to Files properly to At it again... is one step forward, two steps backwards. Its good you've trimmed a lot from your program, but you still have a whole bunch of code which you're not interested in debugging. Get rid of it :) This time there is no sample input, but you can link to the old input, like junk.941732.part2.html, junk.941732.part3.html The title "At it again" doesn't describe what problem you're trying to solve :) See How do I post a question effectively? and How do I compose an effective node title? What I find confusing is, you didn't turn CountZeros solution into a function, you're still trying to parse UL by hand, why? You need more functions. Functions that take arguments, functions that do not depend on global variables. Read Help with locating bp region in chromosome and Return 2 arrays, sort the same, and concatenate them. Read my replies, the nodes I link and esp the readmore sections (code). Then start your project over with extracting data from html using xpath. While CountZeros solution might work and its already written, xpath is much easier, so you might as well switch now, you can use it for this whole project. Speaking of this project, there is a good chance this data is computer generated, and if you contact the admin at http://amiga.resource.cx/about.pl they can provide you with the output you desire, faster than you could write the program :) Good luck	[reply]