bear0053 has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Monks,
here is the situation: i have text from a file stored in a db table. in my program i query that text out of the table and store it in a variable.

is there a way i can use that variable and iterate through it line by line (each line ends in \n) in order to process the text. (in some cases depending on what the line starts with i will need to get the next line as well)

i can get it to work fine if i use a filehandle and open it up for reading but unfortunately i do not have this luxury and the only access i have to the text of the original file is stored in a variable populated by a sql query. here is what i have so far:
foreach my $record (@files_to_process){ # $$record{data} is the way to reference data since array is a +n array of hash refs while(my $line = <$$record{data}>){ next if $line !~ /^06/; #if line isn't needed jump to next rec +ord my $line2 = <$$record{data}>; #get next line for in a way that + it won't be retrieved next time through from $line chomp($line); chomp($line2); #now do some processing on those 2 lines and then continue loo +p }
thanks ahead of time

Replies are listed 'Best First'.
Re: using a variable as a filehandle
by diotalevi (Canon) on Nov 01, 2004 at 18:25 UTC

    See open for the scoop on opening filehandles to stuff you already have in memory.</code>

    open my $fh, "<:raw:scalar", $record{'data'} or die "Couldn't open memory file: $!";
      Note: This requires perl-5.8.0 or greater.
      what is wrong with this: I am running perl 5.8.1.
      open (RF,"<:raw:scalar", $$record{data}) or die "Couldn't open memory + file: $!"; while(my $line = <RF>){ print "\n in while \n"; } close(RF);
      It doesn't die. (i verified there is data in the variable) and it never prints "in while" so it appears <> is finding no lines. any ideas thanks again

      Thanks! Its cool you can read from memory like filehandle (learn something enw everyday!!!) Once i get it to work i will be thrilled!!

        You might try changing the :raw to :crlf. Try also setting $/ to \n.

        open my $fh, "<:crlf:scalar", $record{'data'} or die ...; { local $/ = "\n"; while ( my $line = <$fh> ) { print "in line\n"; } }
Re: using a variable as a filehandle
by dragonchild (Archbishop) on Nov 01, 2004 at 20:10 UTC
    Take a look at IO::Scalar - that's the CPAN solution to your problem.

    Being right, does not endow the right to be rude; politeness costs nothing.
    Being unknowing, is not the same as being stupid.
    Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
    Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

Re: using a variable as a filehandle
by Sandy (Curate) on Nov 01, 2004 at 18:42 UTC
    #!/usr/bin/perl -w use strict; my @files_to_process= ( {data=>"The quick\n" . "brown fox\n" . "ran over\n" . "the purple\n" . "witch. So\n". "there!\n"} ); foreach my $record (@files_to_process) { while ($record->{data} =~/(.*\n)/g) { chomp (my $line = $1); my $second_line = ""; print "$line\n"; if ($line =~ /witch/) { if ($record->{data} =~ /(.*\n)/gm) { $second_line = $1; } print "-> $second_line\n" if $second_line; } } }
      First, I would to see some comparsion to split and why your way is better (which it is).

      Second, why not do:

      while (my ($line) = $record->{data} =~/(.*)\n/g) { ... }

      It saves on the chomp and is more concise, imho ...

      Being right, does not endow the right to be rude; politeness costs nothing.
      Being unknowing, is not the same as being stupid.
      Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
      Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

        That doesn't work, because you're changing the context of =~ from scalar to list. while ($record->{data} =~/(.*)\n/g) { my $line = $1; would be the equivalent without the chomp.

        However, neither solution will always work with Windows text files. See my reply to this post's grandparent.

        It saves on the chomp and is more concise, imho ...
        Yep, you're right. I was tired, and in too much of a hurry

        sigh...

      Unlike unix, it's common but not standard to put a \n on the last line in DOS/Windows. <FILE> handles the missing \n, but your code chops off the last line. The following handles both the presence and absense of the last \n:

      #!/usr/bin/perl -w use strict; my @files_to_process= ( {data=>"The quick\n" . "brown fox\n" . "ran over\n" . "the purple\n" . "witch. So\n". "there!"} ); foreach my $record (@files_to_process) { while ($record->{data} =~/(.*\n|.+$)/g) { chomp(my $line = $1); print "[$line]\n"; } }
      this worked perfectly thanks. I just have a question about how it works? it seems like the assigments of $1 remove that section from the original variable. is that correct? can someone explain how that works in order to keep the pointer for the while in the correct place.

      thanks soo much you were all a great help!
        Not quite. The g forces subsequent calls to the regexp to match from somewhere after the previous match ended, until no match is found.
        print("$1:") while ('abcdefgh' =~ /(.)/g); a:b:c:d:e:f:g:h: print("$1:") while ('vowels' =~ /([aeiou])/g); o:e:

        \G is similiar to ^, but refers to where the last match left off.

        print("$1:") while ('oiseau' =~ /\G([aeiou])/g); o:i:
Re: using a variable as a filehandle
by TedPride (Priest) on Nov 01, 2004 at 20:30 UTC
    Here's two ways of doing it:
    while () { $_ = substr($records, $p, index($records, "\n", $p) - $p); last if (!$_); $p += length($_) + 1; # do whatever with data now in $_ } while ($records =~ /(.*?)\n/g) { # do whatever with data now in $1 }
    I benchmarked both for 10,000 iterations on a 1,000 record string. The substr took 22 seconds vs 46 seconds for the regex. Both assume there's a \n on the end of the string.

    Using split directly cuts it to 10 seconds, but requires about twice the memory:

    for (split(/\n/, $records)) { # do whatever with data now in $_ }
Re: using a variable as a filehandle
by radiantmatrix (Parson) on Nov 01, 2004 at 19:48 UTC

    Is there a reason you are unable to write the file-in-memory to a temporary file-on-disk, then read that back in? Something like:

    open my $TEMPF, '>', 'tempfile.tmp' or die("Unable to write to tempfil +e"); print $TEMPF $file_contents; close $TEMPF; open my $TEMPF, '<', 'tempfile.tmp' or die("Can't read tempfile back i +n"); while (chomp (my $line = <$TEMPF>)) { next unless $line =~ /^06/; #skip unneeded lines chomp(my $line2 = <$TEMPF>); #process lines } close $TEMPF; unlink('tempfile.tmp');
    radiantmatrix
    require General::Disclaimer;
    "Users are evil. All users are evil. Do not trust them. Perl specifically offers the -T switch because it knows users are evil." - japhy
Re: using a variable as a filehandle
by elwarren (Priest) on Nov 02, 2004 at 00:19 UTC