in reply to how to open a data file via http

The http protocol does not directly support what you're trying to do. It's designed to return an entire file specified in an http request, not parts of a file. The easiest solution would be to slurp a copy of the file into a variable. Here are a couple of ways using LWP::Simple.

use LWP::Simple;

# get file into variable
my $data = get('http://www.jaywil.com/libsearch/SAMPLE.TXT');

# print all at once
print $data;

# or break up and print line by line
 # update: to keep blank lines at eof see comment by ikegami below 
@lines = split(/\n/, $data);
for (@lines){
    print "$_\n";
}

[download]

If for some reason you actually need to get the file one line at a time over http, you would need some help on the server side. If someone hasn't already written a module for this, you could write a cgi script that keeps state information and handles a few operations such as *open*, *read_line* and *close*. Basically you would be designing your own mini protocol to run over http.

Replies are listed 'Best First'.
Re^2: how to open a data file via http
by ikegami (Patriarch) on Feb 27, 2007 at 06:53 UTC

    You incorrectly defined a file as "a series of lines seperated by newlines", whereas it's really "a series of lines ending in newlines (except possibly the last one)".

    split(/\n/, $data) doesn't work. It removes blank trailing lines.
    split(/\n/, $data, -1) doesn't work either. It adds a blank line.
    split(/^/m, $data, -1) works. Bonus: It doesn't remove the newline!

    The last one can also be written as split(/^/m, $data) and special handling allows split(/^/, $data) to work too.

[reply]
[d/l]
[select]
      split(/\n/, $data) doesn't work. It removes blank trailing lines.

      What is your definition of "doesn't work" here? The dropping of empty trailing fields in the case of the LIMIT parameter being 0 or not specified works as advertised in the split docs.

      split(/\n/, $data, -1) doesn't work either. It adds a blank line.

      Perl doesn't do this on my system:

      >perl -MData::Dumper -e "$_=qq(1\n2\n3); print Dumper split /\n/,$_,-1
+"
$VAR1 = '1';
$VAR2 = '2';
$VAR3 = '3';

>perl -MData::Dumper -e "$_=qq(1\n2\n3\n); print Dumper split /\n/,$_,
+-1"
$VAR1 = '1';
$VAR2 = '2';
$VAR3 = '3';
$VAR4 = '';

      [download]

      When you read your data from file on the other hand, you have to be careful that your editor does not hide a trailing newline on the display. (E.g. Vim, where you can't tell the difference, other than by an initial [noeol] in the status line.)

[reply]
[d/l]
[select]

        What is your definition of "doesn't work" here?

        The goal was a replacement for <FILE>, so anything that doesn't return what <FILE> is buggy. In hangon's favour, I didn't consider autochomping a bug. 

        my $data = "\n"        # 1
         . "abc\n"     # 2
         . "def\n"     # 3
         . "\n"        # 4
         . "\n"        # 5
         . "ghi\n"     # 6
         . "\n"        # 7
         . "\n";       # 8

{
   print("<FILE>\n");
   my $lines = 0;
   open(my $fh, '<', \$data);
   while (<$fh>) {
      ++$lines;
      chomp;
      print("[$_]\n");
   }
   print($lines, "\n");
}

print("\n");

{
   print("split(/\n/, \$data)\n");
   my $lines = 0;
   foreach (split(/\n/, $data)) {
      ++$lines;
      print("[$_]\n");
   }
   print($lines, "\n");
}

print("\n");

{
   print("split(/\n/, \$data, -1)\n");
   my $lines = 0;
   foreach (split(/\n/, $data, -1)) {
      ++$lines;
      print("[$_]\n");
   }
   print($lines, "\n");
}

print("\n");

{
   print("split(/^/m, \$data)\n");
   my $lines = 0;
   foreach (split(/^/m, $data, -1)) {
      ++$lines;
      chomp;
      print("[$_]\n");
   }
   print($lines, "\n");
}

        [download]

        outputs 

        <FILE>
[]
[abc]
[def]
[]
[]
[ghi]
[]
[]
8       <-- This is what <FILE> gives.

split(/\n/, $data)
[]
[abc]
[def]
[]
[]
[ghi]
6       <-- Not 8. Failed to emulate <FILE>.

split(/\n/, $data, -1)
[]
[abc]
[def]
[]
[]
[ghi]
[]
[]
[]      <-- Extra line
9       <-- Not 8. Failed to emulate <FILE>.

split(/^/m, $data)
[]
[abc]
[def]
[]
[]
[ghi]
[]
[]
8       <-- Bingo!

        [download]

        wc -l says 8 as well, by the way.

        Perl doesn't do this on my system:

        uh, yes it does... $VAR4 = '';. Lines aren't *seperated* by newlines, they are *terminated* by them.

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom