cableguy has asked for the wisdom of the Perl Monks concerning the following question:

Hi folks.

I've got a file (http://home.iwcg.net/spectrum.bin) that has information about a radio observing program.

The first two bytes are a two-byte unsigned integer, which acts as an "index". When that number changes, there is new data. If you miss a number, you've missed a spectrum. The remaining data are 600 2-byte signed integers, representing the "dB" of 16384 channels. Each integer is in fact the maximum of 16384/600 = 27 bins. So the 600 integers are a boiled-down representation of the whole 16384 channel (73 kHz) bandwidth.

Now, with that said, when I'm going to get the data, and trying to split the data, my perl skills are only letting me get the binary data into hex....and the hex data is only so far. 

[snip header, locations, comments, and other fluff]
use LWP::Simple;
require LWP::UserAgent;

# First, let's get the data from the web server.

$ua = LWP::UserAgent->new();
$ua->agent("Argus Remote Waterfall v.01a");
$ua->timeout ($timeout);
my $response = $ua->get($fetchpage);

foreach(split(//,$response)) {
  if (!$response->is_success) {
       die $response->status_line;
  }
  printf("%01x ", ord($_));
  print "\n" if $_ eq "\n";
}

[download]

Now, when I run that code, I get: 

48 54 54 50 3a 3a 52 65 73 70 6f 6e 73 65 3d 48 41 53 48 28 30 78 38 3
+4 38 31 65 34 34 29

[download]

Not exactly 1202 bytes of data.

What am I missing? Where do I need to be looking? Any help would be greatly appreciated

Replies are listed 'Best First'.
Re: Splitting Binary Data
by ikegami (Patriarch) on Nov 24, 2004 at 16:51 UTC

    Shouldn't
    foreach(split(//,$response)) {
    be
    foreach(split(//,$response->content)) {

    if (!$response->is_success) { should be before the loop, not in it.

    What follows is the above fixes, in addition to unpacking two bytes at a time instead of one:

    if (!$response->is_success) {
   die $response->status_line;
}

my $content = $response->content();

$content =~ s/^(..)//;
# One of these, depending on byte ordering of the source.
my $index = unpack('n', $1);
#my $index = unpack('v', $1);
printf("index: %04x\n", $index);

while ($content =~ /(..)/g)) {
   # One of these, depending on byte ordering of the source.
   my $int16 = unpack('n', $1);
   #my $int16 = unpack('v', $1);
   $int16 -= 65536 if ($val > 32767);
   print($int, ' ');
}
print "\n";

    [download]

    Update: I wasn't converting them to signed numbers.

[reply]
[d/l]
[select]
Re: Splitting Binary Data
by trammell (Priest) on Nov 24, 2004 at 17:02 UTC 
    use strict;
use LWP::Simple;

my $bindata = get('http://home.iwcg.net/spectrum.bin');

my ($index,@data) = unpack('Ss*', $bindata);
my $count = @data;

print "index: '$index'\n";
print "data [$count]: @data\n";

    [download]
[reply]
[d/l]

      Warning: unpack 's' and unpack 'S' are not portable. That's why I recommended the use of unpack 'n' or unpack 'v'.

[reply]
[d/l]
[select]
        Yes, I did notice that. My perldoc also has 'n' and 'v' as both unsigned; the OP may have problems with this as well.

        A (long) while ago in grad school I was generating some data files containing a bunch of ints, and I was saving them in binary format. One of the better professors I worked with suggested I just save the data as ASCII text; admittedly there is a lot of space wasted, but ASCII portability is much less of an issue. He was right.

[reply]
Re: Splitting Binary Data
by Zaxo (Archbishop) on Nov 24, 2004 at 17:02 UTC

    This calls for unpack, 

    my ($number,@data) = do {
    my $fmt = 'N';
    $fmt .= ' n' x 600;
    unpack $fmt, $response;
};

    [download]
    I've assumed network order for the data. Change to 'V' and 'v' for little-endian.

    After Compline,
    Zaxo

[reply]
[d/l]
      N/V is is 32 bit unsigned, not 16 bit unsigned.
      n/v is is 16 bit unsigned, not 16 bit signed.
[reply]

Back to Seekers of Perl Wisdom