dichtfux has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks,

I am sending a request to a game server and getting some buffer back via UDP, which works fine. I'm now trying to parse the buffer.

I have a class name package_reader as a member of another class for this. Reading a string works fine: I have created the following function to do it:

Notes:
* $self->index() gets the current position in the buffer
* $self->size() gets the total length() the buffer
* $self->skip(n) sets index += n


## read a string until a '\x00' is encountered (-> null termination of
+ strings in C!)
sub read_complete_string {
    my ($self) = @_;
    my $string = "";

    ## return empty string if we're already at the end of the buffer
    if($self->index() >= $self->size()) {
        return $string;
    }

    my $end_of_string = index($self->buffer(), "\x00", $self->index())
+;

    if(!($end_of_string)) {
            ## just read the rest of the buffer if there is no termina
+tion
            $string =  $self->read_buffer($self->size() - $self->index
+());
    }
    else {
        $string = $self->read_buffer($end_of_string - $self->index());
        ## ignore the nullbyte '\x00'
        $self->skip(1);
    }

    return $string;
}

[download]
The function read_buffer() and skip() are used in the code above and therefor also shown here: 
## read the specified number of bytes from buffer, called by the other
+ read_* functions
sub read_buffer {
    my ($self , $user_length) = @_;
    my $length = 1;
    $length = $user_length if $user_length;

    if(($self->index() + $length) > $self->size()) { print "$tag: WARN
+ING: ##### read_buffer(): not enough data left, ignoring request to r
+ead $length bytes! #####\n"; return 0; }

    my $val = substr($self->buffer(), $self->index(), $length);
    $self->index($self->index() + $length);

    print "$tag: DEBUG: read_buffer(): read raw data '$val' of length 
+$length, now at $self->{_index} (" . $self->data_left() . " bytes of 
+" . $self->size() . " left)\n";

    return $val;
}

## skips the given number of bytes
sub skip {
    my ($self, $user_length) = @_;
    my $default_length;
    my $length;
    $default_length = 1;

    $length = $default_length;
    $length = $user_length if $user_length;

    $self->index($self->index() + $length);

    return $length;
}

[download]
I do now need to parse the following C struct from the data: 
struct client_t {
  short ping;
  int rate;
  char name[32]; // NULL-terminated
  byte clanTagPosition; // 0: prefix, 1: suffix
  char clanTag[32]; // NULL-terminated
  byte isBot;
};

[download]
And I'm trying to get the ping of this client with the following code: 
sub read_short {
    my ($self) = @_;

    my $sys_short_length = $Config{shortsize};
    my $short = unpack('S', $self->read_buffer($sys_short_length));

    return $short;
}


# elsewhere in the code:
print "ping: " . $self->package_reader->read_short() . "\n";

[download]
I'm getting a number sometimes, but most of the time it's just empty or nonsense.

Two questions:
1) Is the read_short() function broken or do I have to do anything special when I print its return value (like use printf and format it properly)?
2) How do I unpack the byte from the C struct shown above? How do I know how long it is (how many characters do I have to read from my buffer)? Which template for 'unpack()' do I use?

Thanks in advance,

dichtfux

Replies are listed 'Best First'.
Re: Parsing protocol data: unpack and bytes
by BrowserUk (Patriarch) on Dec 10, 2007 at 17:11 UTC

    You're making that much too complicated. To unpack the structure listed, you only need do something like: 

    my( $ping, $rate, $name, $tagP, $tag, $isBot ) 
    = unpack 'S N Z32 A1 Z32 A1', $buffer;

    [download]

    You might need to vary that somewhat depending upon the sending system. For example you might need V instead of N if the sending system is little-endian. Or you might need to include some padding.

    But work with Perl's facilities (eg.unpack), rather than fighting them as your code above is doing, and life gets much easier, clearer and quicker.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
      Thanks!

      I did't mean to fight unpack, I just wanted a wrapper class to be able to read from a buffer like this: 

      my $name = $self->package_reader->read_string();
my $rate = $self->package_reader->read_long();
...

      [download]

      Another question: with your method, I need to adapt the index of the buffer manually. How do I know how many characters 'unpack' read from the buffer? Do I have to calculate it myself?

      Like: 
      my $length_read = $Config{shortsize} + $Config{longsize} + ...;
my( $ping, $rate, $name, $ctpos, $clantag, $isbot ) = unpack('S V Z32 
+A1 Z32 A1', substr($self->package_reader->buffer(), $self->package_re
+ader->index(), $length_read));
$self->package_reader->index($self->package_reader->index() + $length_
+read);

      [download]
      ?
[reply]
[d/l]
[select]

        $Config{shortsize} and $Config{longsize} are useless. Those are the sizes of those types your machine/compiler uses. You need the values from the remote machine, which you'll need to hardcode.

        Similarly, I disagree with BrowserUK's use of s. You need to use the remote machine's byte order, so you need to use n or v. (Then you'll have to convert it from unsigned to signed, but that's easy: $ping = unpack('s', pack('S', $ping));.)

[reply]
[d/l]
[select]
        I just wanted a wrapper class to be able to read from a buffer like this:

        But that is exactly what I mean by "fighting unpack". The buffer has a fixed length--2 + 4 + 32 + 1 + 32 + 1 = 72--so deal with it that way.

        Your 'wrapper class' is unnecessary and overkill. Stop trying to write Java in Perl and just use Perl.

        That's just my opinion. You are, of course, perfectly within your rights to completely ignore it.

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

        Yes, I've long wanted "stream" semantics for pack/unpack. If they implemented a "give me the resulting offset" option, then such could be well implemented in a module.

        - tye        

[reply]

Back to Seekers of Perl Wisdom