in reply to Re^8: p0fq.pl and pack
in thread p0fq.pl and pack?

You changed the data. When I said I couldn't see how "L L ..." would have worked on the PC (or any machine), you told me the response was
0x0d 0xef 0xac 0xed 0x78 0x56 0x34 0x12 ...
Now you say it's either
0x0d 0xef 0xac 0xed 0x12 0x34 0x56 0x78 ... or
0xed 0xac 0xef 0x0d 0x78 0x56 0x34 0x12 ...
both different from what you told me.

And you didn't mention anything about the server's response being different based on the server on which it runs.

The problem you have is not related to pack, it's related to not knowing how the data is saved. The structure is meaningless. How the data is serialized is not dependent on its structure in memory. You need to know how the data is serialized. There's no escaping that.

In can be in terms of number of bytes, byte orderings, etc (at which point its dead simple to write the proper pack/unpack patterns), but it doesn't have to be.

If you know the library that was used to serialize the data, you could potentially use the same library or a port of it to deserialize the data.

Replies are listed 'Best First'.
Re^10: p0fq.pl and pack
by macli (Beadle) on Feb 23, 2007 at 20:33 UTC
    Ok, after reading perlpacktut, pack/unpack tutorial on perlmonks, I still can not figure out how to properly write the proper pack/unpack.

    PowerPC: 192.168.1.1
    PC: 192.168.1.2

    I am running p0f both on PowerPC and PC as:

    p0f -Q /var/run/p0f.sock -0 'dst port 443 and dst host 192.168.1.1'
    p0f -Q /var/run/p0f.sock -0 'dst port 443 and dst host 192.168.1.2'

    Here is a updated p0fq.pl script which used function DumpString to dump decmal, hex,and character of $query,$response,$magic,$type

    my $QUERY_MAGIC = 0x0defaced;
my $QTYPE_FINGERPRINT = 1;

die "usage: p0fq.pl p0f_socket src_ip src_port dst_ip dst_port"
  unless $#ARGV == 4;


sub DumpString {
    my @a = unpack('C*',$_[0]);
    my $o = 0;
    while (@a) {
        my @b = splice @a,0,16;
        my @d = map sprintf("%03d",$_), @b;
        my @x = map sprintf("%02x",$_), @b;
        my $c = substr($_[0],$o,16);
        $c =~ s/[[:^print:]]/ /g;
        printf "%6d %s\n",$o,join(' ',@d);
        print " "x8,join('  ',@x),"\n";
        print " "x9,join('   ',split(//,$c)),"\n";
        $o += 16;
    }
}

# Convert the IPs and pack the request message
my $src = new Net::IP ($ARGV[1]) or die (Net::IP::Error());
my $dst = new Net::IP ($ARGV[3]) or die (Net::IP::Error());
my $query = pack("L L L N N S S", $QUERY_MAGIC, $QTYPE_FINGERPRINT, 0x
+12345678,
                 $src->intip(), $dst->intip(), $ARGV[2], $ARGV[4]);
print "query:\n";
DumpString($query);
# Open the connection to p0f
my $sock = new IO::Socket::UNIX (Peer => $ARGV[0],
                                 Type => SOCK_STREAM);
die "Could not create socket: $!\n" unless $sock;

# Ask p0f
print $sock $query;
my $response = <$sock>;
close $sock;
print "response:\n";
DumpString($response);

# Extract the response from p0f
my ($magic, $id, $type, $genre, $detail, $dist, $link, $tos, $fw,
    $nat, $real, $score, $mflags, $uptime) =
  unpack ("L L C Z20 Z40 c Z30 Z30 C C C s S N", $response);
print "magic:\n";
DumpString($magic);
print "type:\n";
DumpString($type);
die "Bad response magic.\n" if $magic != $QUERY_MAGIC;
die "P0f did not honor our query.\n" if $type == 1;
die "This connection is not (no longer?) in the cache.\n" if $type == 
+2;
# Display result
print "Genre    : " . $genre . "\n";
print "Details  : " . $detail . "\n";
print "Distance : " . $dist . " hops\n";
print "Link     : " . $link . "\n";
print "Uptime   : " . $uptime . " hrs\n";

    [download]

    On PC:
    ./p0fq.pl /var/run/p0f.sock 192.168.1.1 0 192.168.1.2 443
    On Power PC:
    ./p0fq.pl /var/run/p0f.sock 192.168.1.2 0 192.168.1.1 443

    PC result:

    query:
     0 237 172 239 013 001 000 000 000 120 086 052 018 192 168 001 001
        ed  ac  ef  0d  01  00  00  00  78  56  34  12  c0  a8  01  01
                                         x   V   4
    16 192 168 001 002 000 000 187 001
        c0  a8  01  02  00  00  bb  01

response:
     0 237 172 239 013 120 086 052 018 000 076 105 110 117 120 000 000
        ed  ac  ef  0d  78  56  34  12  00  4c  69  6e  75  78  00  00
                         x   V   4           L   i   n   u   x
    16 000 000 000 000 000 000 000 000 000 000 000 000 000 050 046 054
        00  00  00  00  00  00  00  00  00  00  00  00  00  32  2e  36
                                                             2   .   6
    32 044 032 115 101 108 100 111 109 032 050 046 052 032 040 111 108
        2c  20  73  65  6c  64  6f  6d  20  32  2e  34  20  28  6f  6c
         ,       s   e   l   d   o   m       2   .   4       (   o   l
    48 100 101 114 044 032 052 041 000 000 000 000 000 000 000 000 000
        64  65  72  2c  20  34  29  00  00  00  00  00  00  00  00  00
         d   e   r   ,       4   )
    64 000 000 000 000 000 000 101 116 104 101 114 110 101 116 047 109
        00  00  00  00  00  00  65  74  68  65  72  6e  65  74  2f  6d
                                 e   t   h   e   r   n   e   t   /   m
    80 111 100 101 109 000 000 000 000 000 000 000 000 000 000 000 000
        6f  64  65  6d  00  00  00  00  00  00  00  00  00  00  00  00
         o   d   e   m
    96 000 000 000 000 104 105 103 104 032 116 104 114 111 117 103 104
        00  00  00  00  68  69  67  68  20  74  68  72  6f  75  67  68
                         h   i   g   h       t   h   r   o   u   g   h
   112 112 117 116 000 000 000 000 000 000 000 000 000 000 000 000 000
        70  75  74  00  00  00  00  00  00  00  00  00  00  00  00  00
         p   u   t
   128 000 000 000 000 001 000 156 255 000 000 000 000 074 003 000 000
        00  00  00  00  01  00  9c  ff  00  00  00  00  4a  03  00  00
                                                         J
magic:
     0 050 051 051 056 049 049 049 056 049
        32  33  33  38  31  31  31  38  31
         2   3   3   8   1   1   1   8   1
type:
     0 048
        30
         0
Genre    : Linux
Details  : 2.6, seldom 2.4 (older, 4)
Distance : 0 hops
Link     : ethernet/modem
Uptime   : 74 hrs

    [download]

    PowerPC result:

    query:
     0 013 239 172 237 000 000 000 001 018 052 086 120 192 168 001 002
        0d  ef  ac  ed  00  00  00  01  12  34  56  78  c0  a8  01  02
                                             4   V   x
    16 192 168 001 001 000 000 001 187
        c0  a8  01  01  00  00  01  bb

response:
     0 013 239 172 237 018 052 086 120 001 000 000 000 000 000 000 000
        0d  ef  ac  ed  12  34  56  78  01  00  00  00  00  00  00  00
                             4   V   x
    16 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
        00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00

    32 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
        00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00

    48 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
        00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00

    64 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
        00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00

    80 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
        00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00

    96 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
        00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00

   112 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
        00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00

   128 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
        00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00

magic:
     0 050 051 051 056 049 049 049 056 049
        32  33  33  38  31  31  31  38  31
         2   3   3   8   1   1   1   8   1
type:
     0 049
        31
         1
P0f did not honor our query.

    [download]
[reply]
[d/l]
[select]

      I looked into p0f's source. It appears the format is just a memory dump of the structures you showed earlier. 

      struct p0f_query q;
recv(c,&q,sizeof(q),MSG_NOSIGNAL)

struct p0f_response r;
send(sock,&r,sizeof(r),MSG_NOSIGNAL)

struct p0f_response* n;
send(sock,n,sizeof(struct p0f_response),MSG_NOSIGNAL)

      [download]

      In that case, Convert::Binary::C would be simpler to use instead of pack and unpack. (This falls under the "If you know the library that was used to serialize the data, you could potentially use the same library or a port of it to deserialize the data." case I mentioned earlier.)

      The layout of a C structure varies by system. If p0f and p0fq.pl are not both executed on the same machine, you'll need to configure your Convert::Binary::C object's ByteOrder and Alignment. 

      use strict;
use warnings;

use Convert::Binary::C qw( );
use IO::Socket         qw( );
use Net::IP            qw( );

use constant QUERY_MAGIC       => 0x0defaced;
use constant QTYPE_FINGERPRINT => 1;

my %p0h_arch = (
   #ByteOrder => ...,
   #Alignment => ...,
);

{
   die "usage: p0fq.pl p0f_socket src_ip src_port dst_ip dst_port"
      unless @ARGV == 5;

   my $c = Convert::Binary::C
           ->new(%p0h_arch)
           ->parse_file('p0f-query.h');

   # Convert the IPs and pack the request message
   my $src = Net::IP->new($ARGV[1]) or die (Net::IP::Error());
   my $dst = Net::IP->new($ARGV[3]) or die (Net::IP::Error());
   print "$ARGV[1]\n";
   my $query = $c->pack(p0f_query =>
      QUERY_MAGIC,
      QTYPE_FINGERPRINT,
      0x12345678,
      $src->intip(),
      $dst->intip(),
      $ARGV[2],
      $ARGV[4],
   );

   # Open the connection to p0f
   my $sock = IO::Socket::UNIX->new(
      Peer => $ARGV[0],
      Type => SOCK_STREAM,
   )
      or die "Could not create socket: $!\n";

   # Ask p0f
   print $sock $query;
   my $response = <$sock>;  # yuck!
   close $sock;

   # Extract the response from p0f
   my ($magic, $id, $type, $genre, $detail, $dist, $link,
       $tos, $fw, $nat, $real, $score, $mflags, $uptime) =
      $c->unpack(p0h_response => $response);
   die "Bad response magic.\n" if $magic != QUERY_MAGIC;
   die "P0f did not honor our query.\n" if $type == 1;
   die "This connection is not (no longer?) in the cache.\n" if $type 
+== 2;

   # Display result
   print "Genre    : " . $genre . "\n";
   print "Details  : " . $detail . "\n";
   print "Distance : " . $dist . " hops\n";
   print "Link     : " . $link . "\n";
   print "Uptime   : " . $uptime . " hrs\n";
}

      [download]
[reply]
[d/l]
[select]
        I appreciate your patience to guide me :) I don't know much about c and I am perl newbie too, but I feel I am understanding more and more under your guidance, I will try your script and get to know more about how to use Convert::Binary::C, please wait for my feedback :)
[reply]

        I tried your script, but it got:
        "Usage: Convert::Binary::C::pack(THIS, type, data = &PL_sv_undef, string = NULL) at ./p0fq.pl line 30." error

        I modifed your script as: 
        use strict;
use warnings;

use Convert::Binary::C qw( );
use IO::Socket;
use Net::IP qw( );
use Data::Hexdumper;

use constant QUERY_MAGIC       => 0x0defaced;
use constant QTYPE_FINGERPRINT => 1;

die "usage: p0fq.pl p0f_socket src_ip src_port dst_ip dst_port"
  unless @ARGV == 5;

my $c = Convert::Binary::C->new(
           LongSize => 4,
           ShortSize => 2,
           Alignment => 4,
           ByteOrder => 'BigEndian',
);

eval { $c->parse_file("p0f-query.h") };
if ($@) {
    die "Parse error: $@";
}

# Convert the IPs and pack the request message
my $src = Net::IP->new( $ARGV[1] ) or die( Net::IP::Error() );
my $dst = Net::IP->new( $ARGV[3] ) or die( Net::IP::Error() );
print "src ip:", ($src->ip()), "\n", "dst ip:", ($dst->ip()), "\n";
my $query = $c->pack(
    'p0f_query',
    {
        magic    => QUERY_MAGIC,
        type     => QTYPE_FINGERPRINT,
        id       => 0x12345678,
        src_ad   => $src->intip(),
        dst_ad   => $dst->intip(),
        src_port => $ARGV[2],
        dst_port => $ARGV[4],
    }
);

print "query:\n", hexdump( data => $query, );

# Open the connection to p0f
my $sock = IO::Socket::UNIX->new(
    Peer => $ARGV[0],
    Type => SOCK_STREAM,
  )
  or die "Could not create socket: $!\n";
# Ask p0f
print $sock $query;
my $response = <$sock>;    # yuck!
close $sock;
print "response:\n", hexdump( data => $response, );

# Extract the response from p0f
my $data = $c->unpack( 'p0f_response', $response );
die "Bad response magic.\n"          if $data->{magic} != QUERY_MAGIC;
die "P0f did not honor our query.\n" if $data->{type} == 1;
die "This connection is not (no longer?) in the cache.\n" if $data->{t
+ype} == 2;

# Display result
print "Genre    : " . $data->{genre} . "\n";
print "Details  : " . $data->{detail} . "\n";
print "Distance : " . $data->{dist} . " hops\n";
print "Link     : " . $data->{link} . "\n";

print "Uptime   : " . $data->{uptime} . " hrs\n";

        [download]

        Now the running result as:

        src ip:192.168.1.2
dst ip:192.168.1.1
query:
  0x0000 : 0D EF AC ED 01 00 00 00 12 34 56 78 00 00 00 00 : .........
+4Vx....
  0x0010 : 00 00 00 00 00 00 01 BB                                    
+      : ........
response:
  0x0000 : 0D EF AC ED 12 34 56 78 02 00 00 00 00 00 00 00 : .....4Vx.
+.......
  0x0010 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : .........
+.......
  0x0020 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : .........
+.......
  0x0030 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : .........
+.......
  0x0040 : 00 00 00 00 00 FF 00 00 00 00 00 00 00 00 00 00 : .........
+.......
  0x0050 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : .........
+.......
  0x0060 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : .........
+.......
  0x0070 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : .........
+.......
  0x0080 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : .........
+.......
This connection is not (no longer?) in the cache.

        [download]

        It is getting close, the strange thing is that the pack method seems eat up the src_ip, dst_ip, from the query hex dump, you can see src_ip and dst_ip are all 0x00, I am lost again.

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom