psyk0tic has asked for the wisdom of the Perl Monks concerning the following question:

Hi all wise Monks!

I need some help from your church in relation to socket data manipulation.

I've made a deamon that accept conections always from the same machine (HOST1), processes the data recieved and send SNMP traps to another machine (HOST2).

The problem is that I wrote it assuming that each time HOST1 has something to send it will connect send data and disconnect, but yesterday I learned we shouldn't make that kind of assumptions...

while (defined(my $line = <$NA_Conn>)) {
  $input.=$line;
}
#processing the $input here...

[download]

So, HOST1 never disconects, it connects to my deamon and whenever it has data to send it send's... But this way the $input is never processed.

What is the best way to process the input in 'real time'?

TIP: I need to send a trap each time I get a complete 'packet' with start and end flags: #S#\n\r<lines of data here >\n\r#E#

Threads? Multiprocess? Something simpler and effective?

PS: I would prefer stability to velocity.

Thanks in advance, AMEN Monks!

Replies are listed 'Best First'.
Re: Real-time socket data processing
by almut (Canon) on Jun 24, 2010 at 16:47 UTC

    Maybe I'm misunderstanding something, but why not simply put the processing in the loop?  (together with some condition that detects when a 'packet' is complete)

    Something like this: 

    while (defined(my $line = <$NA_Conn>)) {
  $input.=$line;
  if ($input =~ / ... /) {
    process($input);
    $input = '';
  }
}

    [download]
[reply]
[d/l]

      Thanks for the reply almut,

      Humm... I guess we're gonna for the ''simpler and efective way'' here ;)

      You're not misunderstanding nothing, it was me who was not seeing the 'almost' obvious. I will give it a try.

      Thanks again! :)

[reply]
Re: Real-time socket data processing
by Generoso (Prior) on Jun 24, 2010 at 17:45 UTC
[reply]

      I'm familiar with it... I've read it today morning...

      Thanks anyway Generoso.
[reply]

      My I suggest the following Tread example, this is for each time I get a connection I spin off a child process.

      The server:

      use strict;
use warnings;

use Socket;
use IO::Select;
#use Time::HiRes;

use threads;
use threads::shared;


$|  = 1;

# The following variables should be set within init_webserver_extensio
+n
use vars qw/
 $port_listen
/;

require "http_handler.pl";
init_webserver_extension();

local *S;

socket     (S, PF_INET   , SOCK_STREAM , getprotobyname('tcp')) or die
+ "couldn't open socket: $!";
setsockopt (S, SOL_SOCKET, SO_REUSEADDR, 1);
bind       (S, sockaddr_in($port_listen, INADDR_ANY));
listen     (S, 5)                                               or die
+ "don't hear anything:  $!";

my $ss = IO::Select->new();
$ss -> add (*S);


while(1) {
    my @connections_pending = $ss->can_read();
    foreach (@connections_pending) {
        my $fh;
        my $remote = accept($fh, $_);

        my($port,$iaddr) = sockaddr_in($remote);
        my $peeraddress = inet_ntoa($iaddr);

    my $t = threads->create(\&new_connection, $fh);
    $t->detach;
    }
}

sub extract_vars {
  my $line = shift;
  my %vars;

  foreach my $part (split '&', $line) {
    $part =~ /^(.*)=(.*)$/;

    my $n = $1;
    my $v = $2;
  
    $n =~ s/%(..)/chr(hex($1))/eg;
    $v =~ s/%(..)/chr(hex($1))/eg;
    $vars{$n}=$v;
  }

  return \%vars;
}

sub new_connection {
  my $fh = shift;

  binmode $fh;

  my %req;

  $req{HEADER}={}; 

  my $request_line = <$fh>;
  my $first_line = "";

  while ($request_line ne "\r\n") {
     unless ($request_line) {
       close $fh; 
     }

     chomp $request_line;

     unless ($first_line) {
       $first_line = $request_line;

      my @parts = split(" ", $first_line);
       if (@parts != 3) {
         close $fh;
       }

       $req{METHOD} = $parts[0];
       $req{OBJECT} = $parts[1];
     }
     else {
       my ($name, $value) = split(": ", $request_line);
       $name       = lc $name;
       $req{HEADER}{$name} = $value;
     }

     $request_line = <$fh>;
  }

  http_request_handler($fh, \%req);

  close $fh;
}

      [download]

      the request_handler

       
sub http_request_handler {
  my $fh     =   shift;
  my $req_   =   shift;

  my %req    =   %$req_;

  my %header = %{$req{HEADER}};

  print $fh "HTTP/1.0 200 OK\r\n";
  print $fh "Server: adp perl webserver\r\n";

  #print $fh "content-length: ... \r\n";

  print $fh "\r\n";

  print $fh "<html><h1>hello</h1></html>";

  print $fh "Method: $req{METHOD}<br>";
  print $fh "Object: $req{OBJECT}<br>";

  foreach my $r (keys %header) {
    print $fh $r, " = ", $header{$r} , "<br>";
  }
}

sub init_webserver_extension {
  $port_listen = 8888;
}


1;

      [download]
[reply]
[d/l]
[select]

        Hi Generoso,

        Thanks for the example! :) Actually I've managed to do it in the "simpler and effective" way... The parsing of data took a lot of work to work correctly, but it's done.

        With the approach I used it performs a little slowly, but I think it will do the job for now. If it will not perform well in the production site I'll just modify it and use the threaded approach.

        Thanks for your help.

        John
[reply]
Re: Real-time socket data processing
by Proclus (Beadle) on Jun 26, 2010 at 10:21 UTC
    You will find POE very helpful.
    It may be confusing at start but it will save you a lot of time once you get the hang of it.

    I have used POE::Component:SNMP for a few projects before along with POE::Component::Client::TCP, POE::Wheel::SocketFactory and POE::Wheel::ReadWrite.

    Good luck!
[reply]

      I'm sending the traps using the Solaris 'snmptrap' command, it saves a lot of work!

      I've never heared of this POE... Could indeed be useful in the future. Thanks for the tip!

      John
[reply]

Back to Seekers of Perl Wisdom