Telnet, XML API, and I/O Buffering

BluePerlDev has asked for the wisdom of the Perl Monks concerning the following question:

Greetings, again, O Wise Monks. I have a bit of an issue in dealing with an API that uses XML to drive it, and I know that there has to be a (relatively) simple way to do what I'm attempting. All the pertinent details are below:

Problem: Need to use the API of an aplication to get a list of items in its internal database.

facts:

The application and reporter programs are on two different servers
to interact with the API of the application, telnet (or a socket-based connection are needed

Analysis:

What I have so far is two different ways to approach this:

use a socket connection, with print and read functions to capture the output
use a socket connection and sysread/syswrite functions to capture the output

The problem that I am running into is, if I use print and read, the interactivity with the application suffers, and I can't seem to get it to recognize that I sent it a request, so my script just hangs.

Likewise, if I use sysread/syswrite, I get the interactivity, and get some of the output, but the XML::LibXML parser dies because I don't capture all of the output...

Code snippets:

This is what I'm working with:

my $NODEREC = " <XML REQUEST RECORD GOES HERE> "

my $sok = IO::Socket::INET->new(PeerAddr => $WYSHOST,
                                 PeerPort => $WYSPORT,
                                 Proto => "tcp")
or die "couldn't connect to WysDM host $WYSHOST : $!\n";

#$sok->autoflush(1);
$| = 1;

my $parser = XML::LibXML->new;
[download]

From there, the two options I have tried are:

print $sok "$NODEREC\n";
while ($line = <$sok>){
   print "reading line\n";
   chomp($line);
   push(@output, $line);
}
[download]

or, using sysread

syswrite $sok, $NODEREC;
syswrite $sok, "\n";
my $blksize = (stat $sok)[11] || 16384;
my $len = sysread $sok, $res, $blksize;
my $ds = $parser->parse_string("$res");
[download]

What recommendations can ou give me on the optimal way to perform this task?

Many Thanks in Advance

Comment on Telnet, XML API, and I/O Buffering Select or Download Code

Replies are listed 'Best First'.
Re: Telnet, XML API, and I/O Buffering by ikegami (Patriarch) on Jul 27, 2010 at 19:49 UTC
You need to know when the XML has been fully received. There are two general approach. Sentinel value Some special value marks the end of the message. This approach prevents the use of the value in the message without some form of escaping. The end of the file is a special sentinel value. For example, C strings. A NUL terminates the string. Length prefix The length of the upcoming message is present before the message. This approach requires the length of the message to be known before the sender can start sending it. For example, Perl strings. The length of the string is kept separate from the payload of the string.	[reply]
Re: Telnet, XML API, and I/O Buffering by ivo_ac (Acolyte) on Jul 27, 2010 at 20:31 UTC
Hi, If you have the book "Advanced Perl programming" (the one by Sriram Srinivasan) you could use the Msg toolkit explained there (chapter 13). Its also downloadable on CPAN: http://search.cpan.org/~sriram/examples/ It provides a wrapper around sysread which makes sure the complete message is received.	[reply]
Re^2: Telnet, XML API, and I/O Buffering by BluePerlDev (Novice) on Jul 29, 2010 at 19:48 UTC
I do have that book, and have tried to use the Msg module, but have met with slightly different results now... details below: First, the new code I am using: `use Msg; my $conn = Msg->connect($WYSHOST, $WYSPORT); die "Couldn't connect to $WYSHOST on $WYSPORT" unless $conn; $conn->send_now($NODEREC); $conn->send_now("\n"); ($res, $err) = $conn->rcv_now(); print Dumper($res), "\n"; print Dumper($err), "\n";` [download] and the results end up with $res and $err both being '': `$VAR1 = ''; $VAR1 = '';` [download] Should I view this as an error in the part f sending over the XML record that requests the data from the application? Or as some strange buffering issue on hte receiving host's side? again, many thanks in advance for all the help so far.	[reply] [d/l] [select]
Re^3: Telnet, XML API, and I/O Buffering by ivo_ac (Acolyte) on Jul 30, 2010 at 20:30 UTC
Hi, Did you implement the Msg toolkit on the sending side as well or only on the receiving side ? It has to be implemented on both sides because it enforces a kind of protocol. For example, one of the protocol requirements is that the server has to mark the beginning of each message with a four byte message length otherwise the rcv_now sub (which calls _rcv) doesn't work correctly. Hope this helps.	[reply]