Given the short transmissions and very simple connect/request/response/close nature of your connections; this application cries out for being written using non-blocking sockets and a select loop.

Forking a process (or spawning a thread) to handle such short-lived transactions simply doesn't seem necessary.

I'd offer some code or a reference to some, but as you've mentioned both *nix and windows and haven't identified where your proxy will run...

Thank you, your insight would be welcomed and appreciated! I mentioned the Windows server to frame the layout of the workflow. The project I am tasked with is for *nix only so it would be for *nix.

Further to your /msg; the reason I didn't offer you any code is because i don't use *nix; and using non-blocking and select on Windows is sufficiently different that it makes it non-portable and I don't have the knowledge to point out the differences; or support the transition.

However, here is the code for a very simple Windows select-mode server. It demonstrates the basic mechanisms involve though some of the details (the IOCTL call etc.) are not necessary or applicable on *nix. Maybe it will give you an overview:

#! perl -slw
use strict;
use IO::Socket;
use IO::Select;
use constant PACKET => pack 'C*', 0 .. 255;

print "Perl -v: ", $];
print "IO::Socket: ", $IO::Socket::VERSION;
print "IO::Select: ", $IO::Select::VERSION;
printf "%s\n", unpack 'H*', PACKET;

my $noBlock = 1;
my $lsn = IO::Socket::INET->new(
    Reuse => 1, Proto => 'tcp', Listen => 100, LocalPort => 12345
) or die "Server failed to create listener: $^E";
print "Listener created";
ioctl( $lsn, 0x8004667e, \$noBlock );
binmode $lsn;

my $sel = IO::Select->new( $lsn );

while( 1 ) {
    warn 'Registered handles:',  $sel->handles;
    warn 'Count: ', $sel->count();
    my @ready = $sel->can_read(10);
    warn "handles ready:[ @ready ]"; <STDIN>;
    for my $ready ( @ready ) {
        if( $ready == $lsn ) {
            my $client = $lsn->accept or next;
            binmode $client;
            ioctl( $client, 0x8004667e, \$noBlock );
            $sel->add( $client );
            warn "Added $client";
        }
        else {
            {
                $ready->recv( my $in, 4, 0 )              or warn( "re
+cv  failed: [$^E]" ), last;
                length $in == 4                           or warn( "Wr
+ong length: [$^E]" ), last;
                my $bytes = unpack 'N', $in;
                $ready->recv( my $dataIn, $bytes, 0 )     or warn( "re
+cv  failed: [$^E]" ), last;
                length $dataIn == $bytes                  or warn( "Wr
+ong length: [$^E]" ), last;
                $dataIn eq PACKET                         or warn( "Da
+ta corrupt: [$^E]" ), last;
                printf "Got: (%d) '%s'\n", $bytes, unpack 'H*', $dataI
+n;
                $ready->send( pack( 'N', 256 ), 0 )       or warn( "Se
+nd  failed: [$^E]" ), last;
                $ready->send( $dataIn, 0 )                or warn( "Se
+nd  failed: [$^E]" ), last;
            }
            $ready->shutdown( 2 );
            $ready->close;
            $sel->remove( $ready );
            warn "Removed $ready";
        }
    }
}

close $lsn;
[download]

Alternatively, you should do a site search (google: "site:perlmonks.org IO::Select"). IDs to look for are zentara and pg; bioth of whom have posted what I believe to be well written select-mode server code for *nix.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)

In the absence of evidence, opinion is indistinguishable from prejudice.

In my experience, you need to open 2 sockets: one to to listen for and accept incoming connections, and the other to connect to the other server.

If I'm understanding your 2 posts correctly, I would imagine a processing flow like:

1. Remote Server listens on a port for incoming connections.
2. When an incoming connection is detected, delegate a worker process/thread.
3. Worker accepts the connection while the Remote Server resumes waiting for incoming connections.
4. Worker receives the message and validates it.
5. If validation fails, worker sends a negative response, closes the connection then becomes idle (or terminates itself).
6. Otherwise, worker sends a positive response, closes the connection then processes the message.
7. If no results message needs to be sent, worker becomes idle (or terminates itself).
8. Otherwise, worker opens a socket, connects to your Local Server and sends the results message.
9. Worker waits for positive/negative response. Once received, closes the connection.
10. Worker performs any needed follow up based on the positive/negative response.
11. Worker becomes idle (or terminates itself).

In the case where the Remote Server is initiating the transaction, a worker would proceed as above as though it were sending a results message. Any results sent back from the Local Server would be handled be handled the same as accepting requests as above.

I am sure there are modules/frameworks in CPAN that can handle the network part, especially if the communication between the Remote and Local servers is HTTP based.

You certainly have a good grasp on the workflow of it!

I showed it in my other node but a visual of the process is like this(read from left to right, top to bottom for clarification):

# RQ = Incoming request sent to listening remote server

# RS = Response sent from local server to remote server
#      OR remote server to local server "flagging"
#      that the socket message was accepted by the server.
#     (Context of usage shown below)

# ST = Status of request at time of processing. More often than not,
#      this would be sent asynchronously from local server to remote
#      server.

Remote    Local
------    -----

RQ    --> 

      <-- RS

      <-- ST

RS    -->
[download]

If you can't find a suitable module/framework, Parallel::ForkManager could be used to maintain a pool of worker processes that you main server process can delegate to. At a minimum, the worker processes would need to be able to handle receiving any of the messages from you local server. Depending on how your code will interface with the ILS software, the workers may also need to be able to handle sending messages as well. Or the ILS software might be able to have it's own workers invoke the sending scripts directly. Or your send-side ILS interface could delegate to workers of it's own.

Forking is probably easier, especially if you use something like Parallel::ForkManager. Alternately, you could try to use threads. I can only assume that there is threads equivalent to Parallel::ForkManager you could use.

Is possible to to do this with out forking or threads by using IO::Select].

there are a lot of servers on cpan already, like mojo which supports websockets

One thing I haven't yet seen mentioned elsewhere in this thread is use of IO::Socket::IP, which can be used as a drop-in replacement for the more outdated IO::Socket::INET class. The newer version supports IPv6, so the only real reason to use the IPv4-only outdated version is when you're using a version of Perl before 5.20 (or need to maintain compatibility) and are unable to use the CPAN version of IO::Socket::IP.

Even if you don't have a current need to support IPv6, you make future expansion harder, and in today's Internet where IPv6 is (finally!) becoming more common, it's getting far harder to ignore. There's often little reason to avoid use of IO::Socket::IP since it supports both IPv4 and IPv6, and is also usable with the family-neutral functions provided by the Socket library (such as Socket::getaddrinfo.)

Consider if there's a good reason you want to write non-portable code that assumes IPv4 will be (exclusively) used everywhere your code will be.

e of IO::Socket::IP, which can be used as a drop-in replacement for the more outdated IO::Socket::INET class.

Is there a good reason why this is a new module (name) rather than integrated transparently into the IO::Socket::* hierachy?

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)

In the absence of evidence, opinion is indistinguishable from prejudice.

My feeling is you'd have to dig into the historical significance of IO::Socket::INET, and I don't have much of a personal interest. From what I've read of the IO::Socket::IP class, this is largely not the default for IO::Socket PF_INET due to reasons of legacy compatibility (eg: scripts that literally choke if their network socket is not of the exact class IO::Socket::INET.)

This said, see the docs for IO::Socket::IP since you can import it using use IO::Socket::IP -register which makes it take over constructor methods when creating either PF_INET or PF_INET6 sockets. This is as close to transparent integration as can be done without possibly breaking older scripts.

Perhaps someone else knows more about this history/legacy than I do.

Update: also note that as of Perl 5.20, IO::Socket::IP is part of Perl's core modules, per the release notes. Presumably this is so that IPv6 sockets work "out of the box" without requiring users of modern Internet sockets to go to CPAN for the replacement.

Interesting! I had not come across mention of that module anywhere. Could you provide an example of say a basic sample server?

:P IO::Socket::IP-> Reverse dependencies -> Mojolicious -> mojo, morbo, hypnotoad

UPDATE: So I was FINALLY able to get the basic part of it working but now I'm running into an odd problem that I can't seem to work out.

I got the following code to complete one full request from a Windows server basic Perl client to the Ubuntu machine OR I can telnet to localhost on the Ubuntu machine and type a manual command. Either option results in a connection refused message if I try to send a second request from either side and for the life of me I can't figure out how (or where) to fix it.

#!/usr/bin/perl -w
use strict;
use warnings;

use IO::Socket::INET;

my $sock = IO::Socket::INET->new(
    LocalPort => 8000,
    Listen      => 10,
    Reuse      => 1
);
die "Cannot create socket $!\n" unless $sock;

print "Server waiting for client to connect on port 8000\n";

$| = 1; # autoflush

my $client = $sock->accept();

my $handle;
my $clientHost = "10.20.0.30";
my $clientPort = "8000";

while(){

    while ($client) {
    
        # receive data, pass data via variable to dedicated subroutine
                # based on pattern matching
        my $recv_msg = &recv(); # handles recv() activity


        if ($recv_msg =~ /ADD/) {
            ADD( $recv_msg );
            # shutdown ($client, 2);
        }
        elsif ( $recv_msg =~ /STK/ ) {
            STK( $recv_msg );
            # shutdown ($client, 2);
        }
    }
}
# $sock->close();

sub connection {

    my ($host, $port) = @_;    

    $handle = IO::Socket::INET->new (
            PeerHost => $host,
            PeerPort => $port,
            Proto      => "tcp")
    or $handle = 0;

    if ($handle != 0) {
        $handle->autoflush(1);    # output gets there right away
    }
    else {
        print "Can't connect to host from sub connection.";
    }
    return $handle;
}

sub ADD {    

    my ( @data ) = @_;

    sendRequest($data[0]);
    print "Data after socket->send: $data[0]\n";
    print "This message is an ADD request.\n";

    $handle->close();
    $sock->close();
}

sub STK {    
    my ( @data ) = @_;

    sendRequest($data[0]);
    print "Data after socket->send: $data[0]\n";
    print "This message is a STK request.\n";

    $handle->close();
    $sock->close();
}

sub recv {
    my $msg = "";

    $client->recv($msg, 12000);

    return $msg;
}

sub sendRequest {
    my ($reqSock, $msg) = @_;
    &sender($reqSock, $msg);
}

sub sender {
    my ($reqMsg) = @_;

    $handle = connection($clientHost, $clientPort);

    eval {
        $handle->send($reqMsg);
    };
    print 'Crash: '.$@ if $@;

    if ($@) {
        print "Connection Reset";
        $handle->close;
        # $handle = connection($clientHost, $clientPort);
        eval {
            $handle->send($reqMsg);
        };
    }
}
[download]

Your proxy code is totally broken.

First, you only ever accept one connection:

my $client = $sock->accept();
[download]

As this is outside of any loop, you'll never even attempt to accept a second or subsequent connection.

And then, you never attempt to read anything from the connecting client:

while(){

    while ($client) {
[download]

The first line is an endless loop. The second line continues to loop whilst the variable $client has some value; which it always will, which means it is also and endless loop.

In the body of that loop, you are test $_ for the strings ADD and STK, but since you've never read anything from the client, $_ will never get any value, so neither string will be found and no action will ever be taken.

Effectively, your entire program accepts one connection and then loops endlessly doing nothing.

So quite how that "results in a connection refused message" is a mystery seeing as you are never even attempting to connect to the Windows machine?

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)

In the absence of evidence, opinion is indistinguishable from prejudice.

BrowserUk, thank you for jump starting my critical thinking! I see what I did wrong now thanks to you!

When you pointed out about how I accept only one connection into an endless loop it made me realize what I did wrong so I changed the following and, from what I can tell so far, it appears to be working so thank you for your criticisms!!!!

# I ended up changing this
my $client = $sock->accept();

# and this
while() {
    while ($client) {
        . . .
}

# to this
my $client;

# and this
while() {
    while ($client = $sock->accept) {
        . . .
}
[download]

and now it works for every request I tried so far! Thank you BrowserUk!

I never intend to connect directly to the machine from the module above. The module above is intended only to be a "processing center" if you will. Let's call this module above module B and the Windows machine module C.

A client, module A, connects to module B and sends a message of value "STK" to module B. Module B accepts the connection, performs pattern matching to determine what kind of message it is and passes it to the appropriate sub, and closes the socket. Then module B should go back to listening after it passes. The module B sub should connect to module C and send the message, then close the connection with C.

Thank you for all help on this problem! I have made progress with this issue! I ended up taking a different approach! Please see solution here