mhutch7714 has asked for the wisdom of the Perl Monks concerning the following question:

Hello,

Am trying to add "features" to an existing C/Unix program to use a seperate Perl program for regex substitutions.

Saw the problem as a single bi-directional pipe, where

1. C program writes a string to the Perl program.

2. Perl progam transforms the string and write it back.

Mocked it up in C using socketpair(0, fork/exec, but how do I hook up C's descriptors with Perl's? (All of the examples I've found perform C<->C, or Perl<->Perl.)

Am I just making this harder than need be? Suggestions welcome.

Thanks in advance.

mhutch

Perl & C code follow: ----------------------------
HelloWorld.pl #!/usr/bin/perl -w # HelloWorld.pl pipe2 - bidirectional communication using socketpair # "the best ones always go both ways" use Socket ; use IO::Handle; # thousands of lines just for autoflush :-( # We say AF_UNIX because although *_LOCAL is the # POSIX 1003.1g form of the constant, many machines # still don't have it. socketpair(CHILD, PARENT, AF_UNIX, SOCK_STREAM, PF_UNSPEC) or die "socketpair: $!"; CHILD->autoflush(1); PARENT->autoflush(1); if ($pid = fork) { # parent #exit ; close PARENT; do { print CHILD "Parent Pid $$ is sending this\n"; print "Parent Pid $$ just read this: `$line`\n"; } while( chomp($line = <CHILD>) ) ; close CHILD; waitpid($pid,0); } else { # child die "cannot fork: $!" unless defined $pid; close CHILD; while( chomp($line = <PARENT>)) { print "Child Pid $$ just read this: `$line`\n"; print PARENT "Child Pid $$ is sending this\n"; } close PARENT; exit; } ---------------------------------------------------------- spair.c #include <stdio.h> #include <stdlib.h> #include <ctype.h> #include <errno.h> #include <unistd.h> #include <sys/types.h> #include <sys/socket.h> char * strupr( char * string ) ; /* Change string to uppercas +e */ int main(void) { int sv[2]; /* the pair of socket descriptors */ char buf[1024]; /* for data exchange between processes */ int i ; /* counter */ char *ch = "a" ; char * mydata[] = { "alfa", "bravo", "charlie", "delta", "echo" } ; if (socketpair(AF_UNIX, SOCK_STREAM, 0, sv) == -1) { perror("socketpair"); exit(1); } if (!fork()) { /* child */ /*execlp ("subsn.pl", "subsn.pl", 0);*/ /* Fire the substitution modul +e */ execlp ("HelloWorld.pl", "HelloWorld.pl", 0); /* Fire the substitution + module */ for ( i = 0 ; i < 5 ; i ++ ) { read(sv[1], &buf, sizeof( buf ) ); printf("child: read [%s]\n", buf); /*buf = toupper(buf);*/ /* make it uppercase */ strupr( buf ) ; /* make it uppercase */ write(sv[1], &buf, sizeof(buf)); printf("child: sent [%s]\n", buf); } } else { /* parent */ for ( i = 0 ; i < 5 ; i ++ ) { strcpy( buf, mydata[ i ] ) ; /* copy data to buffer */ write(sv[0], buf, sizeof( buf)); printf("parent: sent [%s]\n", buf ); read(sv[0], &buf, sizeof(buf)); printf("parent: read [%s]\n\n", buf); } wait(NULL); /* wait for child to die */ } return 0; } /* ** strupr : Change string to uppercase */ char * strupr( char * string ) { char * original = string ; while ( * string ) { * string = _toupper( *string ) ; string++ ; } return ( original ) ; }

Edited by planetscape - added readmore tags

( keep:1 edit:19 reap:0 )

Replies are listed 'Best First'.
Re: How to hook up C's socketpair() with Perl's?
by Fletch (Bishop) on Jul 14, 2006 at 01:58 UTC

    You should be able to call fileno on the descriptors and pass them as arguments (either command line or in the environment) and then use either dup(2) or open( HANDLE, "<&$fd" ) (see perlopentut and the docs for open for the exact syntax) to get a proper local endpoint in either C or Perl.

      Thanks for the reply. I'm on my way out the door, so I will dig through the doc tonight, and try it in the morning. mhutch
      Hi

      Instead of getting fileno from Environment Variable, you can write a program in such a way that read from Shared Memory.

      "Keep pouring your ideas"
        The OP stated that he is running this on unix. The 'disk emulation' concept of shared memory is available under linux and POSIX, but if the unix system isn't POSIX compliant, then the shared memory style to fall back on wouldn't be accessible via a device driver nor is it file-structured. - Oops I thought you meant use a shared memory file until I reread your post.

        -M

        Free your mind

Re: How to hook up C's socketpair() with Perl's?
by aufflick (Deacon) on Jul 14, 2006 at 08:42 UTC
    I don't know what your requirements are, but have you considered linking in the PCRE (Perl Compatible Regular Expressions) Library?

    http://www.pcre.org/

Re: How to hook up C's socketpair() with Perl's?
by polettix (Vicar) on Jul 14, 2006 at 14:27 UTC
    Hi, others (notably jesuashok) have already pointed out the pipe solution in case of colocated processes (which should be the case given the fork/exec approach). I'd only suggest to use dup or dup2 in the child process on the pipe handles, just before calling exec, to redirect the two pipes on STDIN and STDOUT. This would allow you to write an "agnostic" filter, i.e. a filter that reads from STDIN and writes on STDOUT - something that you could use in a canned sequence of pipes on the command line :)

    Just to make me clear, here's the C part:

    /* gcc callfilter.c -o callfilter */ #include <stdio.h> #include <sys/types.h> #include <unistd.h> #include <string.h> int main (int argc, char *argv[]) { int writer[2]; /* From the parent's point of view */ int reader[2]; /* From the parent's point of view */ int pid; char *msg = "hullo"; char buf[10]; int nread; /* Get pipes on */ if (pipe(reader)) { perror("reader pipe(): "); return 1; } if (pipe(writer)) { perror("writer pipe(): "); return 1; } pid = fork(); if (pid < 0) { perror("fork(): "); return 1; } else if (pid == 0) { /* In the child */ /* close unneeded endpoints */ close(reader[0]); close(writer[1]); /* Duplicate handles on STDIN and STDOUT for child */ dup2(writer[0], 0); dup2(reader[1], 1); /* call filter */ execl("./filter.pl", "./filter.pl", NULL); /* patched, thanks to + djp */ perror("execl(): "); return 1; } else { /* In the parent */ /* close unneeded endpoints */ close(reader[1]); close(writer[0]); /* Send something to the filter */ write(writer[1], msg, strlen(msg)); close(writer[1]); /* Read response from the filter */ printf("filter says:\n"); while ((nread = read(reader[0], buf, 9)) > 0) { buf[nread] = '\x0'; printf("%s", buf); } if (nread < 0) { perror("read(): "); return 1; } printf("\n"); close(reader[0]); } return 0; }
    and here's the Perl filter.pl (to be located in the same directory with the correct permissions, see execl above):
    #!/usr/bin/perl use strict; use warnings; local $/; my $in = <STDIN>; print {*STDOUT} scalar reverse $in;
    The result:
    poletti@PolettiX:~/sviluppo/perl/dup$ ./callfilter filter says: olluh

    Update: patched C example, thanks to djp.

    Flavio
    perl -ple'$_=reverse' <<<ti.xittelop@oivalf

    Don't fool yourself.
      execl("./filter.pl", NULL);
      should be
      execl("./filter.pl", "./filter.pl", NULL);
      Frodo72,

      You have nearly what I need! Perhaps I've made the problem more difficult than it needs to be.

      When I substitute my "filter.pl", it nearly works as needed.

      Can I trouble you for advice on how to trigger the pipe to "swallow" additional strings without closing writer1 (on your original line 48). I'll need to read a series of address strings from a database and attempt substitution on each.

      I test my filter.pl by firing from the comandline, after which I'm able to enter text, <cr>, and the substituted string appears. I enter <cr>, and it's ready for the next substitution. Ctl-D terminates it.

      When I invoke it from callfilter, the msg line substitutes, but later lines do not. Is this behavior due to closure of the file/pipe?

      Thanks in advance,

      mhutch

      (p.s. -- Any recommendations on where to RTFM about agnostic filters in perl?)
        The example was clearly only a "proof of concept", you can modify it to suit your needs. In particular you can close the pipes' ends when you really need it. In my example:
        else { /* In the parent */ /* close unneeded endpoints */ close(reader[1]); close(writer[0]); /* Send something to the filter */ write(writer[1], msg, strlen(msg)); close(writer[1]); /* HERE I CLOSE PARENT->FILTER */ /* Read response from the filter */ printf("filter says:\n"); while ((nread = read(reader[0], buf, 9)) > 0) { buf[nread] = '\x0'; printf("%s", buf); } if (nread < 0) { perror("read(): "); return 1; } printf("\n"); close(reader[0]); /* HERE I CLOSE FILTER->PARENT */ }
        I closed the file descriptors immediately when they were not needed any more. I wanted to send only one string, then I closed the writer[1]. But you can keep it open until you're done (note: the following code is "high-level"):
        else { /* In the parent */ /* close unneeded endpoints */ close(reader[1]); close(writer[0]); while (msg = next_string_to_filter()) { /* Send something to the filter */ write(writer[1], msg, strlen(msg)); /* Read response from the filter */ read_and_consume_answer(reader[0]); } /* Close communication with filter */ close(writer[1]); close(reader[0]); }
        A little note is due here about read_and_consume: be very, very careful about synchronisation between the two processes. You probably want to use the filter in a line-oriented fashion (from what I see from your post): be careful that read() blocks! Otherwise, you could prepare all the input (i.e. all the lines that you have to filter), send them all to the filter at once and read the answers. But I'll cut it here before someone reminds me that we're in PerlMonks, not CMonks ;)
        Any recommendations on where to RTFM about agnostic filters in perl?
        With agnostic I tried to express the fact that the filter does not have to know anything more than "read from STDIN, write to STDOUT", which is what any program normally does. I fear that you aren't going to find much about "agnostic filters" in any language :)

        More than this, be careful of buffering, use strict; and use warnings; ;)

        Flavio
        perl -ple'$_=reverse' <<<ti.xittelop@oivalf

        Don't fool yourself.
Re: How to hook up C's socketpair() with Perl's?
by sgifford (Prior) on Jul 14, 2006 at 15:43 UTC
    The file descriptors may be being closed on exec. See fcntl(2) and Fcntl, specifically the FD_CLOEXEC flag.

    The normal way to do this is as frodo72 describes: dup the pipe FDs to the child's STDIN and STDOUT. You may still have to turn off FD_CLOEXEC on them; at any rate it's probably safest and most portable.

Re: How to hook up C's socketpair() with Perl's?
by jesuashok (Curate) on Jul 14, 2006 at 07:29 UTC
    Hi Hi

    If both the programs are running on the same server, you can handle the scenario as follows:-

    open two pipe files, say pipe_1 and pipe_2
    Let C program uses pipe_1 pipe for writing and pipe_2 for reading
    Let perl program uses pipe_2 for writing and pipe_1 for reading.

    In this way both the program can talk each other.

    If both programs are running in different servers then you can use sockets.

    "Keep pouring your ideas"
Re: How to hook up C's socketpair() with Perl's?
by rafl (Friar) on Jul 15, 2006 at 15:54 UTC

    How about simply embedding a perl interpreter into your c program? perlembed has some examples for that.

    Cheers, Flo

Re: How to hook up C's socketpair() with Perl's?
by bsdz (Friar) on Jul 15, 2006 at 09:33 UTC
    Again, without knowing your requirements you may also want to consider Boost's regex library if you're comfortable with C++ instead of C.