in reply to Capture::Tiny alternative

Well, I have made some investigations. Looks like sshuttle daemonizes in a bad way, here is my summary:

+-----------------------------+---------------------------------+---------------------------------+
| Method to read from process | does script hang when reading   | Realtime process stdout teeing  |  
|                             | from sshuttle -D is done  ?     | ( dumping to console )          | +-----------------------------+---------------------------------+---------------------------------+
 open( "$cmd | ")              yes                                yes 
 IPC::open3                    yes                                yes
 Capture::Tiny::capture        no                                 no
 Capture::Tiny::tee            yes                                yes
 IPC::Cmd::run                 yes                                yes               
 Direct run sshsulte from      no                                 yes
 terminal
+-----------------------------+---------------------------------+---------------------------------+

Further reading probably needs to be taken around function waitpid or parent -> demonized processes interaction or something like that. But unfortunately I am not a specialist in this field. So any help would be appreciated.

Replies are listed 'Best First'.
Re^2: Capture::Tiny alternative
by fishy (Friar) on Mar 25, 2017 at 12:08 UTC
    Hi melezhik,

    have you tried IPC::Cmd?
    Good luck!
      Hi Fishy. Will try it as well. Thanks.
        Well, IPC::Cmd::run hangs as well, will add it to summary table.
Re^2: Capture::Tiny alternative
by Anonymous Monk on Mar 25, 2017 at 19:08 UTC

    Is that sshuttle an interactive program or a daemon? What if you run it from command line piped: sshuttle -D | cat?

    Did you try to catch/ignore signals in your perl code? Like $SIG{'TSTP'} = 'IGNORE';

      In my case sshutle gets run as daemon (-D flag). If you run it as is from terminal it gets deamonized - you can see details at https://github.com/dagolden/Capture-Tiny/issues/52

      On "Did you try to catch/ignore signals" - good idea ... but I have not tried it yet. The overall challenge is _my_ code takes sshutle as input parameter, so I can't have any guesses on what the external program could be ( any ), I don't try to get any workarounds ( like catch/ignore signals ) either.

        The overall challenge is _my_ code takes sshutle as input parameter, so I can't have any guesses on what the external program could be ( any ), I don't try to get any workarounds ( like catch/ignore signals ) either.

        That's just not good. You might not have the source code of the program, but you do need to know how it behaves regarding IPC. A good example (if memory serves me well) in the bc program from UNIX. I think Perl documentation mentions that this kind of program is not reliable at all for IPC.

        Worst, some part of the program might be changed in the next release, and due buffering/signals handling it may broke down your IPC handling.

        But wait, you do have access to sshutle code... are you sure CPAN doesn't have something that implements the same you're looking for?

        If not, maybe you would be willing to implement a version in Perl as well. Or even fork the project and make the program behave well through IPC... or even use sockets instead!

        Alceu Rodrigues de Freitas Junior
        ---------------------------------
        "You have enemies? Good. That means you've stood up for something, sometime in your life." - Sir Winston Churchill
        In my case sshutle

        This sshuttle?

        gets run as daemon (-D flag)

        Why? If you want to interact with the program, you don't want it to "break all bridges", fork(2) itself away from your control, disconnect from STDIN, STDOUT, and STDERR. (See daemonize() in client.py.) Simply don't use that flag, sshuttle will work without it.

        If you want sshuttle to run after your script exits, consider using daemontools or one of its clones instead of reinventing the wheel. See also the djb way.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

        Well, we don't really know what your requirements are here, but if the task must run "as if on a command line" then you may have to spawn the jobs on their separate ptys. See IO::Pty. The program in question could be acting differently depending on isatty test.

      BTW.

      This one hangs. Though I can see some sshutle process afterwords

      $ /usr/sbin/sshuttle -v -D -r vagrant@127.0.0.1 192.168.0.0/24 | cat
      Starting sshuttle proxy.
      Listening on ('127.0.0.1', 12300).
      c : connecting to server...
      vagrant@127.0.0.1's password:
      c : connected.
      Connected.
      ^C # need to hit CTRL+C to exit
      
      $ ps uax|grep sshu
      $ ps uax| grep sshu
      root      7414  0.0  0.2  35148 10228 ?        Ss   23:03   0:00 python /usr/lib/sshuttle/main.py python -v --firewall 12300 0 --syslog
      root      7424  0.0  0.1  35524  7732 ?        S    23:03   0:00 python /usr/lib/sshuttle/main.py python -v -D -r vagrant@127.0.0.1 192.168.0.0/24
      

      And this one does not hangs. ( Similar output at STDOUT but result for ps uax|grep sshu afterwords abit different - two more processes seen )

      $ /usr/sbin/sshuttle -v -D -r vagrant@127.0.0.1 192.168.0.0/24
      Starting sshuttle proxy.
      Listening on ('127.0.0.1', 12300).
      c : connecting to server...
      vagrant@127.0.0.1's password:
      c : connected.
      Connected.
      # exit fine
      
      $ ps uax|grep sshu
      root      6860  0.0  0.0  23712  2472 pts/1    S    22:40   0:00 logger -p daemon.notice -t sshuttle
      root      6861  0.0  0.2  35148 10284 ?        Ss   22:40   0:00 python /usr/lib/sshuttle/main.py python -v --firewall 12300 0 --syslog
      root      6862  0.0  0.0  23712  2552 pts/1    S    22:40   0:00 logger -p daemon.notice -t sshuttle
      root      6899  0.0  0.1  35524  7724 ?        S    22:40   0:00 python /usr/lib/sshuttle/main.py python -v -D -r vagrant@127.0.0.1 192.168.0.0/24
      
      

        Interesting how? You have already called out the problem in that thread.

        ... there is no any special about besides it probably demonizes improperly. However this could be considered as example of such a "bad" external programs which Capture::Tiny interacts whit by IPC and which make it hangs ... Anyway probably we need to investigate this more ...
        In general, we don't need to investigate, it is upon you to provide a SSCCE or distill the bug for upstream report.

        • If you anticipate the need to run "bad" external programs, use the necessary precautions and containment.
        • If the program fails to properly redirect stderr when daemonizing, use a workaround a la foo -D 2>&-
        • Alternatively, look for other, well-behaved software to replace the broken tool.