clumsyjedi has asked for the wisdom of the Perl Monks concerning the following question:

I have a requirement to transfer some large (20GB) media files from desktop PCs to a remote FTP server. I need to authenticate the users using client certificates. Neither SFTP or FTP over SSH is an option. So I would like to do is something like:

So my questions:

Replies are listed 'Best First'.
Re: HTTP to FTP proxy
by salva (Canon) on Nov 18, 2010 at 11:41 UTC
    So you want to proxy FTP requests through HTTPS, right?

    I am almost sure this is natively supported by Apache. You will have to configure it to listen for SSL connections and the mod_proxy_ftp module to proxy FTP requests to the final destination.

    Then you will have to find a client supporting that configuration or roll your own!

[reply]
      From the mod_proxy docs. I interpret this to mean no uploads, although it isn't quite clear.

      How can I do FTP upload?

      Currently, only GET is supported for FTP in mod_proxy. You can of course use HTTP upload (POST or PUT) through an Apache proxy.

[reply]
        It seems you are right, PUT is not supported.

        Maybe you could use the CONNECT method instead and let the client talk directly to the FTP server.

        Something as drawn in the following diagram: 

         .---------------.               .------------------.
 | FTP Client    |               | Apache Listening |            .----
+------.
 | with HTTPS    |---------------| for SSL          |            | FTP
+      |
 | over SSL      |    SSL conn.  |                  |            | Ser
+ver   |
 | Proxy support |-------------------.           .---------------.    
+      |
 |               |       FTP conn    |<-CONNECT->|    FTP conn.  |    
+      |
 |               |-------------------'           '---------------'    
+      |
 |               |               |                  |            |    
+      |
 |               |---------------|                  |            |    
+      |
 |               |               |                  |            '----
+------'
 '---------------'               '------------------'

        [download]
[reply]
[d/l]
[select]
Re: HTTP to FTP proxy
by Illuminatus (Curate) on Nov 18, 2010 at 18:09 UTC
    1. You say you need to authenticate the users using client certificates. I assume that this does not mean that the FTP server requires it.
    2. The FTP protocol is implicitly file based (hence the 'F'). It only knows about files. You would need a client that had been modified (ok, hacked) to make the server believe it was getting a single file, while it was actually getting the data from the other connection. If the existing FTP server host is something you can modify, and you can run something like xinetd, it would be pretty easy to "roll your own" application to accept and store files.
    3. If you're stuck with FTP, but have the ability to have programs run on the FTP server (like via cron), you could file-buffer, eg filename.10meg.1, filename.10meg.2. You could write these files sequentially using FTP, then have a cron-based script look for filenames in this format and catenate them together after-the-fact.

      4. fnord

[reply]
      Not sure how to quote your comments sorry

      So...

      1. No, the FTP server is plain text username and password. The End user should not know these credentials.
      2. The protocol is for transferring files yes, but no local file is required to put a file on the remote server. The STOR command identifies the remote file name, the PORT or PASV commands identify network connections over which the data for that remote file should be read (this is my understanding anyway).
      3. No, I have no access to the remote machine, can't run code there. Can't change it's auth mechanisms. Can't do nuthin' :)
[reply]
Re: HTTP to FTP proxy
by NiJo (Friar) on Nov 18, 2010 at 20:35 UTC
    The standard approach does it the other way around:

    1) non-anonymous login to the ftp server

    2) which queries a password data base on the HTTP server

    3) and accepts or denies the password based ftp login.

    You didn't tell us much about the FTP and HTTP server and how much control you have over it. On Linux PAM or GSSAPI could be your ticket. On Windows it might be domain accounts.

    I've no idea how this could work with certificates.

[reply]
      The FTP server is firewalled, the user cannot connect to it directly.

      I have absolutely no control over the FTP server. I have limited control over the HTTPS server.

[reply]
Re: HTTP to FTP proxy
by clumsyjedi (Acolyte) on Nov 19, 2010 at 12:18 UTC
    In preparing an answer to fnord, I took a look at the implementation of Net::FTP and in turn Net::Cmd and it seems that the business of writing the file to the remote FTP server could be handled by using, or plagiarizing, these modules.
[reply]
      Also, for intercepting the HTTPS file upload I found the upload _hook of CGI.pm

      Between these two modules I think I have what I need.

[reply]
Re: HTTP to FTP proxy
by NiJo (Friar) on Nov 18, 2010 at 20:43 UTC
    Alternative:

    As you have some form of trust between HTTP and FTP server, what about a HTPP(S) upload to a mounted disk of the ftp server?

    That seems to be the most standard and efficient approach and turns it into a setup task for the admins.

[reply]
      Sadly no. A shared mount would be ideal, it is in fact exactly the setup we currently use, but we have been pushed into using a new hardware stack and this is no longer an option.
[reply]

Back to Seekers of Perl Wisdom