I haven't used your software before and did not have time to take a deep look at your page.. but it sounds like you want to do an awful lot. To say building "something like DBI" is a big job is an understatement. For example, why can you not use Net::FTP and other modules instead of reinventing them? Have you looked at inheriting from Net::Cmd?

Are you basically trying to implement file transfer over a bunch of different network services, and wrap it in a unified web interface?

You might like to look at IMS::ReleaseMgr which seems to do something similar to what you want to do.

There are also Penguin and One Penguin which claim to abstract away transport which could be email or whatever.

Maybe most useful would be to read LWP::Protocol module code. It does what you are talking about, finding what protocol is implemented, loading its module (just using require), blessing into a class, and dispatching a request and processing it. LWP/Protocol/http.pm shows one implementation. Actually you could look at LWP::Parallel::Protocol to answer your questions about scalability since you can run requests in parallel. Perhaps you would also like to look at LWP::Parallel::UserAgent. 

From LWP::Protocol..

sub new
{
    my($class) = @_;

    my $self = bless {
        'timeout' => 0,
        'parse_head' => 1,
    }, $class;
    $self;
}      

sub create
{
    my $scheme = shift;
    my $impclass = LWP::Protocol::implementor($scheme) or
        Carp::croak("Protocol scheme '$scheme' is not supported");

    # hand-off to scheme specific implementation sub-class
    return $impclass->new($scheme);
}   

sub implementor
{
    my($scheme, $impclass) = @_;

    if ($impclass) {
        $ImplementedBy{$scheme} = $impclass;
    }
    my $ic = $ImplementedBy{$scheme};
    return $ic if $ic;

    return '' unless $scheme =~ /^([.+\-\w]+)$/;  # check valid URL sc
+hemes
    $scheme = $1; # untaint
    $scheme =~ s/[.+\-]/_/g;  # make it a legal module name

    # scheme not yet known, look for a 'use'd implementation
    $ic = "LWP::Protocol::$scheme";  # default location
    $ic = "LWP::Protocol::nntp" if $scheme eq 'news'; #XXX ugly hack
    no strict 'refs';
    # check we actually have one for the scheme:
    unless (@{"${ic}::ISA"}) {
        # try to autoload it
        eval "require $ic";
        if ($@) {
            if ($@ =~ /Can't locate/) { #' #emacs get confused by '
                $ic = '';
            } else {
                die "$@\n";
            }
        }
    }
    $ImplementedBy{$scheme} = $ic if $ic;
    $ic;
}

[download]

By the way the Freshmeat link on your page doesn't work, on purpose I suppose.

I guess it would be easiest if you could limit the number of functions your system has and find a least common denominator between the services you mention. I would guess all you really want would be file directory listing and file download, with your system simulating user sessions to remember the login/password combination of a given user. Hope this is on the same track as what you are thinking about.

In reply to Re: Transparent file transfer module by mattr
in thread Transparent file transfer module by suaveant

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.