I actually have a couple of web parsers that deal with this very issue, and to solve it, I have a sub called, lovingly, CreateSession.

It takes a useragent object and a hashref with security info, opens a cookie jar for the user agent, and traverses all the urls that are necessary to create the proper session information, posting certain authentication info, etc.

Some of the internals might look a little like this:

$ua->cookie_jar(HTTP::Cookies->new(file => "<PATH TO COOKIE FILE",
                                   autosave => 1));
   $ua->timeout(300);
my $req = POST 'https://<SOME URL>/login.cgi',
             [cdsid   =>$$secinfo{USERID}, 
                pw    =>$$secinfo{PWD},
             ];  

    my $res = $ua->request($req);
 
    unless ($res->is_success) {
        print "Login Failed: : ". $res->status_line .  "\n";
        return 0;
    }

    my @urls = qw(<SOME LIST OF NECESSARY URLS>);
    foreach my $url(@urls){
        my $req = HTTP::Request->new(GET=>$url); 
        my $res = $ua->request($req);
        unless ($res->is_success) {
         return 0;
        }
    }
[download]

From there you can use that UserAgent anywhere and the cookies are set up nicely, etc, etc.

C-.