Basically what you need to do is to "fake the website out" into thinking that you are a normal browser.

There is more than one way to do this.

-You normally will need Crypt::SSLeay as usually these log-in pages are https (secure encrypted things).
-Sometimes you need a Cookie Jar, below this is a temporary thing.

Look at Mechanize as that is popular for this sort of stuff.

But the general idea is that the website thinks that you are Firefox 3.0, IE version whatever and you ask for a log-in page. Below I claim to be an ancient browser because the server sends me different stuff depending upon who I claim to be. If I claim to be IE 6, then I get different stuff and different responses are required from me. With IE a major thing is the "view state". Anyway basically claim to be as stupid a thing as you can get away with.

There are 2 ways to send stuff to a web server, one like this: http://www.perlmonks.org/?parent=790461;node_id=3333 which encodes what you want into the URL and "Post" which is a different way. A secure log-in will normally require POST.

Below I just copied the preamble to a LWP program that runs every day for me. It does a secure log-on to a server and then does some stuff that isn't relevant here.

In the case of this log-in form, 'user' => 'username', 'pwd' => "password" aren't really what these things are called in the actual HTML form that is received and there are a couple of other fields that I have to supply. Normally I have to experiment and look at the HTML code that is coming to me to get the form name right, etc.

#!/usr/bin/perl -w
use strict;

use LWP::UserAgent;
use Crypt::SSLeay;
use HTTP::Cookies;

print scalar(gmtime), " :Starting Killspam.pl\n";

my $url ='https://...big...URL....';

my $ua = LWP::UserAgent->new or die "Problem with the new UserAgent\n"
+;
$ua->cookie_jar(HTTP::Cookies->new);
$ua->agent("Mozilla/4.76 [en] (Windows NT 5.0; U)");
print "And now I'm calling myself ", $ua->agent( ), "!\n";

#
# Get the log-on page to the Secure Server
#
my $response = $ua->get($url) or die "Problem with the get $url\n";
$response->is_success or
    die "Failed to GET '$url': ", $response->status_line;
my $html_page = $response->content( );

print scalar(gmtime), " :Acessed log-on page\n";

# 
# post to the secure log-on page with my credentials
#
$response = $ua->post(
  'https://...some...url...',
  [
    'type'  => "voodoo",
    'p908' => "thing",
    'user' => 'username', 
    'pwd' => "password"
  ],
);

$response->is_success or die "Error: ", $response->status_line;
   #print $response->headers_as_string;
$html_page = $response->content( );
   #print $html_page;
   
#Then parse $html_page from here....
[download]