I would take a look at
HTTP LITE. It should be easy enough for Perl to download a web page, do a regexp scan for the files you're looking for, save that file to disk as index.html, and then start downloading all the other items you're looking for. Something like...
use HTTP::Lite;
my $http = new HTTP::Lite;
my $req = $http->request("http://www.something.com")
or die "Unable to get document: $!";
my $mirror_home = '/home/user/mirror_home/';
my (@javascript, @css, @jpg);
my $i = 0;
while ($http->body()){
if ($_ =~ m/*.jpg/){ push $_, @jpg;}
else if ($_ =~ m/*.js/){ push $_, @javascript;}
else if ($_ =~ m/*.css/){ push $_, @css;}
}
open FILE, "> $mirror_home/index.html"
or die "Couldn't open $mirror_home/index.html : $!";
print FILE $http->body();
close FILE;
while ($i <= $#css){
$req = $http->request("http://www.something.com/$css[$i]")
or die "Unable to get document: $!";
open FILE, "> $mirror_home/$css[$i]";
print FILE $http->body();
close FILE;
$i++
}
$i = 0;
# Then repeat for other extensions.
As a fair warning the above is definitely untested and probably horribly over-simplified, but the basic idea seems sound to me.
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
| |
For: |
|
Use: |
| & | | & |
| < | | < |
| > | | > |
| [ | | [ |
| ] | | ] |
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.