What exactly is your question? Is it whether this kind of site diffing has been done before? Or is it whether this kind of thing is doable? (it clearly is in Perl) Or is it a question of needing help on how best to do it?

Usually, you will get helpful feedback if you post code, to show what you have tried so far. Alternatively, post pseudocode showing your intended design. This is not a site where you can go and expect people to write substantial code for you for free.

However, if you just need some tips on which modules to look into, look at the following:

• WWW::Mechanize for doing the LWP donkey work
• HTML::TokeParser for parsing the HTML
• DBI and a DBD database driver
• Class::DBI to provide an object layer and save writing SQL
• VCS::Lite for the diffing

Hope this helps

#save $var->content as html file (to match with next day's file) in th
+e normal manner (not shown here)

$var=$user_agent->get('url') ;
$parsed_html=parse_html($var->content);
$txt_on_the_page=$parsed_html->format;

#for the earlier saved html 
$old_parsed_html=parse_html($oldfile->content);
$txt_on_the_old_page=$old_parsed_html->format;


# now compare $txt_on_the_page & $txt_on_the_old_page. I have put very
+ less code with very optimistic approach. But with little effort you 
+will get what you need.
[download]

The problem with this approache is that most of dynamic pages have in one place or another the date or/and current time so every time you download them they are automatically changed. To cope with that problem you need a mechanism for recognizing such trivial changes and that is not trivial - but I am currently writing a web application for that exact task: Active Bookmarks. You can download the code, it is GPLed.

Additionally at Yet another HTML diff just a few days ago I posted the code of the heart of the application.