Thanks. Test::HTTP::Server certainly looks useful, though there is the issue of which server is contacted. And you are right, Test::LWP::UserAgent is pure perl, so there is nothing stopping me checking it into the source tree, other than an irrational worry that doing so could be the start of a slippery slope that will end with half of CPAN checked in, and the tree growing to many gigabytes in size.

In the end I decided that the simplest way to solve the problem was to directly mock the get and head methods in LWP::UserAgent. My mocked verson of head looks like this:

sub mocked_http_head
{
    my($self, $url) = @_;
    ( $url =~ m!http://(.*)/(.*)! ) or die "malformed URL";
    my $hostname = $1;
    my $file = $2;
    my $test_path = '/tmp/fake_server/'.$hostname.'/'.$file;
    my $response =  Test::MockObject::Extends->new('HTTP::Response');

    if( -f $test_path )
    {
        my $fakeHeaders = HTTP::Headers->new('content-length' => -s $t
+est_path );
        $response->set_true('is_success');
        $response->mock('headers', sub{ return $fakeHeaders } );
    }
    else
    {
        $response->set_false('is_success');
    }

    return $response;
}
[download]

Using this approach I can easily simulate as many servers as I like, each with different content. I can simulate wrong content or corrupt downloads by putting a file in the /tmp/fake_server directory that differs from the database. The only thing I can't easily simulate is a server that is not listening (so LWP times out) or an aborted download so that the size from the content-length header differs from the file actualy delevered.

The other thing this does not simulate is any server side scripting, but that is not a requirement for what I am doing.