in reply to Re^3: Testing with Test::Mock::HTTP::Tiny
in thread Testing with Test::Mock::HTTP::Tiny

Thanks for this bliako
For anyone stumbling this way in future, the method to create the file is captured_data, not mocked_data

Encoding to JSON gives a nice JSON file so that bit seems to be working :)

However, I am still getting an HTTP 599 error when I try to decode it :(

$/ = undef;
open my $fh, '<', 't/mock_html.dat' or die "Can't open datafile";
my $replay = <$fh>;
close $fh;

$replay = eval { decode_json($replay) };

ok ( !$@, 'Parsed JSON' ) or BAIL_OUT($@);

is ( ref($replay), 'ARRAY', '$replay is an ARRAY ' );

die "Nothing to replay" unless $replay;

Test::Mock::HTTP::Tiny->set_mocked_data( $replay );

diag ( Dumper (Test::Mock::HTTP::Tiny->mocked_data) );

my $crawl = WWW::Crawl->new(
    'timestamp' => 'a',
);

[download]

The diag gives the full JSON object as expected but Test::Mock::HTTP::Tiny is not feeding into the HTTP::Tiny->get call.

Replies are listed 'Best First'.
Re^5: Testing with Test::Mock::HTTP::Tiny
by bliako (Abbot) on Sep 28, 2023 at 09:47 UTC
    However, I am still getting an HTTP 599 error when I try to decode it :

    It works for me. To be sure, edit Test/Mock/HTTP/Tiny.pm (perldoc -l Test::Mock::HTTP::Tiny will tell you where this file is located) and put debugging messages after ## no critic. Just in case you haven't noticed, the return from my $resp = $http->get('http://www.way-finder.uk/'); is not an HTTP::response (which offers $resp->decoded_content) but a hash with keys headers, success, content, .... So you are looking for:

    my $content = $resp->{success} && length $resp->{content} ? $resp->{co
+ntent} : undef;
ok(defined $content, "got content");

    [download]
[reply]
[d/l]
[select]
      ...put debugging messages...

      Putting debugging messages in HTTP::Tiny has identified the problem 😀

      The mock HTTP data is working fine...but WWW::Crawl pulls links out of the page and then goes and looks at those. And those linked pages are not in the mocked data. The code is designed to deal with HTTP 404 errors for missing pages but Test::Mock::HTTP::Tiny gives HTTP 599 errors instead.

      Thanks for your help with this. I now need to find a solution where it ignores expected 599 errors and not unexpected ones...

[reply]
[d/l]
      Just in case you haven't noticed, the return from my $resp = $http->get('http://www.way-finder.uk/'); is not an HTTP::response

      Yes thanks - I had noticed that. At the point where we mock the data, we don't use $resp. But within the module we do this: 

      my $resp = $self->{'http'}->get($url);
if (!$resp->{'success'}) {
    croak "WWW::Crawl: HTTP Response " . $resp->{'status'} . " - " . $
+resp->{'reason'};
}

$page = $resp->{'content'};

      [download]
      It's here that the module is croaking when I try to use mocked data.

      perldoc -l Test::Mock::HTTP::Tiny

      That's helpful thanks...
      Definitely easier than searching manually!

      I'll add some debugging code a little later when more time is available. I'll try and get the module uploaded to GitHub later.

[reply]
[d/l]
[select]

        I don't know what goes on in WWW::Crawl. You have the added complication that you change the domain name. I would start from the Mock module, add debugging messages to see what urls/domains it has in store. What I understand you did is: 1) get data from domain A. 2) save it to Mock module with different domain B (how???). 3) try to retrieve mock data by using domain B OR fetch fresh. 4) croak. Well, it looks like step 3 fails: url is neither in Mock module nor out there in the web to be fetched fresh. The latter is true because it has a 'fake' domain. So not in store or domain change failed.

        Oh, in all my tests above I never changed the domain.

        1min edit: can it be that HTTP::Tiny within WWW::Crawl was not affected by Mock (which adds a callback for 'request'). In this case, if WWW::Crawl is yours add an option to have an $http passed on it rather than creating one fresh from inside the module. Long shot but I have no idea how this kind of inter-package interaction works.

[reply]
[d/l]

In Section Seekers of Perl Wisdom