Re^5: Testing with Test::Mock::HTTP::Tiny

Replies are listed 'Best First'.
Solved! (was: Re^6: Testing with Test::Mock::HTTP::Tiny) by Bod (Parson) on Sep 29, 2023 at 21:38 UTC
...put debugging messages... Putting debugging messages in HTTP::Tiny has identified the problem 😀 The mock HTTP data is working fine...but `WWW::Crawl` pulls links out of the page and then goes and looks at those. And those linked pages are not in the mocked data. The code is designed to deal with HTTP 404 errors for missing pages but Test::Mock::HTTP::Tiny gives HTTP 599 errors instead. Thanks for your help with this. I now need to find a solution where it ignores expected 599 errors and not unexpected ones...	[reply] [d/l]
Re^6: Testing with Test::Mock::HTTP::Tiny by Bod (Parson) on Sep 28, 2023 at 11:08 UTC
Just in case you haven't noticed, the return from my `$resp = $http->get('http://www.way-finder.uk/');` is not an HTTP::response Yes thanks - I had noticed that. At the point where we mock the data, we don't use `$resp`. But within the module we do this: `my $resp = $self->{'http'}->get($url); if (!$resp->{'success'}) { croak "WWW::Crawl: HTTP Response " . $resp->{'status'} . " - " . $ +resp->{'reason'}; } $page = $resp->{'content'};` [download] It's here that the module is `croak`ing when I try to use mocked data. `perldoc -l Test::Mock::HTTP::Tiny` That's helpful thanks... Definitely easier than searching manually! I'll add some debugging code a little later when more time is available. I'll try and get the module uploaded to GitHub later.	[reply] [d/l] [select]
Re^7: Testing with Test::Mock::HTTP::Tiny by bliako (Abbot) on Sep 29, 2023 at 06:43 UTC
I don't know what goes on in `WWW::Crawl`. You have the added complication that you change the domain name. I would start from the Mock module, add debugging messages to see what urls/domains it has in store. What I understand you did is: 1) get data from domain A. 2) save it to Mock module with different domain B (how???). 3) try to retrieve mock data by using domain B OR fetch fresh. 4) croak. Well, it looks like step 3 fails: url is neither in Mock module nor out there in the web to be fetched fresh. The latter is true because it has a 'fake' domain. So not in store or domain change failed. Oh, in all my tests above I never changed the domain. 1min edit: can it be that HTTP::Tiny within WWW::Crawl was not affected by Mock (which adds a callback for 'request'). In this case, if WWW::Crawl is yours add an option to have an $http passed on it rather than creating one fresh from inside the module. Long shot but I have no idea how this kind of inter-package interaction works.	[reply] [d/l]
Re^8: Testing with Test::Mock::HTTP::Tiny by Bod (Parson) on Sep 29, 2023 at 20:05 UTC
I don't know what goes on in `WWW::Crawl` This is now on GitHub 2) save it to Mock module with different domain B (how???) I took the data from the domain and replaced the domain this data came from "https://www.onradar.uk" ¹ with the test domain "https://www.testing.crawl" using a simple search and replace. The replacement also changed the email domain in the page source but that's probably not relevant. ¹ - I've changed from using "www.way-finder.uk" in the original question to using "www.onradar.uk" now because the amount of data returned in the HTML source is smaller. Oh, in all my tests above I never changed the domain. I've changed the domain to one that doesn't exist on t'interweb so I can be sure the test code is using the mocked data and not going off and fetching a fresh page.	[reply] [d/l]
Re^9: Testing with Test::Mock::HTTP::Tiny by afoken (Chancellor) on Sep 29, 2023 at 21:57 UTC
Re^10: Testing with Test::Mock::HTTP::Tiny by Bod (Parson) on Sep 30, 2023 at 00:05 UTC
Re^8: Testing with Test::Mock::HTTP::Tiny by Bod (Parson) on Sep 29, 2023 at 21:32 UTC
Long shot but I have no idea how this kind of inter-package interaction works Looking at the source code of HTTP::Tiny the `get($url)` method calls `request('GET', $url)` which, in turn calls `_request('GET', $url)`.	[reply] [d/l] [select]