Re: Testing with Test::Mock::HTTP::Tiny

Test::Mock::HTTP::Tiny's set_mocked_data() expects an arrayref or a hashref. You are providing it with a scalar (string).

The captured data is indeed an arrayref as Test::Mock::HTTP::Tiny->captured_data_dump shows. But what you save to a file is a text representation of a Perl data structure. What you get when you read that data back from file is a scalar string. And that's what you pass to Test::Mock::HTTP::Tiny's set_mocked_data() which silently ignores you!

You need to "undump" the file contents somehow and resurrect the Perl data structure you started with. The simplest way to achieve this is by eval()'ing the text representation minus the $VAR1 = at the beginning of the string (placed there by Data::Dumper) (see the warnings at the end of this post):

$replay =~ s/^\$VAR1\s*=\s*//;
$replay = eval $replay; 
print "ref: ".ref($replay)."\n";
Test::Mock::HTTP::Tiny->set_mocked_data($replay);
[download]

You were unlucky in debugging -- in the source code of Test::Mock::HTTP::Tiny I see this:

    if (ref($new_mocked_data) eq 'ARRAY') {
      ...
    elsif (ref($new_mocked_data) eq 'HASH') {
      ...
    else {
        # TODO: error
    }
[download]

Apropos HTTP mocking, https://blogs.perl.org/users/e_choroba/2016/01/post.html by choroba presents an alternative.

WARNING: Be warned that eval()'ing things from files is a huge security hole. An alternative would be to use Storable to dump/freeze and resurrect/thaw a Perl data structure. But it too has boldface security warnings:

my $mocked_data = Test::Mock::HTTP::Tiny->mocked_data();
store $mocked_data, 'mock_html.dat';
...
my $replay = retrieve 'mock_html.dat';
print "ref: ".ref($replay)."\n";
Test::Mock::HTTP::Tiny->set_mocked_data($replay);
[download]

bw, bliako

Comment on Re: Testing with Test::Mock::HTTP::Tiny Select or Download Code

Replies are listed 'Best First'.
Re^2: Testing with Test::Mock::Tiny::HTTP by haukex (Archbishop) on Sep 27, 2023 at 09:26 UTC
You need to "undump" the file contents somehow and resurrect the Perl data structure you started with. The simplest way to achieve this is by `eval()`'ing the text representation minus the `$VAR1 =` at the beginning of the string (placed there by Data::Dumper) (see the warnings at the end of this post): Shameless plug: For many data structures, it's possible to do this more safely via my Data::Undump::PPI (see the disclaimer in Config::Perl). However, looking at the source, it would seem the simpler solution is to just serialize `Test::Mock::HTTP::Tiny->captured_data` oneself in a more suitable format, perhaps JSON or YAML (AFAICT from looking at the source, the data structure doesn't look like it contains Perl objects that would pervent serialization, though at the moment I don't have the time to test).	[reply] [d/l] [select]
Re^3: Testing with Test::Mock::HTTP::Tiny by bliako (Abbot) on Sep 27, 2023 at 10:19 UTC
`it would seem the simpler solution is to just serialize ...` yep, good point haukex++ as this is the easiest and most secure by far. Since I have more time: `use strict; use warnings; use HTTP::Tiny; use Test::Mock::HTTP::Tiny; use JSON; my $http = HTTP::Tiny->new; my $resp = $http->get('http://www.way-finder.uk/'); # EDIT: it's captured_data() not mocked_data() see below #my $json_str = eval { JSON::encode_json(Test::Mock::HTTP::Tiny->mocke +d_data()) }; my $json_str = eval { JSON::encode_json(Test::Mock::HTTP::Tiny->captur +ed_data()) }; die "failed to encode json" if $@; open my $fh, '>', 'mock_html.dat'; print $fh $json_str; close $fh;` [download] and use strict; use warnings; use Test::More; use Test::Mock::HTTP::Tiny; use WWW::Crawl; use JSON; plan tests => 1; $/ = undef; open my $fh, '<', 't/mock_html.dat' or die "Can't open datafile"; my $replay = <$fh>; close $fh; $replay = eval { JSON::decode_json($replay) }; ok(!$@, "parsed JSON replay data") or BAIL_OUT($@); is(ref($replay), 'ARRAY', "parsed JSON replay data is an ARRAY"); die "Nothing to replay" unless $replay; Test::Mock::HTTP::Tiny->set_mocked_data($replay); my $crawl = WWW::Crawl->new( 'timestamp' => 'a', ); my @links = $crawl->crawl('https://www.testing.crawl', \&link); cmp_ok ( scalar @links, '==', 8, 'Correct link count'); sub link { diag ($_[0]); } [download] note: eval() around JSON subs is because it dies on error last time I checked. bw, bliako	[reply] [d/l] [select]
Re^4: Testing with Test::Mock::Tiny::HTTP by Bod (Parson) on Sep 27, 2023 at 14:39 UTC
Since I have more time Thank you...I was also thinking that JSON might be the sensible solution here. eval() around JSON subs is because it dies on error last time I checked It took a moment, but I worked that out before reading your confirmation... But why fully qualify the methods that are exported by default? `$replay = eval { JSON::decode_json($replay) };` [download] and not `$replay = eval { decode_json($replay) };` [download]	[reply] [d/l] [select]
Re^5: Testing with Test::Mock::HTTP::Tiny by bliako (Abbot) on Sep 28, 2023 at 09:22 UTC
Re^6: Testing with Test::Mock::HTTP::Tiny by Bod (Parson) on Sep 28, 2023 at 11:12 UTC
Re^4: Testing with Test::Mock::Tiny::HTTP by Bod (Parson) on Sep 27, 2023 at 20:33 UTC
Thanks for this bliako For anyone stumbling this way in future, the method to create the file is `captured_data`, not `mocked_data` Encoding to JSON gives a nice JSON file so that bit seems to be working :) However, I am still getting an HTTP 599 error when I try to decode it :( `$/ = undef; open my $fh, '<', 't/mock_html.dat' or die "Can't open datafile"; my $replay = <$fh>; close $fh; $replay = eval { decode_json($replay) }; ok ( !$@, 'Parsed JSON' ) or BAIL_OUT($@); is ( ref($replay), 'ARRAY', '$replay is an ARRAY ' ); die "Nothing to replay" unless $replay; Test::Mock::HTTP::Tiny->set_mocked_data( $replay ); diag ( Dumper (Test::Mock::HTTP::Tiny->mocked_data) ); my $crawl = WWW::Crawl->new( 'timestamp' => 'a', );` [download] The `diag` gives the full JSON object as expected but Test::Mock::HTTP::Tiny is not feeding into the HTTP::Tiny`->get` call.	[reply] [d/l] [select]
Re^5: Testing with Test::Mock::HTTP::Tiny by bliako (Abbot) on Sep 28, 2023 at 09:47 UTC
Solved! (was: Re^6: Testing with Test::Mock::HTTP::Tiny) by Bod (Parson) on Sep 29, 2023 at 21:38 UTC
Re^6: Testing with Test::Mock::HTTP::Tiny by Bod (Parson) on Sep 28, 2023 at 11:08 UTC
Some notes below your chosen depth have not been shown here
Re^2: Testing with Test::Mock::Tiny::HTTP by Bod (Parson) on Sep 27, 2023 at 14:26 UTC
You were unlucky in debugging * kicks self * I looked at the source code and didn't pick that up...I must have been tired or having a senior moment! But, it begs the question... Should I be using this module at all? It hasn't been updated in 8 years It's unlikely to be on the target machine It's only used for testing Its documentation is virtually non-existant It appears unfinished I try not to get users to install non-core modules unless really necessary. Especially for testing. Be warned that `eval()`'ing things from files is a huge security hole Is this perhaps a security hole worth accepting because it is only used in testing? So the input file is known and anyone changing the file would be potentially sabotaging themself. You need to "undump" the file contents somehow and resurrect the Perl data structure you started with This sounds like the perfect use for a JSON object, especially as JSON::PP is core...	[reply] [d/l]
Re^2: Testing with Test::Mock::Tiny::HTTP by Bod (Parson) on Sep 27, 2023 at 14:31 UTC
Apropos HTTP mocking, https://blogs.perl.org/users/e_choroba/2016/01/post.html by choroba presents an alternative Ah yes... choroba shared that article when I was struggling with testing Business::Stripe::WebCheckout. I read it a few times but didn't understand it...I recall being totally in awe. I must revisit it again having gained a couple of years extra knowledge and experience. Thanks for bringing it back to my attention.	[reply]