in reply to Re^2: Running JavaScript from within Perl
in thread Running JavaScript from within Perl

Using GET /read/feed/$feed_url_or_id I can generate a web page containing the number of followers shown as "subscribers_count".

How do I read this page into a perl script? I tried HTML::TreeBuilder and got the error message:

https://public-api.wordpress.com/rest/v1/read/feed/http%3A%2F%2Fthe-art-of-autism.com%2Ffeed returned application/json not HTML

Should I use WWW::Mechanize::Chrome, JSON, JavaScript, or something else? How do I provide them input from a URL?

  • Comment on Re^3: Running JavaScript from within Perl

Replies are listed 'Best First'.
Re^4: Running JavaScript from within Perl
by marto (Cardinal) on Sep 17, 2019 at 06:24 UTC

    Here's a short example using Mojo::UserAgent:

    #!/usr/bin/perl use strict; use warnings; use Mojo::UserAgent; # your URL my $url = 'https://public-api.wordpress.com/rest/v1/read/feed/http%3A%2F%2Fthe-a +rt-of-autism.com%2Ffeed'; # create a Mojo:UserAgent my $ua = Mojo::UserAgent->new; # use $ua to get the url and assign the value of 'subscriber_count' in + the json # to avariable, $subscribers my $subscribers = $ua->get( $url )->result->json->{subscribers_count}; # print the variable to screen print "Number of subscribers: $subscribers\n";

    Running this returns:

    Number of subscribers: 181

    To install the required module, from the command line:

    cpanm Mojolicious (or if you don't have cpanm) cpan Mojolicious
Re^4: Running JavaScript from within Perl
by bliako (Abbot) on Sep 17, 2019 at 08:50 UTC

    To read json data from file, optionally utf8-encoded, into a Perl data structure I do:

    use strict; use warnings; # use this to enable script working with utf8 strings use utf8; # use this in order to print utf8 to console binmode STDOUT, ":utf8"; binmode STDERR, ":utf8"; # contains json-encode/decode routines use JSON; # contains utf8 encode/decode routines use Encode; # where json data is: my $jsonfile = 'data.json'; # open file with json my $INP; die "opening '$jsonfile'" unless open $INP, '<:encoding(UTF-8)', $json +file; # slurp file contents my $contents; { local $/ = undef; $contents = Encode::encode 'UTF-8', <$INP> } close + $INP; die "nothing to convert in '$jsonfile'..." unless $contents; # convert json text to perl data structure my $perldata = JSON::decode_json $contents; die "JSON::decode_json() has failed" unless $perldata; # dump the data for debugging use Data::Dumper; # only needed if you want to dump a data structure print Dumper($perldata); # access element of data print "image is ".$perldata->{'image'}."\n"; print "description (has unicode) is ".$perldata->{'description'}."\n"; print "meta/links/self is ".$perldata->{'meta'}->{'links'}->{'self'}." +\n";

    bw, bliako

      OP wants to hit an endpoint, they don't have a file.

        Thanks for pointing this out. The following code GETs the URL instead of reading contents from file. It can replace the file-reading section of my previous post:

        #use Encode; # used previously use LWP::UserAgent; my $ua = LWP::UserAgent->new(); my $URL = 'https://public-api.wordpress.com/rest/v1/read/feed/http%3A% +2F%2Fthe-art-of-autism.com%2Ffeed'; my $response = $ua->get($URL); die $response->status_line unless $response->is_success; my $contents = Encode::encode 'UTF-8', $response->decoded_content; ...

        All in all, Mojo::UserAgent solution by marto is more high-level. But I love plain LWP::UserAgent. Note: above can be made to authenticate and save cookies.