Script to scrap data (2nd part)

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi again Monks, A few days ago I posted a question about extracting the data of the table in https://desmace.com/provincia/asturias/ I got some good answers (thanks!) and now I think I am close of my goal. Looking at the DevTools, looks like it's a POST request with an AJAX call on admin-ajax.php?action=get_wdtable&table_id=414. Also, looking at the Payload, in the Form Data section there is all the info about the table columns and input values (starting with "draw..."). Finally, looking at Response we find a JSON with the results. To be precise, what I need is a script able to make the request, and also able to get the JSON response, which would be stored in txt, csv or similar. Any help would be highly appreciated. Many thanks in advance.

Comment on Script to scrap data (2nd part)

Replies are listed 'Best First'.
Re: Script to scrap data (2nd part) by Corion (Patriarch) on Dec 08, 2024 at 06:16 UTC
This is not a script writing service. Please tell us where you have problems. Also show the code you have already written. For converting a browser request to a Perl script, see for example curl-to-LWP, which lets you copy a request from your browser (as `cURL` command) and converts that into a Perl script. Please note that simply posting that Perl script here will not be enough - I expect you to put time into understanding the script and also understanding the modules you are using. Then, take a look at JSON (and/or Mojolicious::JSON, Cpanel::JSON::XS), and Text::CSV_XS.	[reply] [d/l]
Re: Script to scrap data (2nd part) by Anonymous Monk on Dec 08, 2024 at 20:22 UTC
What OS environment are you using?	[reply]