Get URL problem

kepler has asked for the wisdom of the Perl Monks concerning the following question:

I have this small problem: I'm trying to query an api - a php script - with different parameters. Sometimes I can do that with a simple get($url) without any problem (actually I've done it a thousand times without any problem). But now, I'm having troubles doing it - I'm running a perl script from my windows PC with PerlID. From a common browser, I don't have any problems also. And I'm making consecutive requests since there's a limit of answers in each one. What might be the best way to achieve this? Is it a header problem? A timeout? Any help is very welcome.

Kind regards,

Kepler

Comment on Get URL problem

Replies are listed 'Best First'.
Re: Get URL problem by Corion (Patriarch) on Nov 03, 2018 at 07:03 UTC
If it "works with a common browser", you have to look at what data the working browser sends and then replicate that using Perl. Usually it is some header that you are not sending but that the browser is sending and which you need to replicate.	[reply]
Re^2: Get URL problem by kepler (Scribe) on Nov 03, 2018 at 12:59 UTC
Hi I believe you are right. How can I see and replicate in Perl the headers that are being sent?	[reply]
Re^3: Get URL problem by Corion (Patriarch) on Nov 03, 2018 at 13:01 UTC
That depends on the browser you use. For Firefox and Google Chrome, it's `ctrl+shift+i` to open the developer tools. For other application, it's usually https://www.wireshark.org/ for me.	[reply] [d/l]
Re: Get URL problem by bliako (Abbot) on Nov 02, 2018 at 22:58 UTC
an agent string maybe? LWP::UserAgent uses a default agent string which maybe is blocked at the server.	[reply]
Re: Get URL problem by localshop (Monk) on Nov 03, 2018 at 01:59 UTC
A code snippet may be helpful. I'm not really familiar with PerlID - could you provide a reference? Assuming you are using some kind of user agent, could you advise which. Could you describe your API URL - is it HTTP or HTTPS? Does it require custom authentication headers? What error code are you receiving or is it timing out? Are you able to provide a Data Dump of the response object?	[reply]