in reply to Curl command does not return the entire html source of Secure Webpage

Where does Perl come into your problem?

This is a site for help about Perl, but you don't seem to have Perl involved in any capacity.

Have you tried the URL in a browser that does not support Javascript? The URL suspiciously looks as if the page would need Javascript to build up what you see as human. Disable Javascript and see if that changes the output you see in a browser. Then talk to the information providers if there is a better way to download the information, like as CSV file.

  • Comment on Re: Curl command does not return the entire html source of Secure Webpage

Replies are listed 'Best First'.
Re^2: Curl command does not return the entire html source of Secure Webpage
by senthil_v (Sexton) on Sep 01, 2016 at 08:19 UTC
    Thanks Corion. i will try with Perl implement this concepts.
[reply]
      Using the search you can find a few entries about Curl in Perl: Curl and Perl Correct way to POST with Perl Curl?

      In your case, you need to add your parameters to $string

      $string = "level=2&pass=pwd&form-id=user_login&op=Log in";
$string_length= length($string);
$curl->setopt( CURLOPT_POSTFIELDS, "$string" );
$curl->setopt( CURLOPT_POSTFIELDSIZE, $length );
$curl->setopt( CURLOPT_POST, 1 );
$curl->setopt( CURLOPT_CONNECTTIMEOUT,8);

      [download]

      as shown in the last post. Good luck!

      Click for a complete list of options you can set.

[reply]
[d/l]

In Section Seekers of Perl Wisdom