senthil_v has asked for the wisdom of the Perl Monks concerning the following question:

Good Day Everyone!!

i was tried to login and read the webpage by using Curl command on Unix Os which i mentioned below. but mentioned command does produced the result but it was not returned the entire html structure of given url of the webpage. it has returned few html content along with javascript portion only.

Please advise me further on it.

curl --cookie cjar --cookie-jar cjar --data 'name=username' --data 'pass=pwd' --data 'form-id=user_login' --data 'op=Log in' --location --output ~/tmp/login.html https://tableau-dev.swissre.com /#/signin?disableAutoSignin=yes
[download]
  • Comment on Curl command does not return the entire html source of Secure Webpage
  • Download Code

Replies are listed 'Best First'.
Re: Curl command does not return the entire html source of Secure Webpage
by Corion (Patriarch) on Sep 01, 2016 at 07:49 UTC

    Where does Perl come into your problem?

    This is a site for help about Perl, but you don't seem to have Perl involved in any capacity.

    Have you tried the URL in a browser that does not support Javascript? The URL suspiciously looks as if the page would need Javascript to build up what you see as human. Disable Javascript and see if that changes the output you see in a browser. Then talk to the information providers if there is a better way to download the information, like as CSV file.

      Thanks Corion. i will try with Perl implement this concepts.
        Using the search you can find a few entries about Curl in Perl: Curl and Perl Correct way to POST with Perl Curl?

        In your case, you need to add your parameters to $string

        $string = "level=2&pass=pwd&form-id=user_login&op=Log in"; $string_length= length($string); $curl->setopt( CURLOPT_POSTFIELDS, "$string" ); $curl->setopt( CURLOPT_POSTFIELDSIZE, $length ); $curl->setopt( CURLOPT_POST, 1 ); $curl->setopt( CURLOPT_CONNECTTIMEOUT,8);

        as shown in the last post. Good luck!

        Click for a complete list of options you can set.