frazap has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to extract the 5 lines below the buttons (starting with localisation) in this page http://data.rero.ch/01-R008641003/html?view=FR_V1. With LWP or WWW::Mechanize the content extract from the url miss these lines. And the content return has 
 <div id="loader-item-R008641003">
   <i class="uk-icon-spinner uk-icon-spin"></i> Chargement
 </div>
 <div id="error-item-R008641003" class="uk-hidden uk-text-danger">
   <i class="uk-icon-warning uk-icon-small"></i> Erreur de chargement
 </div>

[download]
In my browser, displaying the source code of a newly charged page also miss these lines. Selecting the whole page and asking for the selection source return the html that was missing. Adding sleep(1) before fetching the response content does not help.

Any chance to fetch these lines with a perl script ?

Thanks

F.

Replies are listed 'Best First'.
Re: html code not in response but content display on a page
by Corion (Patriarch) on Jul 14, 2017 at 06:24 UTC

    Maybe that website uses Javascript to display the information or these elements?

    Have you tried visiting the site with Javascript switched off in your browser to see whether the elements behave as you expect?

    WWW::Mechanize does not support Javascript.

    You can try WWW::Mechanize::Firefox or WWW::Mechanize::Chrome to see if they can fetch the information from the page, as these support Javascript.

[reply]
      Well thanks: without javascript the lines are missing. Will try Mechanize::Firefox

      Done ... Work great !

      Thanks again F.

[reply]

Back to Seekers of Perl Wisdom