in reply to Re: Web scraping
in thread Web scrapping

At least i have tried something here is the try

But still cant get the output and no response

HTML Part to Scrap

<div id="g_1_UFOgEYGu" class="event__match event__match--withRowLink e +vent__match--static event__match--twoLine" data-event-row="true"> <a href="https://www.flashscore.com/match/football/UFOgEYGu/#/matc +h-summary" target="_self" rel="nofollow" class="eventRowLink" aria-de +scribedby="g_1_UFOgEYGu" title="Click for match detail!"></a> <span class="event__check--hidden"></span> <div class="event__time">12.05. 17:00</div> <div class="wcl-participant_7lPCX event__homeParticipant" data-tes +tid="wcl-matchRow-participant"> <img data-testid="wcl-participantLogo" data-size="XXS" class=" +wcl-assetContainer_ZFzzG wcl-logo_EkYgo wcl-sizeXXS_-SmAO" alt="Brigh +ton" loading="lazy" src="https://static.flashscore.com/res/image/data +/G0q9xjRq-b92lfEJC.png" /> <span class="wcl-simpleText_Asp-0 wcl-scores-simpleText-01_pV2 +Wk wcl-name_3y6f5" data-testid="wcl-scores-simpleText-01">Brighton</s +pan> </div> <div class="wcl-participant_7lPCX event__awayParticipant" data-tes +tid="wcl-matchRow-participant"> <img data-testid="wcl-participantLogo" data-size="XXS" class="wcl-assetContainer_ZFzzG wcl-logo_EkYgo wcl-sizeXXS +_-SmAO" alt="Manchester City" loading="lazy" src="https://static.flashscore.com/res/image/data/0vgscFU0 +-lQuhqN8N.png" /> <strong class="wcl-simpleText_Asp-0 wcl-scores-simpleText-01_p +V2Wk wcl-bold_roH-0 wcl-name_3y6f5" data-testid="wcl-scores-simpleTex +t-01">Manchester City</strong> </div> <span class="wcl-matchRowScore_jcvjd wcl-isFinal_Am7cC event__scor +e event__score--home" data-testid="wcl-matchRowScore" data-state="fin +al" data-highlighted="false" data-side="1">1</span> <span class="wcl-matchRowScore_jcvjd wcl-isFinal_Am7cC event__scor +e event__score--away" data-testid="wcl-matchRowScore" data-state="fin +al" data-highlighted="false" data-side="2">4</span> </div>

Full code

#!/usr/bin/perl use Mojo::UserAgent; use Mojo::DOM; my $ua = Mojo::UserAgent->new; my $res = $ua->get('https://www.flashscore.com/football/england/premie +r-league-2018-2019/results/')->result; if ($res->is_success) { my $dom = Mojo::DOM->new($res->body); my @matches = $dom->find('div.event__match event__match--withRowLi +nk event__match--static event__match--twoLine')->each; # Iterate through the event__match event__match--withRowLink even +t__match--static event__match--twoLine and extract text foreach my $results (@matches) { my $Time = $results->find('div.event_ _time')->map('text')->jo +in; my $teamA = $results->find('div.wcl-participant_7lPCX event__h +omeParticipant')->map('text')->join; my $teamB = $results->find('div.wcl-participant_7lPCX event__a +wayParticipant')->map('text')->join; my $teamA_Scores = $results->find('span.wcl-matchRowScore_jcvj +d wcl-isFinal_Am7cC event__score event__score--home')->map('text')->j +oin; my $teamB_Scores = $results->find('span.wcl-matchRowScore_jcvj +d wcl-isFinal_Am7cC event__score event__score--away')->map('text')->j +oin; print "$teamA : $teamA_Scores: $Time"; print "$teamB : $teamB_Scores: $Time"; } } else { print "Cannot parse the result. " . $res->message . "n"; }

Replies are listed 'Best First'.
Re^3: Web scraping
by marto (Cardinal) on Jul 11, 2025 at 18:41 UTC

    Is this "AI" generated code? It looks like something it would come up with since it doesn't understand what the problem is, or have any domain knowledge. The content of the page is constructed after it's loaded. The table you're trying to parse simply doesn't exist at the point your code runs. The data 'is' in the page, see cjs.initialFeeds['results'], however this is very fragile and subject to change, even automatically. As usual 1nickt has given a sensible response, you could also use a headless browser such as WWW::Mechanize::Chrome to automate an actual browser to visit the page, waiting for everything to actually happen and acquiring the data via the query mechanisms it provides, or passing the complete HTML to your Mojo::DOM code. The sensible option is the free APIs already provided, rather than playing whack a mole with this other site.

      its not AI generated

      this is the reference where been learning how to web scrap and took so +me examples from https://brightdata.com/blog/web-data/web-scraping-wi +th-perl

      you can see that my code is similar to the examples on that web

        While this page leaves a lot to be desired, the section Scraping Dynamic Websites does touch on the unreliability of basing parsing code on 'view source', since there's a great deal more to consider (previously mentioned domain specific knowledge, MDN Web Docs is a good starting point). I still automate scraping frequently, but only where there's no alternative sensible data source, see previous posts. If you can write it well, and once then this makes for a happier life than playing Whac-A-Mole with a sites such as this which frequently change their layout, html IDs and so fourth.