in reply to Web scrapping

#!/usr/bin/perl - wT

Well, that line is not going to do what you think it is. Can you see what you have done wrong here?


🦛

Replies are listed 'Best First'.
Re^2: Web scraping
by joyfedl (Acolyte) on Jul 11, 2025 at 15:14 UTC

    At least i have tried something here is the try

    But still cant get the output and no response

    HTML Part to Scrap

    <div id="g_1_UFOgEYGu" class="event__match event__match--withRowLink e +vent__match--static event__match--twoLine" data-event-row="true"> <a href="https://www.flashscore.com/match/football/UFOgEYGu/#/matc +h-summary" target="_self" rel="nofollow" class="eventRowLink" aria-de +scribedby="g_1_UFOgEYGu" title="Click for match detail!"></a> <span class="event__check--hidden"></span> <div class="event__time">12.05. 17:00</div> <div class="wcl-participant_7lPCX event__homeParticipant" data-tes +tid="wcl-matchRow-participant"> <img data-testid="wcl-participantLogo" data-size="XXS" class=" +wcl-assetContainer_ZFzzG wcl-logo_EkYgo wcl-sizeXXS_-SmAO" alt="Brigh +ton" loading="lazy" src="https://static.flashscore.com/res/image/data +/G0q9xjRq-b92lfEJC.png" /> <span class="wcl-simpleText_Asp-0 wcl-scores-simpleText-01_pV2 +Wk wcl-name_3y6f5" data-testid="wcl-scores-simpleText-01">Brighton</s +pan> </div> <div class="wcl-participant_7lPCX event__awayParticipant" data-tes +tid="wcl-matchRow-participant"> <img data-testid="wcl-participantLogo" data-size="XXS" class="wcl-assetContainer_ZFzzG wcl-logo_EkYgo wcl-sizeXXS +_-SmAO" alt="Manchester City" loading="lazy" src="https://static.flashscore.com/res/image/data/0vgscFU0 +-lQuhqN8N.png" /> <strong class="wcl-simpleText_Asp-0 wcl-scores-simpleText-01_p +V2Wk wcl-bold_roH-0 wcl-name_3y6f5" data-testid="wcl-scores-simpleTex +t-01">Manchester City</strong> </div> <span class="wcl-matchRowScore_jcvjd wcl-isFinal_Am7cC event__scor +e event__score--home" data-testid="wcl-matchRowScore" data-state="fin +al" data-highlighted="false" data-side="1">1</span> <span class="wcl-matchRowScore_jcvjd wcl-isFinal_Am7cC event__scor +e event__score--away" data-testid="wcl-matchRowScore" data-state="fin +al" data-highlighted="false" data-side="2">4</span> </div>

    Full code

    #!/usr/bin/perl use Mojo::UserAgent; use Mojo::DOM; my $ua = Mojo::UserAgent->new; my $res = $ua->get('https://www.flashscore.com/football/england/premie +r-league-2018-2019/results/')->result; if ($res->is_success) { my $dom = Mojo::DOM->new($res->body); my @matches = $dom->find('div.event__match event__match--withRowLi +nk event__match--static event__match--twoLine')->each; # Iterate through the event__match event__match--withRowLink even +t__match--static event__match--twoLine and extract text foreach my $results (@matches) { my $Time = $results->find('div.event_ _time')->map('text')->jo +in; my $teamA = $results->find('div.wcl-participant_7lPCX event__h +omeParticipant')->map('text')->join; my $teamB = $results->find('div.wcl-participant_7lPCX event__a +wayParticipant')->map('text')->join; my $teamA_Scores = $results->find('span.wcl-matchRowScore_jcvj +d wcl-isFinal_Am7cC event__score event__score--home')->map('text')->j +oin; my $teamB_Scores = $results->find('span.wcl-matchRowScore_jcvj +d wcl-isFinal_Am7cC event__score event__score--away')->map('text')->j +oin; print "$teamA : $teamA_Scores: $Time"; print "$teamB : $teamB_Scores: $Time"; } } else { print "Cannot parse the result. " . $res->message . "n"; }

      Is this "AI" generated code? It looks like something it would come up with since it doesn't understand what the problem is, or have any domain knowledge. The content of the page is constructed after it's loaded. The table you're trying to parse simply doesn't exist at the point your code runs. The data 'is' in the page, see cjs.initialFeeds['results'], however this is very fragile and subject to change, even automatically. As usual 1nickt has given a sensible response, you could also use a headless browser such as WWW::Mechanize::Chrome to automate an actual browser to visit the page, waiting for everything to actually happen and acquiring the data via the query mechanisms it provides, or passing the complete HTML to your Mojo::DOM code. The sensible option is the free APIs already provided, rather than playing whack a mole with this other site.

        its not AI generated

        this is the reference where been learning how to web scrap and took so +me examples from https://brightdata.com/blog/web-data/web-scraping-wi +th-perl

        you can see that my code is similar to the examples on that web