Web scrapping
5 direct replies — Read more / Contribute
|
by joyfedl
on Jul 10, 2025 at 12:57
|
|
|
I have a problem with my script. i cant output the response very well
this is the error i get
away_teamsARRAY(0x55fbf77894a0)timesARRAY(0x55fbf743be78)home_teamsARRAY(0x55fbf71ab7b0)scoresARRAY(0x55fbf71aba98)
this is the output i expect
times home_teams scores away_teams
12.05. 17:00 Brighton 1 - 4 Manchester City
Full code
#!/usr/bin/perl - wT
use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Request;
use HTML::TreeBuilder;
use Data::Dumper;
use CGI::Carp qw(fatalsToBrowser warningsToBrowser); # show errors in
+browser
print "Content-type: text/html\n\n"; # print output to browser
my $url = 'https://www.flashscore.com/football/england/premier-league-
+2018-2019/results/';
my $ua = LWP::UserAgent->new;
sub get_scores {
my $response = $ua->get($url);
if ($response->is_success) {
my $tree = HTML::TreeBuilder->new;
$tree->parse($response->decoded_content);
return $tree;
} else {
die $response->status_line;
}
}
my $results = get_scores();
my @times = map { $_->as_text } $results->look_down(_tag => 'div', cla
+ss => 'event__time');
my @home_teams = map { $_->as_text } $results->look_down(_tag => 'div'
+, class => 'event__participant event__participant--home');
my @scores = map { $_->as_text } $results->look_down(_tag => 'div', cl
+ass => 'event__scores fontBold');
my @away_teams = map { $_->as_text } $results->look_down(_tag => 'div'
+, class => 'event__participant event__participant--away');
my %dict_res;
for my $ind ($results) {
push @{$dict_res{'times'}}, @times[$ind];
push @{$dict_res{'home_teams'}}, @home_teams[$ind];
push @{$dict_res{'scores'}}, @scores[$ind];
push @{$dict_res{'away_teams'}}, @away_teams[$ind];
print %dict_res;
}
|
learning about warnings
2 direct replies — Read more / Contribute
|
by Ivonnita
on Jul 08, 2025 at 15:58
|
|
|
I am getting the following warning for a perl script to generate a volume: 1234surface 6900 surface WARNING:Your grid dimensions are mutually prime. Convergence is very unlikely. I understand this is not an error, but what does it mean? How much this warning can affect the accuracy or reliability of my results?
Thank you!
|
Reading tables in MS Word
2 direct replies — Read more / Contribute
|
by spiral
on Jul 08, 2025 at 15:06
|
|
|
I need to automate some tasks and this involves reading some MS Word files.
I recently came across MsOffice::Word::Surgeon.
Specifically what I want to do is read a word file, recognize tables in it and read tables, where specific columns may have relevant information, such as a key. Does anyone have or can give an example of this?
E.g.
use MsOffice::Word::Surgeon;
my $surgeon = MsOffice::Word::Surgeon->new(docx => $file);
my $main_text = $surgeon->document->plain_text;
# if I serialize this way, I lose table info
|
DBD::SQLite::db commit failed: database is locked; cpan shell error
1 direct reply — Read more / Contribute
|
by Intrepid
on Jul 08, 2025 at 14:11
|
|
|
When I installed CPAN::SQLite and set cpan opt use_sqlite to true, the next time I started cpan shell I got this error:
2025-07-08 12:40:23 (3.53 MB/s) - ‘/home/somian/.cpan/sources/modules/
+03modlist.data.gz.tmp11438’ saved [248/248]
Database was generated on Mon, 07 Jul 2025 15:41:26 GMT
Updating database file ... DBD::SQLite::db commit failed: database is
+locked at /usr/local/lib/perl5/site_perl/CPAN/SQLite/Populate.pm line
+ 663, <DATA> line 268361.
DBD::SQLite::db commit failed: database is locked at /usr/local/lib/pe
+rl5/site_perl/CPAN/SQLite/Populate.pm line 663, <DATA> line 268361.
Catching error: "system /usr/local/bin/perl -MCPAN::SQLite::META=setup
+,update,check -e update failed: 2816 at /usr/local/lib/perl5/site_per
+l/CPAN/SQLite/META.pm line 318.\cJ" at /usr/local/lib/perl5/5.40.1/CP
+AN.pm line 397.
CPAN::shell() called at -e line 1
EDIT
Oh, s**t. I had an earlier instance of CPAN's shell running in the background in a terminal. I'm writing this update very quickly so I can say NEVER MIND, before anyone decides to waste their time telling me what's wrong. ;-).
This is on Gnu/Linux, perl 5.40.1, CPAN.pm version 2.36. The error is repeatable; if I set use_sqlite via o conf, I will get the error.
I understand what a locked database is, although I haven't had to learn the
specifics of database management (I've long been an "enthusiast" - self-taught -
rather than a "professional" who would be required to know databases for w$rk). My
questions are: has anyone else seen this error with CPAN?
And, What's the magnitude of the performance penalty for not
using SQLite with CPAN?
Soren
Jul 08, 2025 at 18:11 UTC
|
File types not being picked up by script
1 direct reply — Read more / Contribute
|
by MShoaib
on Jul 06, 2025 at 11:33
|
|
|
Hi,
I am trying to pick files and directories from multiple directories and folders for further processing in my programme logic.
I am able to pick majority of the files and directories. However, with -f and all other remaining file operators (that I have already tried so far), I am unable to pick following type of files from the cache folder.
1: /(a directory)/4-http???wscont1.apps.microsoft.co?winstore?1.8x?2d60181a-b6d7-499a-9414-f07b0f932419?AppTile.1.396657.404061.png
2: /(a directory)/4-https???wscont.apps.microsoft.com?winstore?6.3.0.1?100?GB?en-us?MS?467?features1de1406b-b4ec-4d86-9066-68bf9c5d67f2.json
3: (another file path)etc.
Can someone advise, which code to use to pick this type of files being present in 'a directory'?
I am using following code:-
```
for (@files) {
$element = $_;
if (-d $_) {
push @directories, process_files ($_);
++$dir_counter;
} elsif (-f $_) {
++$file_counter;
next;
} elsif ($_ =~ /\.dat$/i || $_ =~ /\.png$/i) {
++$file_counter;
next;
} else {
print NOW "I am in else statement: process_files($_)\n";
} # else loop end
```
|
viability of new perl module
2 direct replies — Read more / Contribute
|
by Anonymous Monk
on Jul 05, 2025 at 20:31
|
|
|
I frequently use Python's matplotlib to make figures. I've made my own perl module for making quick plots in perl using matplotlib. I've found matplotlib to make figures inconsistently, and requires a lot more time for me to switch between the languages. So my perl scripts make a python temp file, and execute it. It's not super-efficient, I know, but it saves me time in both writing scripts and reading documentation.
Plotting a hash is very easy:
use Matplotlib::Simple 'plot';
plot({
'output.filename' => 'svg/single.barplot.svg',
data => { # simple hash
Fri => 76, Mon => 73, Sat => 26, Sun => 11, Thu => 94, T
+ue => 93, Wed => 77
},
'plot.type' => 'bar',
xlabel => '# of Days',
ylabel => 'Count (Applications)',
title => 'Rejections by Days',
});
My question: Should I put this module on CPAN?
|
Package VERSIONs not parseable by the cpan module
1 direct reply — Read more / Contribute
|
by Intrepid
on Jul 05, 2025 at 13:49
|
|
|
I don't see a better place than SoPW at the Monastery to put this so I'll
just dump it here and see what happens.
Just a simple inquiry: I just ran cpan (not cpanplus, which I post about a lot); I
entered the command r for "reporting outdated/upgradeable modules" and cpan
reported 92 unparseable module versions. Whoa. That makes me want to ask: is
it really that difficult to write a package with a sane / parseable version? Or is
cpan's programmatic notions of "parseable" odd or too strict? Opinions and
knowledge welcomed.
Soren
Jul 05, 2025 at 17:47 UTC
A just machine to make big decisions
Programmed by fellows (and gals) with compassion and vision
We'll be clean when their work is done
We'll be eternally free yes, and eternally young
Donald Fagen —> I.G.Y.
(Slightly modified for inclusiveness)
|
better way to get last named capture group
5 direct replies — Read more / Contribute
|
by ysth
on Jul 01, 2025 at 11:07
|
|
|
In python, I can abuse capture group names in a substitution to simplify the substitution code, like:
$ python -c 'import regex; print(regex.sub("^(?<a>.)|(?<c>.)$|(?<b>.)"
+, lambda m:m.lastgroup, "xxx"))'
abc
In perl, if there is a single variable equivalent to lastgroup, I can't find it, the closest I see is %+:
$ perl -E'say "xxx" =~ s/^(?<a>.)|(?<c>.)$|(?<b>.)/@{[keys %+]}/gr'
abc
which isn't awful, but is there a better way I'm missing?
A math joke: r = | |csc(θ)|+|sec(θ)| |-| |csc(θ)|-|sec(θ)| |
|
MySQL datetime
3 direct replies — Read more / Contribute
|
by joyfedl
on Jul 01, 2025 at 08:35
|
|
|
my $last_update = ('2025-06-30 14:40:26'); #convert to days without co
+nsidering time
if ($last_update < 20) { # if last update was done less than 20 days a
+go
print "20 days has not passed for you to update again"
}
|
binmode copy loses final byte
1 direct reply — Read more / Contribute
|
by aplonis
on Jun 30, 2025 at 14:46
|
|
|
Having an issue with code below. It's to be for a one-time pad encryption routine (just for fun).
Having an issue with binmode input and output. Losing a byte at the very end of output file. Not sure whether it happens on read or write. Someone please point out my error. TIA
#!user/bin/perl -w
my $infile = '/home/me/Documents/Sample.txt';
my $outfile = '/home/me/Documents/Sample_Copy.txt';
open(IN, "< $infile") or die "Can't open $infile: $!";
binmode IN;
open(OUT, "> $outfile") or die "Can't open $outfile: $!";
binmode OUT;
my $blksize = (stat IN)[11] || 16384;
print "\nBlock Size = $blksize\n";
my @bufAry = [];
while (my $len = sysread IN, my $buf, $blksize)
{ if (!defined $len)
{ next if $! =~ /^Interrupted/;
die "System read error: $!\n";
}
@bufAry = unpack('I*', $buf); # Disassemble buf into array.
print "\nIn = " . scalar @bufAry;
# Encryption subs here...
undef($buf); # Barrier to buf falling through.
# Decryption subs here...
$buf = pack('I*', @bufAry); # Reassemble buf from array.
print " Out = " . scalar @bufAry;
$offset = 0;
while ($len)
{ defined($written = syswrite OUT, $buf, $len, $offset)
or die "System write error: $!\n";
last if $written == 0; # Avoid infinite loop.
$len -= $written;
$offset += $written;
}
print "\n";
}
close(IN);
close(OUT);
|