Dear Monks,

I am trying to extract (in csv, txt or any human-readable format), the data from the table results in this website: https://desmace.com/provincia/asturias/

According to table headings, I want my script to ask for two dates for the top letf column "FECHA DEL TRAMITE" (prodedure date), and set two fixed dates (always 01/01/1900 and 31/12/2000) in the "FECHA MATRICULA" (plate date) column.

You can also add more columns to the table by clicking in "Columnas" at the topright. By doing this, I add "Prov. Matriculacion", and I also want my script to ask input for this field.

Then, according to this criteria, table results are displayed, and this is what I want to store in csv, txt or similar.

I have this code so far...it runs ok, but does not properly store the data.

I would be super grateful if I can get some help to make the script work. 

use strict;
use warnings;
use LWP::Simple;
use HTML::TreeBuilder;
use Text::CSV;
 
# URL of the website
my $url = 'https://desmace.com/provincia/asturias/';
 
# Filter criteria
my $fecha_tramite_min = '24/11/2024';
my $fecha_tramite_max = '27/11/2024';
my $fecha_matricula_min = '01/01/1900';
my $fecha_matricula_max = '31/12/2000';
my $prov_matriculacion_filtro = 'ASTURIAS';
 
# HTML page content
my $html = get($url) or die "No se pudo acceder a la URL: $!";
 
# Parse HTML
my $tree = HTML::TreeBuilder->new;
$tree->parse($html);
 
# Open csv file
my $csv = Text::CSV->new({ binary => 1, eol => "\n" });
open my $fh, ">", "resultados.csv" or die "No se pudo crear el archivo
+ CSV: $!";
 
# Headings for csv file
$csv->print($fh, ['FECHA DEL TRÁMITE', 'TRÁMITE', 'FECHA MATRÍCULA', '
+MARCA', 'MODELO', 'BASTIDOR (VIN)', 'PROV. MATRICULACIÓN']);
 
# This is the table that contains data
my @rows = $tree->look_down(_tag => 'tr');
 
foreach my $row (@rows) {
    my @columns = $row->look_down(_tag => 'td');
    my @data;
   
    # Extract values from columns
    foreach my $col (@columns) {
        push @data, $col->as_text;
    }
   
    # Row filtering
    if (@data >= 9) {
        my ($fecha_tramite, $fecha_matricula, $prov_matriculacion) = @
+data[0, 2, 7];
       
        if ($fecha_tramite ge $fecha_tramite_min && $fecha_tramite le 
+$fecha_tramite_max &&
            $fecha_matricula ge $fecha_matricula_min && $fecha_matricu
+la le $fecha_matricula_max &&
            $prov_matriculacion eq $prov_matriculacion_filtro) {
            $csv->print($fh, \@data);
        }
    }
}
 
# Close file
close $fh;
$tree->delete;
 
print "Datos filtrados guardados en 'resultados.csv'.\n";

[download]
Many thanks in advance!

In reply to Script to scrap data by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.