I'm still not sure why my <br/> always got stripped away,
In the code you show above, ->find('address') is finding the <address> element, and then inside the ->map(sub { ... }), $_ is referring to that element, of which $_->text is getting only the text content, hence the missing <br/>. In the code I showed two nodes above, first I'm getting the <address> element into $addr, which preserves the document's structure, replacing the <br/>, and only then using ->text to get the text content.
I guess my main mistake was to assume, that it's necessary to start out from the $dom all over again, for each and every HTML element.
->find will use whatever node you call it on as the context, so it depends on what part of the document you want to search and where in the document the nodes you're looking for can occur.
The crazy idea I had was to somehow grab "Sample Street 123" into one variable (starting from the "address" element), and "45678 Randomcity" into another, by somehow targeting, and starting from, the first <br/> element after the "address" element.
It's possible, sure - in the Document Object Model, the <address> element has three children: a text node "Sample Street 123", the <br/> element, and another text node "45678 Randomcity" - you'll see this if you try looking at $addr->child_nodes.
But I think this goes back to what I was saying about example code being brittle if written based on too few examples, and writing lots of test cases: so far, you've only shown two snippets of data out of what you said are 10,000 *.html files. So for example, marto's code makes the assumption that the phone and fax will always be the 2nd and 4th <p>s, respectively, my code here makes the assumption that it's always the next node after the <p class="title"> that will contain the data (and that there are no double keys in the hash, and one or two other assumptions), my code here assumes that any element of class="address" contains only one <address> element that we're interested in, my code here assumes that the <p>s in elements of class="phone" are always in key+value pairs, and so on.
My suggestions would be for you to first survey your input files, and see how much variation there is, so that you can boil it down to a representative set of test cases, and to code defensively, i.e. testing all of the assumptions I named above. Here's what that could look like:
use warnings; use strict; use Mojo::DOM; use Mojo::Util qw/trim/; # this sub should really be in its own package for modularity sub get_data { my $html = shift; my %data; my $dom = Mojo::DOM->new($html); my $addr = $dom->find('.address address'); # could add some conditionals here # in case there are separate fields for street / city / zip etc. die "Didn't find exactly one address" unless @$addr==1; $addr = $addr->first; $addr->find('br')->map('replace',"\n"); $data{address} = { Address => trim( $addr->text ) }; my $phone = $dom->find('.phone p'); die "Didn't find an even number of elements in phone" if @$phone%2; while (@$phone) { my $key = trim( shift(@$phone)->text ); die "Duplicate key '$key' in phone data" if exists $data{phone}{$key}; $data{phone}{$key} = trim( shift(@$phone)->text ); } return \%data; } use Test::More; is_deeply get_data(<<'HTML'), <div class="address"> <div class="icon"></div> <address> Sample Street 123<br/>45678 Randomcity </address> </div> <div class="phone"> <div class="icon"></div> <p class="title">Telephone</p> <p>0123-4 56 78 90 <p class="title">Telefax</p> <p> </div> HTML { address => { Address => "Sample Street 123\n45678 Randomcity" }, phone => { Telephone => "0123-4 56 78 90", Telefax => "" }, }; # TODO: many more test cases here done_testing;
In reply to Re^5: How to parse not closed HTML tags that don't have any attributes?
by haukex
in thread How to parse not closed HTML tags that don't have any attributes?
by Rantanplan
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |