in reply to Re^2: Match text from txt to html
in thread Match text from txt to html

Just as a text file is both a set of lines and a stream of bytes, an HTML document is both a tree and a stream of elements. HTML::Parser extracts the latter, which is equivalent to walking the DOM tree in some order. The advantage of using HTML::Parser for an application like this is the same as the advantage of processing a text file line-by-line without reading the whole file into memory.

While it is unlikely that an HTML document would not fit into memory on a client, our questioner could be building something that runs on a server, with an instance of the program for each concurrent client connection which can quickly become very large in aggregate if many clients are active. In this case, building the entire tree in memory is unnecessary because the transformation to be applied is very simple: find and mark ocurrances of certain text in a finite sliding window. If this is running on a server, building the DOM tree in memory is both wasteful and foolish, creating an opportunity for easy DoS attacks.

Put simply, if you do not actually need the DOM tree, do not waste time and memory building it!

Replies are listed 'Best First'.
Re^4: Match text from txt to html
by Anonymous Monk on Sep 06, 2019 at 04:04 UTC
    Ever used XML::Twig or XML::LibXML? Ever heard of them? They both give you all the DOM goodness in steaming mode, perlmonks is full of examples
[reply]

      HTML is not XML, and you cannot parse HTML with an XML parser.

      You might also want to make an account, so you can edit your posts and fix your typos, like "steaming mode" in the post above.

[reply]
[reply]

        HTML is not XML, and you cannot parse HTML with an XML parser.

        Hi

        Hmm, lets see, html libxml, [google://site:perlmonks.org html libxml]

        ..scanning... 2003 HTML tidy, using XML::LibXML

        second check, html twig ... 2004 XML::Twig and HTML Entities

        I'm sure a check of the previously linked docs would have revealed the same , xml parsers can read html

        Even when I'm confident in my memory, I always check to make sure

        You might also want to make an account, so you can edit your posts and fix your typos, like "steaming mode" in the post above.

        Thanks , I already have account

        xmltwig.org is all about "streaming mode" as a concept (dont load whole document into memory)

        XML::LibXML also supports it -- I checked before I posted

        Also both documentations mention "stream"

        A person can't know/remember everything, thats why we have perlmonks and search

        When Duty Calls, real pedants check the fact not just their memories

[reply]
[d/l]

In Section Seekers of Perl Wisdom