Re: Using Perl to snip the end off of HTML

OK... referencing your Reply, coupla more questions:

Are you trying to scrape this or are you working with a saved_local_file? (if so, suspect you can swap some extra processing for much better results)
Either it's waaay to late to comment (cuz brain went to sleep before body) or there is some very odd usage in the Boland blockquote (near "Libraries> which might be the close of a quoted link, except that I can't find a start). So question is: Can you count on well_formed html? Mere fragments (like your sample data)?
added> This is NOT a complete .html page. If, in fact, you have such a page, you could use the </body> tag as a marker to determine which (possibly nested) blockquotes are at the end of the page.
added> If all the .html fragments you want to clean up come from the same origin (the web4lib listserv), you can clean up the end of the file by starting near the end. As it turns out (depending on your example) there's a very easy place to truncate your file... just before the very first instance of
_______________________________________________
Web4lib mailing list

where there are 47 underlines, a newline, and the web4lib... credit.

added> In fact, the more I puzzle over this, the more I suspect that despite your certainty about your intent (as indicated by your caution to 'hold all "Why would you want to do that?" responses.' may reflect inadequate analysis of what you need to do to achieve your (still vague) objective.

added> Others, more skilled, may approach this differently, but IMO, writing complex regexen is only about 10% knowing syntax; the rest is analysing your dataset so that the logic of the regex is plain. So, consider writing (or at least, 'thinking through') a detailed example of what output you want from the processing and compare that -- in detail -- to the source, to get a clear view of what you need to do.

And again: This clearly is a case where you'll ascend unto the heavens more quickly and surely on the backs of those who've written "gold standard" modules as HTML::Parser, etc. than by re-inventing the wheel.

By way of confession; writing the regex you're seeking is not quite as simple as I suggested above for your source example,
... but ...

update Last 2 bullets and the 2 following paragraphs added (2005 Jun 8 2100 GMT), after 'puzzling' for a bit.

Comment on Re: Using Perl to snip the end off of HTML

Replies are listed 'Best First'.
Re^2: Using Perl to snip the end off of HTML by eastcoastcoder (Sexton) on Jun 08, 2005 at 23:09 UTC
It seems like everyone would like some background. I thought it was orthagonal to my objective and so ommitted it, but, apparently it's not, so here it is. I'm working on adding code to an (already working) mailing list converter. The converter converts emails to html, suitable for use in archives. Some features that I would like to add are: to get rid of whitespace at the end (easy), HTML whitepsace (medium), and trailing blockquotes (quite hard). The email is already stored locally in a file, and is converted to a variable in memory. By end of the file, I simply mean the last byte (character for those of us who speak Unicode) of the variable. The `</body>` etc tags will be added on later. The emails will be coming from a variety of sources and lists. The one I posted above is simply an example, but I can't count on its specifics. The html shoul be well formed, with the understanding that the `<body>`and`</body>` tags will be left out. There should be no tables or anything - it should really be more or less straight text, with simple markup - as you would find in an email sent as text/html. Again, by "end" I only mean the end of the variable. I hope I've provided everything that you've asked - if not, please let me know. And I hope someone here has some ideas, since I'm stumped, specifically on the last one (removing (possibly nested blockquotes)). PS Yes I realize that regex should never be used for real html, they can get confused by html in comments, and whitespace or attributes in the middle of a tag, etc - I was only trying to give a simple example of why they were totally inadequate here. The code, as is, can already convert	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^2: Using Perl to snip the end off of HTML
by eastcoastcoder (Sexton) on Jun 08, 2005 at 23:09 UTC

I'm working on adding code to an (already working) mailing list converter. The converter converts emails to html, suitable for use in archives. Some features that I would like to add are: to get rid of whitespace at the end (easy), HTML whitepsace (medium), and trailing blockquotes (quite hard).

The email is already stored locally in a file, and is converted to a variable in memory. By end of the file, I simply mean the last byte (character for those of us who speak Unicode) of the variable. The </body> etc tags will be added on later.

The emails will be coming from a variety of sources and lists. The one I posted above is simply an example, but I can't count on its specifics.

The html shoul be well formed, with the understanding that the <body>and</body> tags will be left out. There should be no tables or anything - it should really be more or less straight text, with simple markup - as you would find in an email sent as text/html. Again, by "end" I only mean the end of the variable.

I hope I've provided everything that you've asked - if not, please let me know.
And I hope someone here has some ideas, since I'm stumped, specifically on the last one (removing (possibly nested blockquotes)).

PS Yes I realize that regex should never be used for real html, they can get confused by html in comments, and whitespace or attributes in the middle of a tag, etc - I was only trying to give a simple example of why they were totally inadequate here. The code, as is, can already convert

[reply]
[d/l]
[select]