Re^5: string pattern match, limited to first 1000 characters?

So your are just checking whether there's a <html> tag in some data stream scraped off a webserver? That explains why the regexp is fastest - normally opening html tags are pretty much at the beginning of a HTML page.

Dunno whether partial gunzipping is possible with a perl module, but you can always open a pipe to/from gunzip and kill the process off when you have 1000 bytes read.

May I ask what nefarious purpose you need all that for?

--shmem

_($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                              /\_¯/(q    /
----------------------------  \__(m.====·.(_("always off the crowd"))."·
");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}

Comment on Re^5: string pattern match, limited to first 1000 characters? Download Code

Replies are listed 'Best First'.
Re^6: string pattern match, limited to first 1000 characters? by ManFromNeptune (Scribe) on Jun 25, 2007 at 17:32 UTC
Ok, a few hours of trying various CPAN modules reveals the following working code: use IO::Uncompress::AnyUncompress qw(anyuncompress) my $input = `cat testfile.gz`; my $output; anyuncompress $input => \$output, { 'InputLength' => 1000 }; print $output; [download] I've confirmed that this works by benchmarking with vs. without the InputLength option ... using it yields about a 4x speedup on files of about 50KB (gzipped), since it's only uncompressing the leading 1000 characters. Fyi, this is for a local web cache analysis program -- I need to periodically clean out certain files while leaving others. One of the conditions is whether the file is HTML or not, and you'd think you could rely on Content-Type headers, but as it turns out some web sites mis-type javascript and css content as text/html. Thus, I really need to inspect the file to be absolutely sure of the content type. Anyways, problem is essentially solved at this point. Thanks to all for the help and suggestions! cheers MFN	[reply] [d/l]

Replies are listed 'Best First'.

Re^6: string pattern match, limited to first 1000 characters?
by ManFromNeptune (Scribe) on Jun 25, 2007 at 17:32 UTC

use IO::Uncompress::AnyUncompress qw(anyuncompress)
my $input = `cat testfile.gz`;
my $output;
anyuncompress $input => \$output, { 'InputLength' => 1000 };
print $output;
[download]

[reply]
[d/l]