Re: Getting/handling big files w/ perl

HTTP::Range (OK, unmaintained for 10 years) as well as LWP::Parallel::UserAgent might be worth to take a look at.

Update: I just reread the curl manpage.

Curl has a --range option:

host:~ # curl --range 0-79 -o out.html perlmonks.org
  % Total    % Received % Xferd  Average Speed   Time    Time     Time
+  Current
                                 Dload  Upload   Total   Spent    Left
+  Speed
  0    80    0    80    0     0    368      0 --:--:-- --:--:-- --:--:
+--     0
host:~ # wc -c out.html
80 out.html
host:~ # cat out.html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
[download]

See also WWW::Curl

Regards, Karl

ŤThe Crux of the Biscuit is the Apostropheť

Comment on Re: Getting/handling big files w/ perl Select or Download Code

Replies are listed 'Best First'.
Re^2: Getting/handling big files w/ perl by Gisel (Novice) on Nov 17, 2014 at 07:00 UTC
Karl— Actually I have been looking pretty hard at LWP::Parallel::UserAgent. It might be the tool I am looking for, but I am just not clear yet on how I would fit my problem into it. Also, this whole "information transmission" issue is not one I really have a firm footing on. The spin-up is taking me a while... Thanx for your input.	[reply]