Re: Exporting Curl content to html

UPDATE: This may not be a coding issue. I spoke to the original author and he ran it 'as is' on his console and the expected results.

Perl version: perl 5, version 20, subversion 2 (v5.20.2) built for MSWin32-x64-multi-t

Here's what I downloaded to get perl to run on Win7

1. install perl http://www.activestate.com/activeperl?gclid=CP6J0LzNx8oCFQovHwod280O_A

2. install c== redistrubtable https://www.microsoft.com/en-us/download/details.aspx?id=48145

3. Install curl http://curl.haxx.se/download.html

So am I running it in the right environment?

Comment on Re: Exporting Curl content to html

Replies are listed 'Best First'.
Re^2: Exporting Curl content to html by poj (Abbot) on Jan 26, 2016 at 20:47 UTC
Did the author run it on Unix ?. I'm guessing the line `$json =~ /^.+\r\n\r\n:?(.+)$/s;` is removing the header but on windows should be `\n\n`. However, if you remove the `-i` the curl download doesn't include the header. Using `-o` you can write the result to a file. Try this revised script (untested). #!perl use strict; my $base_url= 'https://'; my $user = ''; my $pass = ''; my $out_dir = 'pages'; my $format = 'html'; my $out= `curl -u $user:$pass -i $base_url/file/root/tree?format=ids`; if ($out =~ /.+\s*([\d,]+)$/){ my @ids = split /,/, $1; foreach my $id (@ids) { print "$id\n"; my $cmd = "curl -u $user:$pass $base_url/files/$id/contents?format +=$format -o $out_dir/$id.$format"; my $status = system($cmd); if ($status) { die "system error: $?" } } } else { print "Error - No ids found"; } [download] poj	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^2: Exporting Curl content to html
by poj (Abbot) on Jan 26, 2016 at 20:47 UTC

Did the author run it on Unix ?. I'm guessing the line $json =~ /^.+\r\n\r\n:?(.+)$/s; is removing the header but on windows should be \n\n. However, if you remove the -i the curl download doesn't include the header. Using -o you can write the result to a file.

Try this revised script (untested).

#!perl
use strict;
my $base_url= 'https://';
my $user    = '';
my $pass    =  '';
my $out_dir = 'pages';
my $format  = 'html';

my $out= `curl -u $user:$pass -i $base_url/file/root/tree?format=ids`;

if ($out =~ /.+\s*([\d,]+)$/){
  my @ids = split /,/, $1;
  foreach my $id (@ids) {
    print "$id\n";
    my $cmd = "curl -u $user:$pass $base_url/files/$id/contents?format
+=$format -o $out_dir/$id.$format";
    my $status = system($cmd);
    if ($status) {
      die "system error: $?"
    }
  }
} else {
  print "Error - No ids found";
}
[download]

[reply]
[d/l]
[select]