Re^10: HTTP response: 400 Bad Request

never had any idea downloading plain text would be so difficult..

This is what finally worked....

use LWP::UserAgent ();
use parent 'HTTP::Message';
$mess = HTTP::Message->new();
$mess->encode(gzip,deflate);
$filename='/temp/edgar/workfile.txt';
$url='https://www.sec.gov/Archives/edgar/daily-index/2023/QTR3/form.20
+230712.idx';
my $ua = LWP::UserAgent->new(timeout => 10);
$ua->default_header('Accept-Encoding' =>$mess = HTTP::Message->new());
$ua->default_header( USER_AGENT =>'COMPANY youremail@email.com' );
my $res = $ua->mirror( $url, $filename );
[download]

Comment on Re^10: HTTP response: 400 Bad Request Download Code

Replies are listed 'Best First'.
Re^11: HTTP response: 400 Bad Request by marto (Cardinal) on Jul 16, 2023 at 11:20 UTC
How about: `#!/usr/bin/perl use strict; use warnings; use feature 'say'; use Mojo::UserAgent; my $url = 'https://www.sec.gov/Archives/edgar/daily-index/2023/QTR3/fo +rm.20230712.idx'; my $ua = Mojo::UserAgent->new; $ua->transactor->name('COMPANY youremail@email.com'); say $ua->get( $url )->res->body; # or $ua->get( $url )->res->save_to('derp.txt');` [download]	[reply] [d/l]

Replies are listed 'Best First'.

Re^11: HTTP response: 400 Bad Request
by marto (Cardinal) on Jul 16, 2023 at 11:20 UTC

How about:

#!/usr/bin/perl
use strict;
use warnings;
use feature 'say';
use Mojo::UserAgent;

my $url = 'https://www.sec.gov/Archives/edgar/daily-index/2023/QTR3/fo
+rm.20230712.idx';
my $ua  = Mojo::UserAgent->new;
$ua->transactor->name('COMPANY youremail@email.com');
say $ua->get( $url )->res->body;
# or
$ua->get( $url )->res->save_to('derp.txt');
[download]

[reply]
[d/l]