Re^4: Using a Fetchrow with LWP

The error I am getting is 403 forbidden.

so the issue is not with the database related code. it is with the HTML::Tiny or LWP settings.

here is the output with use warnings on

Unquoted string "gzip" may clash with future reserved word at testdown
+.pl line 9.
Unquoted string "deflate" may clash with future reserved word at testd
+own.pl line 9.
Unquoted string "gzip" may clash with future reserved word at testdown
+.pl line 30.
Unquoted string "deflate" may clash with future reserved word at testd
+own.pl line 30.
Unquoted string "gzip" may clash with future reserved word at testdown
+.pl line 33.
Unquoted string "deflate" may clash with future reserved word at testd
+own.pl line 33.
Failure: 403, Forbidden
Failure: 403, Forbidden
Failure: 403, Forbidden
Failure: 403, Forbidden
Failure: 403, Forbidden
[download]

These is what Securities and Exchange Commission says about downloading from them

Fair access
Current max request rate: 10 requests/second.
To ensure everyone has equitable access to SEC EDGAR content, please u
+se efficient scripting. Download only what you need and please modera
+te requests to minimize server load.

SEC reserves the right to limit request rates to preserve fair access 
+for all users. See our Internet Security Policy for our current rate 
+request limit.

The SEC does not allow botnets or automated tools to crawl the site. A
+ny request that has been identified as part of a botnet or an automat
+ed tool outside of the acceptable policy will be managed to ensure fa
+ir access for all users.

Please declare your user agent in request headers:

Sample Declared Bot Request Headers:

User-Agent:

Sample Company Name AdminContact@<sample company domain>.com

Accept-Encoding:

gzip, deflate

Host:

www.sec.gov
[download]

Comment on Re^4: Using a Fetchrow with LWP Select or Download Code

Replies are listed 'Best First'.
Re^5: Using a Fetchrow with LWP by marto (Cardinal) on Jul 19, 2023 at 11:34 UTC
The line numbers you've posted errors at don't exist in the code you've posted. The sensible approach is to post a SSCCE when asking for help. `$mess = HTTP::Message->new(); $mess->encode(gzip,deflate); my $ua = LWP::UserAgent->new(timeout=>10); $mess = HTTP::Message->new();# Again $mess->encode(gzip,deflate); # ?` [download] Why are you doing this twice? Why not take steps to fix problems before posting code here? With strict/warnings: marto@Marto-Desktop:~/code/perlmonks$ perl -c junk.pl Global symbol "$mess" requires explicit package name (did you forget t +o declare "my $mess"?) at junk.pl line 6. Global symbol "$mess" requires explicit package name (did you forget t +o declare "my $mess"?) at junk.pl line 7. Global symbol "$mess" requires explicit package name (did you forget t +o declare "my $mess"?) at junk.pl line 9. Global symbol "$mess" requires explicit package name (did you forget t +o declare "my $mess"?) at junk.pl line 10. Global symbol "$mess" requires explicit package name (did you forget t +o declare "my $mess"?) at junk.pl line 11. Global symbol "$dbh" requires explicit package name (did you forget to + declare "my $dbh"?) at junk.pl line 14. Global symbol "$dbh" requires explicit package name (did you forget to + declare "my $dbh"?) at junk.pl line 14. Global symbol "$dbh" requires explicit package name (did you forget to + declare "my $dbh"?) at junk.pl line 15. Global symbol "@row" requires explicit package name (did you forget to + declare "my @row"?) at junk.pl line 18. Global symbol "@row" requires explicit package name (did you forget to + declare "my @row"?) at junk.pl line 18. junk.pl had compilation errors. [download] Update: also it's good form to mark significant updates to a post.	[reply] [d/l] [select]
Re^5: Using a Fetchrow with LWP by marto (Cardinal) on Jul 19, 2023 at 11:37 UTC
"These is what Securities and Exchange Commission says about downloading from them" I know.	[reply]
Re^6: Using a Fetchrow with LWP by justin423 (Scribe) on Jul 19, 2023 at 13:04 UTC
Wow.. Sorry about that. that worked for the download, but failed on the save. Can't locate object method "save_to" via package "Mojo::Message::Response" at testdown1.pl line 27. cpan> install Mojo::Message::Response Mojo::Message::Response is up to date (undef).	[reply]
Re^7: Using a Fetchrow with LWP by marto (Cardinal) on Jul 19, 2023 at 13:08 UTC
Your CPAN repo doesn't seem up to date https://metacpan.org/dist/Mojolicious/changes#L567: `8.02 2018-10-01 ... - Added save_to method to Mojo::Message.` [download] previously.	[reply] [d/l]
Re^8: Using a Fetchrow with LWP by justin423 (Scribe) on Jul 19, 2023 at 14:06 UTC
Re^9: Using a Fetchrow with LWP by marto (Cardinal) on Jul 19, 2023 at 14:38 UTC
Some notes below your chosen depth have not been shown here
Re^9: Using a Fetchrow with LWP by justin423 (Scribe) on Jul 19, 2023 at 14:08 UTC
Some notes below your chosen depth have not been shown here