Re^6: Correct Perl settings for sending zipfile to browser

Beautiful! The following code is now in place and, having tested it with both Windows and Mac, it is working exactly as desired. Thank you so much for taking time to help with this. I have learned a few things.

None of the fields should ever contain tabs, so that should be no issue. I always use strict, but only use warnings while troubleshooting. It's presently commented out to save space in the log files (when working with UTF8, wide character warnings seem to appear for no reason at times, when everything is working properly, and sometimes I get the uninitialized errors that are also extraneous--and the logs simply aren't monitored often enough to make filling them with such errors useful). For what it's worth, when the script with this subroutine is run, the script is adding a "wide character" error to the error log even with warnings turned off, despite the fact that everything is working beautifully, and the zip file that is created contains perfect text in an Asian language. The error comes from having Asian text in one of the HTML headings for the page, even though the "use utf8;" pragma is in place. To avoid it I'd probably need to convert that text to HTML-entities which makes it totally unreadable in my code, and ugly. I'd rather get useless warnings.

sub exportdatabase { 
#INCOMING GLOBAL VARS: $statement, $db_export_file,
# $cur_date, $cur_time, $OS, $table, $quest

fork: {

my ($recnum,$revnum,$book,$chap,$verse,$text) = '';
my @resp     = ();
my $encoding     = "UTF-8";
my $zipdata     = '';
my $CRLF     = "\n";
my $zipfile     = "$db_export_file.zip";
my $q = CGI::Simple->new();
my $z = IO::Compress::Zip->new(\$zipdata,
    Name => "$db_export_file" )
    or die "zip failed: $ZipError\n";

    if ($OS eq "Windows") { 
        $CRLF = "\r\n";
    } 

    $statement = qq| SELECT a.RecordNum, a.RevisionNum, a.Book, a.Chap
+ter, a.Verse, a.Text from $table a INNER JOIN (SELECT RecordNum, max(
+RevisionNum) RevisionNum FROM $table GROUP BY RecordNum) b USING (Rec
+ordNum,RevisionNum); |;

    &connectdb('exportdatabase');

    $z->print( encode( $encoding, "RECORD#\tREVISION#\tBOOK#\tCHAP#\tV
+ERSE#\tTEXT, AS EDITED BY: $curdate $curtime (Pacific Time), EXPORTED
+ IN $OS FORMAT$CRLF", Encode::FB_CROAK|Encode::LEAVE_SRC ) );
    while ( ( $recnum, $revnum, $book, $chap, $verse, $text ) = $quest
+->fetchrow_array() ) {
        $z->print( encode( $encoding, "$recnum\t$revnum\t$book\t$chap\
+t$verse\t$text$CRLF", Encode::FB_CROAK|Encode::LEAVE_SRC ) );
    }
    $z->close();

    print $q->header( -type => 'application/zip',
    -Content_Disposition => qq{attachment; filename="$zipfile"},
    -Content_Length => length($zipdata) );

    binmode STDOUT;  # just to play it safe
    print $zipdata;

} # END fork
} # END SUB exportdatabase
[download]

Comment on Re^6: Correct Perl settings for sending zipfile to browser Download Code

Replies are listed 'Best First'.
Re^7: Correct Perl settings for sending zipfile to browser by haukex (Archbishop) on Nov 17, 2019 at 21:28 UTC
Glad to help! when working with UTF8, wide character warnings seem to appear for no reason at times, when everything is working properly, and sometimes I get the uninitialized errors that are also extraneous ... I'd rather get useless warnings. While I agree that when building strings for output, the "uninitialized" warnings can often be turned off (`no warnings 'uninitialized';`, limited to the scope where it's needed), the UTF-8 / wide character warnings are actually useful and are pointing to the fact that something is wrong, even if it may appear to be "working beautifully". For example, at my console, I can do this: $ perl -le 'print "\N{U+263A}"' Wide character in print at -e line 1. ☺ Since the smiley was printed correctly, it appears to be working. But the fact that it worked is only because my terminal's encoding happened to match the encoding Perl used. So if you're getting these warnings, likely you've still got an encoding issue somewhere, and it might come back to bite you later. It would probably be helpful if you can reduce the issue down to an SSCCE, then you could post it as a new question here to get help with it. even though the "use utf8;" pragma is in place The utf8 pragma only tells Perl that the source code is encoded in UTF-8. It doesn't change the encoding of `STDIN`, `STDOUT`, etc. - for that, you could use the open pragma, e.g. `use open qw/:std :utf8/;`.	[reply] [d/l] [select]
Re^8: Correct Perl settings for sending zipfile to browser by Anonymous Monk on Nov 17, 2019 at 22:01 UTC
Adding the "use open qw/:std :utf8/;" pragma did not remove the useless warning. It shouldn't, if I understand things correctly. That is because the source of the "wide characters" is embedded in the code itself, and is not brought in from an external file which must be opened. I am unaware of a way to encode the source code itself, or to decode it, save for the "use utf8;" pragma within the code itself. I have tested the code both with and without those characters in that HTML header, and removing them is the only way to erase the error message. I'm not willing to do that, because I want that heading. I suppose I should really just add a "no warnings;" to the code, the opposite of "use warnings;". That would silence the false alarm. To be honest, in this day and age, having UTF8 characters in the code should be a complete non-issue. The "use utf8;" pragma should be unnecessary. It should be the built-in default. Ditto with the open pragma. It may be worth noting that I have already required UTF8 for the HTML form, and have specified it in the output, i.e.: `<HTML lang="utf8"> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf8">` [download]	[reply] [d/l]
Re^9: Correct Perl settings for sending zipfile to browser by haukex (Archbishop) on Nov 17, 2019 at 22:23 UTC
That is because the source of the "wide characters" is embedded in the code itself, and is not brought in from an external file which must be opened. The source of the warnings is most likely that some string in your Perl program contains Unicode characters that you are trying to output through a stream that is not set to the proper encoding. Those characters can get there in a number of different ways: a file, a value submitted on an HTML form, from the database, or a Perl string such as the one I showed above - no `use utf8` necessary. That would silence the false alarm. It's not a false alarm - something really is wrong. It may seem to work for now, but it's possible that if a user with a browser different from yours visits your site, things may break in mysterious ways. Adding the "use open qw/:std :utf8/;" pragma did not remove the useless warning. That could mean that it's some other output stream, or possibly that some module reset `STDOUT` back to the default, and you might have to do an e.g. `binmode STDOUT, ':raw:encoding(UTF-8)';` at runtime. I have tested the code both with and without those characters in that HTML header, and removing them is the only way to erase the error message. As I said, a SSCCE would be helpful in demonstrating that to us, if you'd like help with that. The "use utf8;" pragma should be unnecessary. It should be the built-in default. Yes, that's certainly a very valid argument, and one that others have raised. Perl has a long history of backwards-compatibility though, so (unfortunately) it's unlikely to be changed. It may be worth noting that I have already required UTF8 for the HTML form, and have specified it in the output Yes, that's good (except `utf8` is not a valid `lang`, it should probably be `en` or whatever language your page is in). You could also specify it in the HTTP headers (e.g. by supplying `-charset=>'UTF-8'` to CGI::Simple's `header` method).	[reply] [d/l] [select]
Re^7: Correct Perl settings for sending zipfile to browser by Anonymous Monk on Nov 18, 2019 at 08:18 UTC
`> my ($recnum,$revnum,$book,$chap,$verse,$text) = '';` That only initializes the first var, try this instead: `my ($recnum,$revnum,$book,$chap,$verse,$text) = ('') x 6;` TIMTOWTDI `my $CRLF = $OS eq 'Windows' ? "\r\n" : "\n"; use Socket ':crlf'; my $CRLF = $OS eq 'Windows' ? CRLF : LF;` [download]	[reply] [d/l]