CodeJunkie has asked for the wisdom of the Perl Monks concerning the following question:

Hi,
I have a big problem... I've written a script that uses gzip to compress log files then transfers them every night using Net::FTP to another machine. The script is running on a UNIX server and putting the files on a windows server. These log files are quite large, we are talking about 10megs compressed per day. The script runs every night and just transfers the previous days files for a series of virtual hosts.

Anyway, my problem is that the logs seem to becoming corrupted somewhere and once they are on the destination server I cannot uncompress them. When using WinZip I get the message 'Invalid Compressed Data -- unable to inflate.'

My first thought was that I needed to transfer the files in binary mode and maybe the default was ascii mode FTP, can anyone confirm this? I tried changing the mode type using $ftp->quot("bin") or die "Failed to change to binary type:$!"; because I could not find any other way of changing file transfer type. Can anyone tell me the proper way of doing this?

All comments and ideas very welcome, I need to get this sorted ASAP because we have clients screaming for their web stats....

Cheers!
Tom

Replies are listed 'Best First'.
Re: Net::FTP Corrupting zip files
by robartes (Priest) on Mar 24, 2003 at 10:56 UTC
    It's simply $ftp->binary according to the docs: 
    type (TYPE [, ARGS])
           This method will send the TYPE command to the remote
           FTP server to change the type of data transfer. The
           return value is the previous value.

ascii ([ARGS]) binary([ARGS]) ebcdic([ARGS]) byte([ARGS])
           Synonyms for "type" with the first arguments set cor-
           rectly

    [download]

    CU
    Robartes-

[reply]
[d/l]
[select]
      cheers mate that's got it sorted! ;-) I'm pretty new to Perl and although I did look at the docs I guess I didn't understand them! Thanks.
[reply]
Re: Net::FTP Corrupting zip files
by jmcnamara (Monsignor) on Mar 24, 2003 at 11:00 UTC

    If you are transferring a zip file to a Windows machine the transfer mode should be binary to avoid any acsii/bin problems on the Windows side. Try this before using put(): 
        $ftp->binary();

    [download]

    --
    John.

[reply]
[d/l]
      Thanks a lot, you solved also my trouble... Leo
[reply]
Re: Net::FTP Corrupting zip files
by bronto (Priest) on Mar 24, 2003 at 12:12 UTC

    Just an humble advice about transferring logs.

    First, did you consider the option to replicate your logs to another one or a couple of machines via syslogd? Unless your network is heavily congested, that could be a good thing to do: if you send your logs to a couple of secure hosts, you still have the chance to read what happened to your machine even if an attacker goes in and destroyes the log files (BTW, you get a little information if he stops syslogd on the attacked host, but a little would be better than nothing...)

    Second, instead of transferring a whole log, you could preprocess it using swatch and send just the interesting parts of it; swatch happens to be a Perl program and is a very good tool for log monitoring and filtering.

    Ciao!
    --bronto

    The very nature of Perl to be like natural language--inconsistant and full of dwim and special cases--makes it impossible to know it all without simply memorizing the documentation (which is not complete or totally correct anyway).
    --John M. Dlugosz
[reply]
      Thanks i'll take a look at swatch, anything that can reduce the size of the log and keep the interesting bits would be very good!
[reply]

Back to Seekers of Perl Wisdom