Re^2: problems with decoding by using base64 method

Thanks for your reply.

Actually we have legacy PERL application which encodes(using base64) the file content and puts data into the DB.

Now we have written a script to download & decode those attachment from that DB.

Below is the code we are using for this:

#############################

#!/usr/brcm/ba/bin/perl

use strict;
use warnings;
#****************************************************************
#               MODULE(S) USED
#****************************************************************
use FindBin;
use lib "$FindBin::Bin/lib";
use Database;
use Configs_DB;
#use MIME::Base64;
use MIME::Base64 qw( decode_base64 );
use File::Basename;
use File::Copy;
our ($host, $database,$user, $password);

#****************************************************************
#         Taking input CSV file for the script from user 
#****************************************************************
my $TicketFile     = $ARGV[0]; #E.g. -> "APP_Attachments/Jan2016 APP t
+icket Numbers.csv";
if (!defined $TicketFile)
{

    print "Please provide the correct path for the CSV file to process
+! \n";
    exit 1;
}

#****************************************************************
#       Creting Directory for Storing attachments
#****************************************************************
my $tktFolder    = "APP_Attachments";
if (!-d $tktFolder)
{
    mkdir $tktFolder;
}
#****************************************************************
#               VARIABLES DECLARATION
#****************************************************************

my @CSVcontent     = ();
my @Tickets     = ();
#my @foldersAtt    = ();
#my @dates        = ();
my $dbh         = Database::Connect( Database::DSN( $host, $database )
+,$user, $password );
my $destFolder    = "";
#****************************************************************
#      "uniq" Function for selecting unique values in an array
#****************************************************************
#sub uniq {
#    my %seen;
#    return grep { !$seen{$_}++ } @_;
#}

#****************************************************************
#                  Copy the CSV File into Array
#****************************************************************
open ( IFILE, "< $TicketFile" ) || die "$TicketFile File not found";
@CSVcontent = <IFILE>;
close IFILE;
shift( @CSVcontent );


#****************************************************************
#   Creating array containing Dates & HD Ticket nos.
#****************************************************************
foreach my $line ( @CSVcontent )
{
    chomp($line);
    my @arr=split(",", $line);
    push ( @Tickets, $arr[0] );
    #push( @dates, $arr[1] );
    my @partDate = split("/", $arr[1]);
    $destFolder = $partDate[2];
    if ( !-d "$tktFolder/$partDate[2]" )
    {
        mkdir "$tktFolder/$partDate[2]";
    }
    #foreach my $val ( @arr )
    #{
        #chomp( $val );
        
        #if ( $val =~ /^HD[0-9]{13}$/ )
        #{
        #    push ( @Tickets, $val );
        #}
        #if ( $val =~ /\d{1,2}\/\d{1,2}\/\d{2}/ )
        #{    
            
            #my $data=substr( $val, 0, index($val, ' ') );
            
        #    push( @dates, $val );
        #}
    #}
}
#my @folderDates = uniq( @dates );
#$destFolder = grep {$_ =~ /20*/}
#****************************************************************
#              Creating Year wise Directories
#****************************************************************

#foreach my $dateT ( @folderDates )
#{
#    chomp($dateT);
    
#    my @partDate = split("/", $dateT);
#    my $partYear = $partDate[2];
    
#    if ( !-d "$tktFolder/$partYear" )
#    {
#        mkdir "$tktFolder/$partYear";
#    }
    
#}

#opendir(my $dh, $tktFolder) || die "Can't opendir $tktFolder: $!";
#@foldersAtt = grep { -d "$tktFolder/$_" and $_ ne '.' and $_ ne '..' 
+} readdir($dh);
#closedir $dh;

print "Script is running...Please Wait, it may take 10-15 mins to comp
+lete! \n";
#****************************************************************
#              Copying attachments from DB & Filer
#****************************************************************
foreach my $tkt ( @Tickets )
{
    chomp( $tkt );
    my $sql = "SELECT RequestID,TicketNumber
            FROM Request 
            WHERE TicketNumber = '$tkt'";
    my $sth = $dbh->prepare($sql) or die  "cant execute -- $DBI::errst
+r";
    $sth->execute();
    my @RequestID = $sth->fetchrow_array;
    
    my ($id, $tNum) = @RequestID;
    chomp($id);
    
    my $sql2 = "SELECT FileName,Filecontents,RequestId,ItemID,COUNT(Fi
+leName)
                OVER (PARTITION BY FileName) TOT_CNT 
                FROM Attachments 
                WHERE RequestId = '$id'";
    my $sth2 = $dbh->prepare($sql2) or die  "cant execute -- $DBI::err
+str";
    $sth2->execute();
    my $dbAttachments = $sth2->fetchall_arrayref();
    $sth2->finish;
    
    my $sizeAttArray = @$dbAttachments;
    
    if ( @$dbAttachments ) #if attachments are present in DB for the t
+icket
    {
        foreach my $row ( @$dbAttachments ) 
        {
            my ($fileName, $fileContent, $reqID, $itmID, $count) = @$r
+ow;
            chomp( $fileName );
            $fileName =~ s/\\/\//g;
            my $fName = basename( $fileName );
                my $filePath1 = "/tools/busapps/LAMP/dev/APP/htdocs/AP
+P/APP_Attachments/2016/LGop.pdf"; 
               open (my $upload_filehandle,'<',$filePath1);
                 binmode $upload_filehandle;
                 open ( OFILE1, '>', "$filePath1" );
                 print OFILE1 $fileContent;
                 close OFILE1;
                 my $filecontents;
                 {
                    local $/ = undef; $filecontents = <$upload_filehan
+dle>;
                 } 
                my $fContent = decode_base64( $filecontents );
            #my $fContent = decode_base64( $fileContent );
            #foreach my $dir(@foldersAtt)
            #{                
                #chomp( $dir );
                chomp( $tNum );
                #if( index($reqDt,$dir) != -1 )
                #{       
                    if ( $count == 1 )
                    {
                        my $filePath = "$tktFolder/$destFolder/".$tNum
+."_".$fName;
                        open ( OFILE, '>', "$filePath" );
                        print OFILE $fContent;
                        close OFILE;
                    }
                    else #($count > 1)
                    {                        
                        my $filePath = "$tktFolder/$destFolder/".$tNum
+."_".$itmID."_".$fName;
                        open ( OFILE, '>', "$filePath" );
                        print OFILE $fContent;
                        close OFILE;                            
                    }
                #}
            #} 
            
        }
    }
    Database::Disconnect($dbh);
}
[download]

Comment on Re^2: problems with decoding by using base64 method Download Code

Replies are listed 'Best First'.
Re^3: problems with decoding by using base64 method by derby (Abbot) on Aug 03, 2016 at 12:12 UTC
That was tough to wade through, you should put code in-between <code>...</code> tags. But basically, I see this `my $filePath1 = "/tools/busapps/LAMP/dev/APP/htdocs/APP/APP_Attachment +s/2016/LGop.pdf"; open (my $upload_filehandle,'<',$filePath1); binmode $upload_filehandle; my $filecontents; { local $/ = undef; $filecontents = <$upload_filehandle>; } my $fContent = decode_base64( $filecontents );` [download] and unless your file naming conventions are strange, a pdf file is not base64 encoded. What you need to do is first identify and isolate a file that is failing to decode. Move it off to the side and then write a very small script that just opens that problem file and tries to decode_base64 it. More than likely, you'll find the problem is with the actual file (it's not base64 encoded) and not the decode_base64 routine. -derby	[reply] [d/l]
Re^4: problems with decoding by using base64 method by anil1.perl (Initiate) on Aug 03, 2016 at 12:55 UTC
Hi Derby, Thanks for your reply. We have created a script & doing what you said as below 1.uploading the encoded file to DB 2.Downloading the file & decoding it. Files which have images/scanned images/headers & footers are not opening but the other files (text) are opened. Any thoughts on this? Regards, Anil	[reply]
Re^5: problems with decoding by using base64 method by CountZero (Bishop) on Aug 03, 2016 at 19:10 UTC
Did you try to decode an encoded file before it was put in the database? If that works, then your database somehow alters the file between storage and retrieval. If that doesn't work then there is probably something wrong with the encoding. CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James My blog: Imperial Deltronics	[reply]
Re^6: problems with decoding by using base64 method by anil1.perl (Initiate) on Aug 04, 2016 at 10:28 UTC
Re^7: problems with decoding by using base64 method by Anonymous Monk on Aug 04, 2016 at 10:53 UTC
Some notes below your chosen depth have not been shown here
Re^3: problems with decoding by using base64 method by coicles (Sexton) on Aug 07, 2016 at 00:02 UTC
If this is running under Windows, extra bytes are probably being added when you write your decoded binary data from the `$fContent` variable to `OFILE`. If this is the case, you could fix it by adding the line `binmode OFILE;` immediately after lines which begin with `open ( OFILE,`...	[reply] [d/l] [select]