ash1351 has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I have this front end CSS data stored in a web page. Here is the data in csv file.

secid,Initial_shares 002826,3777 0028262,3777 0028262,3777 0028262,3777 0028262,3777 0028262,3777
I need to convert this text file into the below format. once, i convert it to below format, i will be able to display it on the front end. this format below is used to display data in jqgrid.
var secid = [ "002826", "0028262", "0028262", "0028262", "0028262", +"0028262"]; var Initial_shares = [ "3777", "3777", "3777", "3777", "3777", "3777"1 ];
In order to convert the text to above format, i have used the below perl code. Please note. This conversion is static. i.e. it knows how many columns are there. In this case, there will be 2 columns. secid and Initial_shares. so here is the static code.
my @a=(); my @b=(); my @a1=(); my @b1=(); my @perl_array=(); my @filena = split(/\|/, $filename); open (TXT, "<$filename") || die "Can't open $filename: $!\n"; while (my $line=<TXT>) { chomp($line); ($a2, $b2) = split /,/, $line; push @a,('"',"$a2",'"',','); push @b,('"',"$b2",'"',','); } splice @a,0,4; #this splice is used to remove the header name. i.e. +first row data- secid, Initial_shares splice @b,0,4; push @a1,"var secid=[@a]"; push @b1,"var Initial_shares=[@b]"; push @perl_array, "@a1; @b1"; close TXT;
The @perl_array will be then exactly similar to the kind of data we were expecting at the start. i wil transfer this perl variable to front end for displaying then. I need help in the following case. What if instead of 2 columns, there are 5 columns. How can we convert the same csv file to the format mentioned earlier. it should be all dynamic. Can someone shed some light please.

Replies are listed 'Best First'.
Re: Get CSV data and convert it to other format data dynamically.
by VinsWorldcom (Prior) on Aug 24, 2015 at 21:02 UTC

    Have a look at Array::Transpose for the column to row transposition once you've read in with Text::CSV (as mentioned above).

      Hi VinsWorldcom, The text file has csv data itself. I didnt get why we need to use Text:CSV first and then proceed. We can straight away use Array::Transpose ??

        Text::CSV offers a more robust interface than:

        my @csv; while (<$fh>) { chomp $_; my @p = split /,/, $_; push @csv, \@p; }

        or something like that. If your goal is to be robust, I'd use the module that is solely dedicated to reading that type of data and then another that helps you transform it. Not sure about the JSON output, you're getting a bit out of my league there, but again, if you're looking for robust, readable, supportable code, use the module that's been vetted by all CPAN'ers versus rolling your own.

Re: Get CSV data and convert it to other format data dynamically.
by stevieb (Canon) on Aug 24, 2015 at 21:55 UTC

    When you cross-post at many sites, it's polite to advise you've done so.

    This saves people from doing work in forming an answer when the question may have already been answered somewhere else.

Re: Get CSV data and convert it to other format data dynamically.
by kevbot (Vicar) on Aug 25, 2015 at 04:45 UTC

    Hello ash1351,

    Here is one way to do it with the help of Data::Table and Cpanel::JSON::XS. There are a few different choices for modules to parse csv files and to emit JSON...I just happen to be familiar with these ones.

    #!/usr/bin/env perl use strict; use warnings; use Data::Table; use Cpanel::JSON::XS; my $table = Data::Table::fromCSV('data.csv'); my @column_names = $table->header; foreach my $col_name (@column_names) { my @column_values = $table->col($col_name); my $json = encode_json(\@column_values); print "var $col_name = $json;\n"; } exit;
    The output is:
    var secid = ["002826","0028262","0028262","0028262","0028262","0028262 +"]; var Initial_shares = ["3777","3777","3777","3777","3777","3777"];
    This code should handle data.csv files with different numbers of columns.
Re: Get CSV data and convert it to other format data dynamically.
by shmem (Chancellor) on Aug 25, 2015 at 08:46 UTC
    my @a=(); my @b=(); my @a1=(); my @b1=();

    Whenever you feel the need to declare variables like @a,@b,@c,... or @a1,@a2,... it is almost always better to use a container, specially if you want to extend those items without having to declare @d,@e or @a12,@a13 later on.

    This also greatly simplifies your code. You want an array of columns, without having to know how many columns there will be? Use a two-dimensional array, i.e. an array of arrays. This is done with references in perl - see perlref.

    open (TXT, "<$filename") || die "Can't open $filename: $!\n"; my @array; while (my $line=<TXT>) { chomp($line); my @list = split /,/, $line; # now distribute the elements into each column's array for ( 0 .. $#list ) { push @{$array[$_]}, $list[$_]; } }

    Your @array now looks like this:

    @array = ( [ 'secid', '002826', '0028262', '0028262', '0028262', '0028262', '0028262' ], [ 'Initial_shares', '3777', '3777', '3777', '3777', '3777', '3777' ] );

    Now populate @perl_array with the stringified elements of @array:

    my @perl_array; foreach $array_ref ( @array ) { my $colname = shift @$array_ref; my $items = join ",", @$array_ref; push @perl_array, "var $colname=[$items]"; } for ( @perl_array ) { print $_,"\n"; }

    Output:

    var secid=[002826,0028262,0028262,0028262,0028262,0028262] var Initial_shares=[3777,3777,3777,3777,3777,3777]

    Adding more columns to your data results in more elements in @perl_array. Using as input

    secid,Initial_shares,Char 002826,3777,a 0028262,3777,b 0028262,3777,c 0028262,3777,d 0028262,3777,e 0028262,3777,f

    results in

    var secid=[002826,0028262,0028262,0028262,0028262,0028262] var Initial_shares=[3777,3777,3777,3777,3777,3777] var Char=[a,b,c,d,e,f]

    And yes, you should use Text::CSV or such as mentioned elsewhere.

    perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: Get CSV data and convert it to other format data dynamically.
by Anonymous Monk on Aug 24, 2015 at 20:52 UTC

    You should use Text::CSV to read the data and one of the JSON modules to output it.

      Will i be able to do it dynamically ? that is my concern.

Re: Get CSV data and convert it to other format data dynamically.
by james28909 (Deacon) on Aug 25, 2015 at 02:21 UTC

    Here is my input on it, which I am pretty new to hashes as well so please keep that in mind ;)

    use strict; use warnings; use Data::Dumper; my %hash = (); my $i = 0; while (<DATA>) { chomp; my ( $sec_id, $Initial_shares ) = split( /,/, $_ ); $hash{secid}{$i} = $sec_id; $hash{init_sh}{$i} = $Initial_shares; $i++; } #print Dumper \%hash; foreach my $outter_key ( keys %hash ) { print "$outter_key => {\n"; while ( my ( $inner_key, $inner_value ) = each $hash{$outter_key} +) { print "\t\t" . qq('$inner_key') . " => " . qq('$inner_value') +. "\n"; } print "};\n\n"; } __DATA__ 002826,3777 0028262,3777 0028262,3777 0028262,3777 0028262,3777 0028262,3777

    Outputs:

    init_sh => { '4' => '3777' '1' => '3777' '3' => '3777' '0' => '3777' '2' => '3777' '5' => '3777' }; secid => { '4' => '0028262' '1' => '0028262' '3' => '0028262' '0' => '002826' '2' => '0028262' '5' => '0028262' };

    This method looks like it would be a sure fire way to keep track of what belonged to what as well.

    EDIT: I dont know why, but I thought OP was looking for a solution with hashes... anyway on a side note, while I was messing around with this, I did figure out another way to load modules haha;

    EDIT: Updated code.

      If you use Text::CSV_XS' csv function with the filter attribute, you don't also keep the CSV data and stream the required content right into the arrays:

      use 5.14.2; use warnings; use Text::CSV_XS qw(csv); my (@a, @b); my $aoa = csv ( in => *DATA, headers => "skip", filter => { 1 => sub { push @a, $_; 1 }, 2 => sub { push @b, $_; 0 }, }); { local $" = ", "; say "var secid = [@a]"; say "var Initial_shares = [@b]"; } __END__ secid,Initial_shares 002826,3777 0028262,3777 0028262,3777 0028262,3777 0028262,3777 0028262,3777

      var secid = [002826 0028262 0028262 0028262 0028262 0028262] var Initial_shares = [3777 3777 3777 3777 3777 3777]

      The entries in the filter will be used in ascending order. All but the last should return a true value. The last should return a false value to reject the record: You already stored all required data in the two arrays.

      update: Alternatively, you can put everything in a hash:

      my %h; my $aoa = csv ( in => *DATA, headers => "auto", filter => { 1 => sub { for my $f (keys %_) { push @{$h{$f}}, $_{$ +f} }; 0 }}, );

      { Initial_shares => [ 3777, 3777, 3777, 3777, 3777, 3777 ], secid => [ '002826', '0028262', '0028262', '0028262', '0028262', '0028262' ] }

      Enjoy, Have FUN! H.Merijn