in reply to Splitting the records into multiple worksheets

Excel is the wrong tool if you are processing > 10k rows.

I would recommend a sqlite database.
It has perl interfaces via the DBI and DBD::Sqlite perl modules.

There are plenty of snippets to get you started - check out this cookbook.

                "Avoid strange women and temporary variables."

  • Comment on Re: Splitting the records into multiple worksheets

Replies are listed 'Best First'.
Re^2: Splitting the records into multiple worksheets
by chandantul (Scribe) on Feb 20, 2021 at 04:43 UTC

    Thanks for the suggestion but Excel is the requirement as we need to generate the report with pivot chart with another sheet in the same excel file. The above solution will change complete code design as well. Do you have any other solution?

      After creating the raw data table in SQLite, you could use SQL to generate your pivoted data.

      The SQL "GROUP BY" clause can do the pivot functions to summarize.

      The result can then easily export into a csv file suitable for import into Excel.

                      "Avoid strange women and temporary variables."

        Hello, Do you have any specific example of pulling the raw data into DB? Please check below how i am pulling raw data and let me know next step if possible. How i should import into csv and then excel if possible.

        my $urlstringlog = $apiurllog . $sortor . $filter . $target . $appID . + $date2; $strExcelFilename = "C:/" . "Analysis-Details-automatic-bk" . $date . +".xlsx"; do { # Run get the users run_api_call($urlstringlog); @responsetext = parse_json ($client->responseContent()); push @responsetextall, @responsetext; $linkheader = $client->responseHeader("link"); if ($linkheader=~ m/next/) { (my $link1 = $linkheader) =~ s/ .*? (self)/$1/gx; (my $link2 = $link1) =~ s/self/$1/g; print "Post Pagination: $link10"; $urlstringlog = $link2; } } while ($linkheader=~ m/next/); for my $i (0..$#responsetextall) { $responsetextall[$i] =~ s/]\[/,/g; for my $j (0..$#{$responsetextall[$i]}) { @responsests1 = $responsetextall[$i][$j]{tat}; $responseid = $responsetextall[$i][$j]{act}{id}; $responsdisp = $responsetextall[$i][$j]{act}{Name}; $responsalter = $responsetextall[$i][$j]{act}{Id}; $responseclientipadd = $responsetextall[$i][$j]{cln}{ipAd +}; $responseappsdebug = $responsetextall[$i][$j]{deb}{Data}; + for my $m (0..$#responsests1) { for my $n (0..$#{$responsests1[$m]}) { $responseapps2id = $responsests1[$m][$n]{id}; $responseapps2 = $responsests1[$m][$n]{ty}; $responseapps1 = $responsests1[$m][$n]{Name}; if ( $responseapps2 eq 'AppInstance' ){ @responseapps3 = $responsests1[$m][$n]{displayName}; $responseapps5 = $responseapps3[0]; print "Its Matches Appinstances"; $responseapps4 = $responsests1[$m][$n]{Name}; if ( $responseapps2 eq 'AppUser'){ $responseapps3id = $responsests1[$m][$n]{a +Id}; } } push @responseapps3id,$responseapps3id; push @responseapps5 , $responseapps4; push @responseapps7, $responseapps7; } } push @responsalter, $responsalter; push @responseclientipadd,$responseclientipa +dd; } }