I have an issue that I'm not sure how to approach. I apologize in advance if I do not explain my delima clearly the first time. I'm not sure how to explain it...

I have CSV files that contain statistical data. I need to be able to extract this data in a format that gives me the TOP 5% (highest value) for certain columns, along with certain the data contained in the same rows. What's got me stumped, is that I don't need the TOP 5% of the table, but the top 5% of each server having data collected on it. I also need the TOP 5% of multiple columns. Let me provide an example:

My data looks similar to this (just LOTS more of it). There are a lot more columns, so I trimmed this down a little. The first row is the headers: 
PDHCSV40EasternDaylightTime240,ERWWCOMMUNITIES_MemoryPagessec,ERWWCOMM
+UNITIES_NetworkInterfaceCompaqEthernetFastEthernetAda
pterModuleBytesTotalsec,ERWWCOMMUNITIES_NetworkInterfaceMSTCPLoopbacki
+nterfaceBytesTotalsec,ERWWCOMMUNITIES_NetworkInterfac
eCompaqEthernetFastEthernetAdapterModuleCurrentBandwidth,ERWWCOMMUNITI
+ES_NetworkInterfaceMSTCPLoopbackinterfaceCurrentBandw
idth,ERWWCOMMUNITIES_PhysicalDisk0CPCTDiskTime,ERWWCOMMUNITIES_Physica
+lDisk1DPCTDiskTime,ERWWCOMMUNITIES_PhysicalDiskTotalP
CTDiskTime,ERWWCOMMUNITIES_PhysicalDiskTotalAvgDiskQueueLength,ERWWCOM
+MUNITIES_ProcessorTotalPCTProcessorTime,ERWWDC1_Memor
yPagessec,ERWWDC1_NetworkInterfaceIntelPRO100NetworkConnectionBytesTot
+alsec,ERWWDC1_NetworkInterfaceMSTCPLoopbackinterfaceB
ytesTotalsec,ERWWDC1_NetworkInterfaceIntelPRO100NetworkConnectionCurre
+ntBandwidth,ERWWDC1_NetworkInterfaceMSTCPLoopbackinte
rfaceCurrentBandwidth,ERWWDC1_PhysicalDisk0CPCTDiskTime,ERWWDC1_Physic
+alDiskTotalPCTDiskTime,ERWWDC1_PhysicalDiskTotalAvgDi
skQueueLength,ERWWDC1_ProcessorTotalPCTProcessorTime
06302005141157.588,16.581489221609786,644.14319700475187,129.796028843
+00556,100000000,10000000,4.6331155655937869e-009,4.42
86442834463436e-009,4.5308799245200652e-009,9.0617598490401305e-011,99
+.985420635482711,0.35448523206811611,3025.96021122622
05,2095.5728628079773,100000000,10000000,2.8398789187023344e-009,2.839
+8789187023344e-009,2.8398789187023345e-011,99.9902273
56518915,1.7378886565248808,1690.2294964462301,13.292340672916287,1000
+00000,10000000,0.00015427048318971214,0.0001542704831
8971214,1.5427048318971215e-006,99.990256224867423

[download]

The first column is a date/time stamp. The rest of the columns are server name & statistic collected. For each column with "ProcessorTotalPCT" in it, I need the TOP 5 % numbers along with the data from the same column. Then I also need the same thing done with for the columns with "MemoryPagessec" in it. I would also like the data split into to files. One for CPU and one for the RAM info.

Final output should look similar to:

Date/time,servername,data,data,data,...
Date/time,servername,data,data,data,...
Date/time,servername,data,data,data,...
Date/time,servername,data,data,data,...

[download]

Thanks in advance. I hope that I wasn't too confusing.

In reply to Advances SQL calculations in CSV file by DrAxeman

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.