in reply to Remove array if data inconsistent
As moritz pointed out, you must know what concepts data represent to decide on what is important, e.g. a series of 0-s might be useless as suggested by you, when in sound silence-finding application it is crucial.
My advice is to try a little bit looking what might be promissing for you. Experiment with regular expressions filtering and see it obtained results are interesting. If so, go on and filter/ group/ sort more, ad infinitum. Something like
perl -ne 'print if /^INSAUTH\b/'
would produce the output you mention.
Further, Text::CSV is a possible improvement on data analysis preparations.
|
|---|