Dear PerlMonks,

I have a formatting & data printing question for you. I have a file that has hundreds of lines containing columns of organized information. The data can be separated by a tab, or multiple spaces. In each line, the only “valid sets” are those sets that contain the structures: SATID XX VAL1 XX VAL2 XXX SIGNAL XX.

Hence, since my file sometimes has “invalid sets” that do not match exactly the structure of the “valid sets” mentioned earlier. These “bad data” can be found a single or numerous times in a single line. An example of these “invalid” sets could be:
SATID 18 SATID 17 VAL1 49 VAL2 038

Therefore, is there a way that Perl could automatically find the “invalid sets” and replace them with blank spaces (so as to preserve the original spacing/alignment of columns in the large file?

For example, if I have the following raw data:
Timestamp: 00:55:46     SATID 17 VAL1 49 VAL2 038 SIGNAL 39 SATID 18 S
+ATID 17 VAL1 49 VAL2 038  SATID 19 VAL1 69 VAL2 015 SIGNAL NA        
+                            SATID 39 SATID 28 VAL1 36 VAL2 073 SIGNAL
+ 21


The “corrected” data line should be:
Timestamp: 00:55:46     SATID 17 VAL1 49 VAL2 038 SIGNAL 39           
+                          SATID 19 VAL1 69 VAL2 015 SIGNAL NA        
+                            SATID 39 SATID 28 VAL1 36 VAL2 073 SIGNAL
+ 21

[download]
I’m not sure how to tackle this… I think that using an array would be the way to go, but I’m not sure how to do this so that the matching structure is enforced & the correct substution (when needed) is implemented.


Does anyone have any ideas/examples that could be of help? Below is the code that I have so far:
#!/usr/bin/perl -l

use strict;
use warnings;

my @lines;


while(<DATA>)
  {
    push (@lines, $_);
  }


print @lines;    # see if it worked


__DATA__
Timestamp: 00:55:46     SATID 17 VAL1 49 VAL2 038 SIGNAL 39 SATID 18 S
+ATID 17 VAL1 49 VAL2 038  SATID 19 VAL1 69 VAL2 015 SIGNAL NA        
+                            SATID 39 SATID 28 VAL1 36 VAL2 073 SIGNAL
+ 21

[download]

Thanks =)

In reply to Preserving "Valid" Data? by coding1227

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.