in reply to splitting text into lines -- code -> regex

Examples: 
012345678901234567890123456789012345678901234567890
 1234567890 VON 14.06.2004 BUCHUNGSKONTO 1234567890

[download]
Should become: 
123456789012345678901234567
1234567890 VOM 14.06.2004__
BUCHUNGSKONTO 1234567890___

[download]
(underscores and count on first line added for clarity) 
0123456789012345678901234567890123456789012345678901234567890123456789
+012345678901234567890123456
 R12345678-A1234567 INCL.EUR 3,31 MWST JULI MONATL. GEB HR T-DSL FLAT 
+01.07.04-31.07.04     12,34

[download]
Should become: 
123456789012345678901234567
R12345678-A1234567 INCL.EUR
3,31 MWST JULI MONATL. GEB_
HR T-DSL FLAT______________
01.07.03-31.07.04     12,34

[download]
(underscores and count on first line added for clarity) (Note that in this case, my algorithm doesn't split it in quite the same place that the original did -- and not that yes, DTAG does spell Gebuehr as two words, for some strange reason, for those that know German.)

It appears that my bank is doing (the equivlent of) $comments = join (' ', '', map {s/ *$//; $_} @comments;; I'm attempting to undo that, and get back @comments.

Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).

Replies are listed 'Best First'.
Re^2: splitting text into lines -- code -> regex
by davido (Cardinal) on Jul 12, 2004 at 08:13 UTC 
    use strict;
use warnings;

my $text =   " R12345678-A1234567 INCL.EUR 3,31 MWST JULI MONATL. GEB 
+HR T-DSL FLAT 01.07.04-31.07.04     12,34";

while ( $text =~ / (.{0,27}(?!\S))/g ) {
    my $out = $1 . ( '_' x ( 27 - length( $1 ) ) );
    print "$out\n";
}

    [download]

    Underscores added to output for clarity.

    Update: Just for the record, (?!\S) is a negative lookahead assertion. It ensures that lines are always split on the last possible space character up to the 27 character line limit. In other words, each line must contain a leading space, followed by up to 27 characters, and the next character can't be a non-space. That way the next line will always synch up with the requisite 'space' delimiter.


    Dave

[reply]
[d/l]
[select]
Re^2: splitting text into lines -- code -> regex
by bart (Canon) on Jul 12, 2004 at 08:40 UTC
    This regex seems to match both your examples. 
       /(?:\s|^)(.{1,27})(?=\s|$)/g

    [download]
    You must do some padding, with sprintf or pack for example, to append extra padding spaces. 
    @padded = map { sprintf "-%27s", $_ } /(?:\s|^)(.{1,27})(?=\s|$)/g;

@padded = map { pack "A27", $_ } /(?:\s|^)(.{1,27})(?=\s|$)/g;

    [download]
[reply]
[d/l]
[select]
Re^2: splitting text into lines -- code -> regex
by pelagic (Priest) on Jul 12, 2004 at 09:19 UTC
    To me this looks like a wrapping problem.
    And there's a CPAN Module out there that might help: Text::Wrap.

    pelagic
[reply]
Re^2: splitting text into lines -- code -> regex
by graff (Chancellor) on Jul 13, 2004 at 03:08 UTC
    I would agree with pelagic: it smells like a wrapping problem. Looking at your second example, what if it had been: 
    0123456789012345678901234567890123456789012345678901234567890123456789
+01234567890123
 R12345678-A1234567 INCL.EUR 3,31 MWST JULI MONATL. GEB HR T-DSL FLAT 
+01.07.04 12,34

    [download]
    Should it then come out like this? 
    123456789012345678901234567 
R12345678-A1234567 INCL.EUR 
3,31 MWST JULI MONATL. GEB_
HR T-DSL FLAT 01.07.04_____
12,34______________________

    [download]
    The point is that, if your resulting @comments elements must all be 27 characters long, with space-padding at the end where necessary, and you must not break up any existing (\S+) token across elements, just add tokens to a current element string until the next token would put it over 27 characters, pad to 27 if necessary, and make the next token the start of the next element. I believe Text::Wrap supports this sort of logic, but it doesn't seem that hard to roll it from scratch.

    (Or maybe the bank really is doing something more complicated than that, or maybe they just screwed up big-time and made it impossible for you to reliably do what you want.)

[reply]
[d/l]
[select]

      It does sound an awful lot like a wrapping problem. However, with reference to your parenthetical: The bank already makes it impossible to figure out exactly what the original was, by de-wrapping. The reason is that the 27-character lines can be mostly spaces. For example, the dewrapping I gave for the T-DSL example above was how my algo de-wrapped it. The original looks like (pipes added to show end-of-field):

      R12345678-A1234567 INCL.EUR|
3,31 MWST JULI             |
MONATL. GEB HR T-DSL FLAT  |
01.07.04-31.07.04     23,99|

      [download]
      Unfornatly, there is no way of telling that without domain knowladge. (So how do I know that? I load the more-info page on that transaction, but I can't figure out how to do that programatacilly reliably, as the bank keeps changing session IDs. Worse, if I get it wrong, I don't get an error reliably -- I simply get information on the wrong transaction!)

      As to using Text::Wrap... well, I was going to point out the bit in the docs where it says "it will destory any whitespace in the original text", but it turns out that only applies to Text::Wrap::fill, not Text::Wrap::wrap. OTOH, I don't see how it could be better then the current solution, and the current solution is already working, tested, and readable.

      Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).

[reply]
[d/l]
        Is there some reason for that i link in the last paragraph? Just curious.
[reply]

In Section Seekers of Perl Wisdom