Preserve whitespace with PPI::Tokenizer

andreas1234567 has asked for the wisdom of the Perl Monks concerning the following question:

I am writing one a small and naīve script that uses String::CamelCase to decamelize perl source code. The script uses PPI::Tokenizer to split the input into tokens. It works as expected except for the added whitespace. I would like the whitespace to be preserved as is, but the output contains lots of newlines.

How do I get rid of the extra newlines in the output?

use strict;
use warnings;

use String::CamelCase qw(decamelize);
use PPI::Tokenizer;

my @lines     = <DATA>;
my $tokenizer = PPI::Tokenizer->new(\@lines);

while (my $token = $tokenizer->get_token) {
    no warnings q{uninitialized};
    my $no_decam = 0;
    ++$no_decam if $token =~ m/^\s*$/;           # skip whitespace
    ++$no_decam if $token =~ m/^__\w+__$/;       # skip packages
    ++$no_decam if $token =~ m/::/;              # skip modules
    ++$no_decam if $token =~ m/^['"].*['"]$/;    # skip quoted strings
    
    ($no_decam) ? print $token : print decamelize($token);
}

1;

__DATA__
sub new {
    my $Class = shift;
    my $Self  = {};
    bless($Self, $Class);

    $Self->{FooBar} = Foo::Bar->new(base => __PACKAGE__);
    if (!$Self->{FooBar}) {
        croak("Foo::Bar init failed!");
    }

    return $Self;
}
[download]

Actual output:

sub new {

    my $class = shift;

    my $self  = {};

    bless($self, $class);



    $self->{foo_bar} = Foo::Bar->new(base => __PACKAGE__);

    if (!$self->{foo_bar}) {

        croak("Foo::Bar init failed!");

    }



    return $self;

}
[download]

Wanted output:

sub new {
    my $class = shift;
    my $self  = {};
    bless($self, $class);

    $self->{foo_bar} = Foo::Bar->new(base => __PACKAGE__);
    if (!$self->{foo_bar}) {
        croak("Foo::Bar init failed!");
    }

    return $self;
}
[download]

--
No matter how great and destructive your problems may seem now, remember, you've probably only seen the tip of them. [1]

Comment on Preserve whitespace with PPI::Tokenizer Select or Download Code

Replies are listed 'Best First'.
Re: Preserve whitespace with PPI::Tokenizer by ikegami (Patriarch) on Feb 27, 2009 at 14:24 UTC
It's obviously preserving whitespace, just adding newlines. Fix: `chomp(@lines);` [download]	[reply] [d/l]