jockel,
Sometimes it is easier to break the problem up into smaller pieces, and that's why it is so nice that there is always more than one way to do it.

#!/usr/bin/perl -w
use strict;

my $string = 'column1    |column two       |might be column3';
my @columns = split /\|/ , $string , -1;
$_ =~ s/\s+$// for @columns;
$string = join '|' , @columns;
[download]

Cheers - L~R
Updated: Added -1 as 3rd argument to split to account for possible trailing |||| suggested by Abigail