I think that is a bit "too clever".

I was surprised but, this \1 idea did work although it generates a warning message.

I would argue that easy to understand code is better. And it often executes faster!

#!/usr/bin/perl -w
use strict;

my $previous_line;

while (<DATA>)
{
   if (/;/)
   {
      chomp;
      $previous_line = $_;
   }
   else
   {
      print "$previous_line; $_";
   }  
}
__DATA__
Some Company; Some Street 52
12345 City1
Another Company; Another Street 63
23456 City2
Yet Another Company; New Street 11
34546 City3
[download]