It might be worth your while to check the content of $base64_image against the actual JPEG string in the SVG file, just in case there's something you missed (random scenario: it might be prefixed/postfixed with some string in the SVG).

my @jpg = split //, $extracted_jpg_string;
my @ref = split //, $base64_image;

for (0 .. ($#jpg > $#ref ? $#jpg : $#ref)){
     print "Images differ: \$ref[$_]: $ref[$_] | \$jpg[$_]: $jpg[$_]\n
+";
}
[download]

Update: I somehow managed to miss that you'd already verified it. As penance :), I hunted up a random SVG on my system that contained a JPG, ran base64_decode() on the JPG string inside it, and saved the result as a JPG file. Although it does indeed start with a JPG header, none of my image viewers were able to open it, indicating that it's probably not just a simple conversion.

Also, I'd move that first substitution to a position below the JPG replacement. If you happen to have a '##whatever##' in the JPG string, that's going to cause a problem.

-- 
Education is not the filling of a pail, but the lighting of a fire.
 -- W. B. Yeats