One problem that you may have is that the NET function ConvertFromBase64 does not give the same result as perl MIME::Base64. The results look very similar but doing a hex dump of the perl and the NET versions shows that they are different. My ramblings on this subject are below.
Don't treat it as character data ???
A true Base64 encoding function would be:
Create a byte array which is of an appropriate size, then populate it
by taking each pair of characters in turn and using Byte.Parse with the
number style of HexNumber.
http://bytes.com/topic/c-sharp/answers/234105-base64-encode-characters-127-a
Base64 encoding turns 3 bytes into 4?
http://bytes.com/topic/c-sharp/answers/224907-base64-string-byte-array-whats-length-ratio