AFAIK, prime number factorization has little to do with symmetric encryption algorithms. you are correct when it comes to asymmetric ciphers (e.g. RSA, DSA), which are indeed vulnerable to attack based on the fact that the public and private keys are tied to each other by their relationship to a particular, large prime; but since Blowfish is a symmetric block cipher, it is not susceptible to attacks based on primes.