In many situations with modern cryptographic algorithms, it will not make a difference since we can specify how hard we want decryption to be. In others, it will matter.

In essence the context will help with weak cryptographic algorithms, or if the same exact message will be passed many times. Also in some situations you will be in a better position to do a dictionary attack if you only have one password to guess (and know the algorithm).

Whether this matters depends on the exact circumstances you find yourself in.