All files are binary. What makes the difference is your assumptions about them, and how you treat them.

I'm guessing you have some code that fails when you encounter unexpected bytes in your file, perhaps outside the range [0-9A-Za-z] plus the usual ensemble of CR, LF, TAB, and punctuation symbols.

If you can answer BrowerUK's question - and I encourage you to give it some serious thought - then I also think you will be very close to answering your own question.

I would also note that while you might be able to guess that a file contains undesirable characters (your 'binary' files), you will not know for sure until you open it and look inside.