Could you elaborate on your problem? If you are trying to avoid the special directories named '.' and '..', don't you want the 'and' logical operator instead of 'or'?

Keep in mind that readdir returns both files and all sub-directories, not just the specially-named ones. If you want to filter out all directories, you can use -X.