At the risk of bringing religion into the question: A number of various belief systems refer to their deity returning at some point. Assuming the return from another location in time and space, that would make them extraterrestrials. And assuming they are in fact returning, that would mean they have already been here.

From the purely scientific standpoint, I am unaware of any hard evidence that extraterrestrials have been here, and am equally unaware of any hard evidence that they have not.

Until we have hard evidence that there are extraterrestrials, it will be rather hard to determine where they have been and where they have not been I would think.