I had misunderstood that you were saying that the law itself is unethical. I think that distinction was probably lost on many others as well.

However, as I think the subsequent discussion clarified, my views are not as simple as saying that thou shalt always do what robots.txt says. (Paying attention to it does not mean that you necessarily agree or do what it says...)