I guess that is a bug.

Going by the comment in perlmonks.org/robots.txt, it is not a bug, but rather deliberate.

Perhaps, www.perlmonks.org, serves only periodic, static snapshots, rather than server intensive, dynamic content?