Thanks for your help.

Just to note, the server is our own server, so the respect for robots.txt isn't really an issue. What I need is an automated tool to check sites for basic compliance with our policy, i.e. "all pages must have meta-tags" and "all pages must have a descriptive title".
--

($_='jjjuuusssttt annootthheer
     pppeeerrrlll haaaccckkeer')=~y/a-z//s;print;
[download]