I remember that in a previous post, this matter was discussed and that someone pointed out that Google absolutely hates bots on their site and will go even as far as blocking the IP-address of recalcitrant bots. If you have a dynamic IP-address this could really upset your provider.

The prefered way to address them automatically is through their published API.