[...] because the server is trying to distinguish between programs (such as those that one might write in Perl), and humans sitting behind browsers clicking on buttons.

Some sites do this by using Javascript, cookies, use Javascript to set cookies, or set cookies in another page that gets loaded by a browser too (typically images, like ads), in order to make this distinction, in an attempt to make the screenscraper's work as close to impossible as they can.

They also often check the REFERER header.