I don't know what would be causing the problem in your last section of code, but in the second section you aren't removing the newline from <STDIN> before feeding it to the crawler. This could be the problem.
Comment on Re: Strange way a string or array works
... in the second section you aren't removing the newline from <STDIN> before feeding it to the crawler. This could be the problem.
That was my first guess as well, but his problem is It won't even let me type a URL in. Which means, there's nothing to feed the crawler - the program doesn't get this far.