I see this sometimes also. It's for sure a bug somewhere. It's a strange one as it's non-deterministic. You can set your program to crawl exactly the same pages and run it several times as a test. The point at which it starts having this bug will vary within a few gets. It won't crash out on the same page each time, but usually within 0-5 gets of one page.

One method that often fixes it for me is to completely quit Firefox (check that no firefox ps's are leftover) and restart it.

Sometimes even that won't fix it though. I find having your program go slower (insert lots of 5-10s sleeps) will help with the problem. In my case my scrapers remember "seen" pages and so once I make it past the problematic area the bug usually disappears again for a while.