in reply to LWP question

There's no general solution to this sort of problem. You either need to find the parameter (if it exists) that specifies how many results are shown per page or you need to write your user-agent to pull down successive pages (for you yahoo page it's passing the ".ss" parameter.)

One hint is to look on the "Advanced Search" page to see what are the possible parameters you can use. In this case, it doesn't look like there's an option to specify the page length.