You're right, you usually can't get directory listing but
it's sometimes possible to 'browse' directories if the server is setup to display directory indexes
(Options indexes with Apache)
It's not so uncommon nowadays...
With so configured server you can parse the html page returning the directory content
and 'browse' the way you want using LWP.

UPDATE :
Oops, It seems I duplicate Beatnik answer, see below...

"Only Bad Coders Badly Code In Perl" (OBC2IP)