Do I need to specify "use Moose" in every package I define in that .pm file?

I'd say so. use Moose; will, among other things, import functions like has into the current package, and although I'm sure there are ways to be tricky about it, IMO the best thing really is to put a use Moose; in every package using it.

All those "our" declarations and the "require", generated by h2xs

It depends, OO modules usually don't export stuff as well, so personally I would remove the require Exporter; and the our declarations, except for our $VERSION.

Also, note that it's usually considered good practice to only have one package per file. Since Moose also turns on strict and warnings, the bare minimum you need in a .pm file using it is:

package Foo::Bar;
use Moose;
# your code here
1;
[download]

(The use 5.022; might be helpful because it turns on the 5.22 feature bundle, if you're sure you want to limit the minimum Perl version to that.)