I'm not familiar with the zsh shell, but I'm pretty sure that shouldn't be necessary.
How did you install perlbrew? Did you follow the basic steps on the perlbrew website?
Were you logged in as the same user you're trying to install modules under? As FryingFinn stated, installing a perlbrew specific cpanm should be all you need after a typical install.