Obligatory disclaimer that I only used it the one time, so there are some gaps in my knowledge on this, but perlbrew lets you build a private perl environment (or multiple private perl environments) separate from your system perl environment, and everything in that environment is essentially its own sandbox, with its own bin/, lib/, and man./ directories. Any modules need to be installed for the perlbrew environment through cpan inside of perlbrew. Perl/Tk is a wrapper around Tcl/Tk and may reference system libraries (i.e. X11 or image libraries) that are not part of perlbrew, and perlbrew does not provide them.

In practice, if you install XQuartz, Perl/Tk build scripts should detect X11 headers and libraries installed in /opt/X11 as normal. I believe XQuartz will install libjpeg for you, but if not, installing those through Homebrew or Macports should work. Modules like Tk will compile against the system libraries that they find -- which is something you might have to be careful about if there are multiple instances of those installed on your system.

The advantage to the perlbrew method was to build an isolated perl environment for the .app bundle that would not rely on the presence of whatever perl environment comes on a mac, which will let you ensure cpan module compatibility for the private environment in your app bundle. But this is why you would want your .app bundle to check for the presence of XQuartz and direct users to install it if they don't have it, as those system resources still need to be installed at the system level.