I don't think there's any book about audio synthesis with perl. I learned most of the stuff I know from playing with my sythesizer, reading various online texts and writing C++ plugins for Buzz tracker (win32) & Amiga & C64 assembly programming.

As far as I'm concerned, the best current module for audio synthesis in perl is my Audio::LADSPA module - but that's mostly because there isn't really anything else :-) I'm planning a rewrite. Besides that, you will probably have to look at Ecasound or C-Sound for scriptable "external" sound engines. I also found Net::LibLO to be quite nice for controlling OSC capable audio programs.