The state of audio processing with Perl

Hi there, fellow monks!

My name is Joost and I'm going to talk about audio processing in perl.

I've been playing with "live" (not really realtime, but close enough) audio generation/processing in perl for some time now and I'm not really impressed by the code on CPAN to do that.

For one thing, there appears to be no portable module to play or record audio with any kind of precision. I mean just some code to throw an audio buffer at the soundcard (OIW, in-process handling of data vs playing an audio file). Sure there's Audio::Play but that only supports monophonic sound and doesn't record.

Also, most audio modules use incompatible formats for their data and lots of them are badly documented and/or haven't been updated for years. In other words, coupling these modules is a mess (and slow).

More issues: reading and writing different audio file formats. I've taken a stab at that just now - but it still lacks some important formats.

Here are some of the things I feel are needed:

One audio data interchange format. Audio::Data seems to me to be a good starting point, but it lacks low-level access (i.e. a C level interface and packed float/double strings for fast processing).
A portable audio playback/recording library. Preferably one that will handle everything that Audio::Play handles but with unlimited channels and a JACK interface.
Some kind of processing API that allows coupling of perl & XS code. Something like my Audio::LADSPA module, but more flexible and with an easier Perl interface.

So here's my question: is anyone interested in this kind of thing? By myself I might be able to get this stuff done in a year or so, considering the time I've got to spend, but I'd really like people to supply questions, burning needs, code and/or documentation to get this thing on the right track.

Also, if you are writing any audio modules on CPAN, I'd like to know about it.

Please let me know I'm not the only one interested in this kind of thing :-)

In other words: audio people of perl, let's hear your crazy ideas!

Cheers. J.

"What should it profit a man, if he should win a flame war, yet lose his cool?"

Comment on The state of audio processing with Perl

Replies are listed 'Best First'.
Re: The state of audio processing with Perl by ambrus (Abbot) on Oct 16, 2006 at 08:20 UTC
I don't know if it would work, but you can try popening sox. It can record and play raw (no header) sound files.	[reply]
Re^2: The state of audio processing with Perl by Joost (Canon) on Oct 16, 2006 at 09:31 UTC
Hmmm I hadn't thought about that. I doubt it'll be very responsive. Also, it uses OSS instead of ALSA on linux and it looks like it doesn't support more than 2 channels of hardware I/O (but I might be wrong there). On the other hand, it will probably work on almost any unix system, so it might be worth it if only as a fallback. "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re: The state of audio processing with Perl by zentara (Cardinal) on Oct 16, 2006 at 12:57 UTC
I think I mentioned before about using SDL. Right now it's cross-platform, handles multiple channels, and is probably as fast as you get with Perl (it's used for Perl games). I havn't seen an example of recording with it though, and it dosn't really handle all the various audio formats well. SDL::Mixer and SDL::Music P.S. I just googled for "recording SDL" and found some interesting links. I'm not really a human, but I play one on earth. Cogito ergo sum a bum	[reply]
Re^2: The state of audio processing with Perl by Joost (Canon) on Oct 16, 2006 at 14:03 UTC
The problem with Perl/SDL, is that as far as I know, you can only play existing sound files. In other words, you can't stream and you can't generate your own samples on the fly. That's probably great if you want to use pregenerated samples in games, but if you want to do your own "live" effects etc it's pretty useless. update: after some browsing, it seems that SDL itself should be reasonably good for this kind of thing, but the Perl API has abstracted away the required low-level hooks. "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re: The state of audio processing with Perl by talexb (Chancellor) on Oct 16, 2006 at 14:26 UTC
While I'm a big fan of Perl, and I love playing with audio and video -- just setting up a Debian workstation to do just that right now -- I don't know if I'll have any spare cycles to actually work on Perl audio stuff. But I'd be happy to test whatever gets produced. My C chops are a little out of date, but in decent shape otherwise. Sounds cool! Count me in. Alex / talexb / Toronto "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds	[reply]
Re: The state of audio processing with Perl by creamygoodness (Curate) on Oct 17, 2006 at 13:29 UTC
In a past life I made my living as an audio mastering engineer. I'm intrigued, but I haven't ever looked into the Audio:: namespace on CPAN. I've never done any real audio programming either, since I started coding in earnest only after leaving the recording biz. But by now I'm an half decent C and XS hacker, and I know a bunch about digital audio theory and audio production QC. It would be fabulous to connect those two worlds, and maybe my esoteric skillset will come in handy. I'd love to teach myself about synthesis. Is there an "Audio Synthesis with Perl" book out there or something like it? I always assumed there wasn't because audio stuff requires a lot of bit twiddling. -- Marvin Humphrey Rectangular Research ― http://www.rectangular.com	[reply]
Re^2: The state of audio processing with Perl by Joost (Canon) on Oct 18, 2006 at 19:10 UTC
I don't think there's any book about audio synthesis with perl. I learned most of the stuff I know from playing with my sythesizer, reading various online texts and writing C++ plugins for Buzz tracker (win32) & Amiga & C64 assembly programming. As far as I'm concerned, the best current module for audio synthesis in perl is my Audio::LADSPA module - but that's mostly because there isn't really anything else :-) I'm planning a rewrite. Besides that, you will probably have to look at Ecasound or C-Sound for scriptable "external" sound engines. I also found Net::LibLO to be quite nice for controlling OSC capable audio programs. "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re: The state of audio processing with Perl by asz (Pilgrim) on Oct 17, 2006 at 11:04 UTC
hello, what i would like to see is a Perl interface for Csound. Csound allows anything imaginable with regard to digital sound manipulation and is only limited by ones creativity. it is licensed under LGPL. with the introduction of a API in Csound 5 (links to a PDF document), a number of interfaces have appeared for Java, LISP, Python, etc. the Python interface for Csound has been used for creating a graphical audio application, named TamTam, that will be included in the laptops from One Laptop Per Child project. unfortunately i have very minimal C coding skills. when i first got the idea of a Csound Perl module, a while ago, i read a little about XS and SWIG, but currently i think i couldn't write that interface, but i would be very happy if somebody else would do it :) also related to this topic is an article about Hacking Perl in Nightclubs that you might find interesting. :)))))	[reply]
Re^2: The state of audio processing with Perl by pemungkah (Priest) on Oct 20, 2006 at 20:20 UTC
May I suggest taking a look at ChucK? I'm very impressed with it as a language. I've done some programming to make music with Applescript (of all things!) using ChucK as an embedded audio processor, and it's worked great. The Chuck language lacks a number of the programming conveniences that you get with Perl, but it does encapsulate the ideas of time and synchronization better than any other music language I know (Csound and Supercollider being the other two I'm at least passingly familiar with). Some modules using ChucK as the engine might be a good idea. Chuck's also designed to be an on-the-fly language: you can add and delete "shreds" in the ChucK VM from the command line, and, if properly programmed, the shreds happily sync up. Worth a look anyway.	[reply]
Re: The state of audio processing with Perl by tsee (Curate) on Oct 17, 2006 at 13:47 UTC
Perhaps you should look at what the SlimDevices developers are doing: They offer (an open source) audio streaming server "slimserver". In that context, at least a couple of Audio::* CPAN modules have been created. Have a look at this CPAN author, for example http://search.cpan.org/~daniel/ Steffen	[reply]
Re^2: The state of audio processing with Perl by Joost (Canon) on Oct 17, 2006 at 22:26 UTC
Hey, that looks interesting. Thanks for the info :-) "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re: The state of audio processing with Perl by jpollack (Novice) on Oct 18, 2006 at 14:19 UTC
Hello! I am, in fact, working on a perl module which provides a real-time interface to the crossplatform PortAudio API. This would provide perl with the ability to record and play audio in real-time via perl callback functions. I currently have ugly proof-of-concept code which works on Windows and Linux (my test app is a .wav player which lets you pause and resume with the keyboard), and hope to clean up and upload to CPAN soon. I had forgotten about this project of mine, and relegated it to the back burner, assuming I was the only one interested in it. :) I'm enthused to see that there is more interest in adding this much needed ability to my favourite language.	[reply]
Re^2: The state of audio processing with Perl by Joost (Canon) on Oct 18, 2006 at 19:00 UTC
Oh cool. I had completely forgotten about PortAudio. That would be just what I'm looking for. If you want any help or feedback, I'd love to take a look at it. Just send me a chatterbox /msg. Cheers, Joost. "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re: The state of audio processing with Perl by Magnachef (Initiate) on Oct 21, 2006 at 07:05 UTC
This has been something of interest to me for a while now. I have been researching the best way to be able to create music with Perl. I gave a talk and did a performance at a Buffalo Perl Mongers meeting last April while discussing my personal adventures of trying to create (decent sounding) music with Perl. A good portion of computer generated (or algorithmic) music, tends to sound like R2-D2 during a screaming match - which can be cool, but not all the time. My goal was to try and create random, dynamic, and improvisational music, controlled by a computer program. The music genre that best encompasses these qualities (and my favorite) is Jazz. I figured that I could encode the fundamentals of playing Jazz music in Perl along with another audio processing backend. I have some slides and links from my talk on the website (http://buffalo.pm.org/#apr18). The slides aren't very informative, but may give you an idea of what I'm doing. I have used Feedback.pl with SuperCollider, and ChucK. I am currently working on a livecoding framework that will use Perl and PD (PureData) as the backend, and allow users to write Perl code that creates and sends OSC messages to the Perl/PD backend (that will play music based on the sent messages). My primary focus is on creating Jazz music, but this should be something that people wanting to play all genres can use.	[reply]