It's not nearly as bad as it sounds. You can use Parallel::ForkManager to handle the parallelism. Don't get hung up on dynamically managing the number of workers -- it should be good enough to have a limit. Use a relational database to store the queue. Have a status field for each job that you can switch between "new", "in-progress", and "complete." When a worker takes a new job from the queue, change its status, and use locking to prevent two workers from grabbing the same job. When a job completes, write the result back into the database and update the status again. Hopefully that's enough to get you started.