How can I stop a thread on demand? Do I need to create some sort of listener or watchdog thread that kills the worker thread?

It's the same as all other threaded Perl apps. You can setup a shared variable, which you constantly test in the thread, telling it to return. Or if you use the latest threads, it has

 # Send a signal to a thread
    $thr->kill(’SIGUSR1’);
[download]

which would be useful for stopping a long running executable.