Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??
Not really perl related, but we've had great success with openpbs, aka "Open Portable Batch Scheduler".

  • open source
  • provides a tcl ( I know, I know ) interface where you can even write your own scheduler
  • stdout/stderr watching, as you mentioned
  • an open api for reporting status back
  • dependency chain ( only run job b if job a succeeds, etc)
I am only using it for running sql batch jobs, as I needed the job dependency chains. Would have gotten by with cron otherwise. So, since I'm not using it like you would, I'm not sure if it covers your other needs. I will note that it expects to run scripts, and not binaries ( it reads the script into stdin for later feeding to whatever is on the shebang line ). You would have to create wrapper scripts for your render jobs...

Here's the blurb from their website:

The Portable Batch System (PBS) is a flexible batch queueing and workload management system originally developed for NASA. It operates on networked, multi-platform UNIX environments, including heterogeneous clusters of workstations, supercomputers, and massively parallel systems. Development of PBS is provided by Altair Grid Technologies.


In reply to Re: Parallel Processing, Queueing and Scheduling by kschwab
in thread Parallel Processing, Queueing and Scheduling by submersible_toaster

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (1)
As of 2024-04-19 00:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found