Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

RE: TCP/IP processing systems

by Odud (Pilgrim)
on Jun 13, 2000 at 23:39 UTC ( [id://17957]=note: print w/replies, xml ) Need Help??


in reply to TCP/IP processing systems

If writing it yourself you would need to be very careful with error handling and recovery. At our site we use a expensive commercial product (not written Perl!). Most of the problems arise when servers "drop out" either because of server faults or network faults. The recovery (i.e. working out where each server is up to with the schedule, what has failed, what has finished) is the area that gives us the most problems.

The basic principle is that one machine is the master. It organises the others and gives them a schedule of what they need to run. When a machine needs to find out the status of a job on another it asks the master. There is a fairly high network load as all the agents are in communication with the master and vice versa. If the master fails then one of the agents can become a new master, for short failures then the agents can run independently and then catch up with the master when it again becomes available.

I'm sure this is very do-able in Perl and I'm not trying to put you off - it is just that you need to have an idea of the magnitude of the task before you start.

Having said all that if we are running thousands of jobs across >100 machines each night - if you just want to write a better cron replacement with a central console I don't think it would be impossible.

It might be worth searching to see if someone has anything written in C that wouldn't be too hard to port.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://17957]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others wandering the Monastery: (2)
As of 2024-04-20 03:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found