in reply to Re^2: Why Coro?
in thread Why Coro?

Every one minute, device will store a 8 decimal number(like 00004000) in its register.

Can you clarify the definitions of "device" (type?), and "register" in this context?

Specifically, will there be custom software running on each of these devices that receives the TCP requests, reads the "register" and replies with the value read?

Or are the registers on the devices accessible directly via a TCP request? (Perhpas via firmware?)

If the former is the case, then you could more simply have that custom software read the register and send it to the DB directly. The read and send cycle could be synchronised to the local hardware, and the DB & TCP take care of queuing the data at the DB end. Very simple and reliable.

If the latter, the problem of whether Coro alone is suitable comes down to whether a single thread can dispatch & retrieve, with context switches, to 12000 machines in 60 seconds. 12000/60 = 200. Assuming permanent connections, that's 200 writes, 200 reads and 400 context switches per second. On a Gigabit LAN maybe you'd be okay, but I think you'd be pushing your luck across a WAN or the internet. With 8 or 12 cores, it would probably be okay. Though I'd want more information about latencies to convince myself of that.

Multiple Coro instances (processes), whether running on the one or more multi-core machines, sounds like a possible solution. Though you still have the problem that if all the devices allocated to a particular instance decide to respond at the same instance, they will all be serialised through one core/machine, whilst all the others (potentially) stand idle. Ensuring that you poll every device one per minute reliably would require extensive testing.

The upside is that you should be able to determine the maximum number of devices a single instance can reliably support during testing, and then split your devices in to suitably sized groups. As the number of machines ramps up, you just start more processes. The downside is that you have to carry the overheads of running many more instances than are required for the average case in order to ensure for the occasional confluence spike.

If you have the opportunity to have the devices post their data to the DB directly rather than polling, you should definitely opt for that. But reading between the lines I suspect that you are talking about SNMP devices?

If these are SNMP requests, then you should also look closely at the non-blocking request protocols in Net::SNMP. It may be all you need.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy

Replies are listed 'Best First'.
Re^4: Why Coro?
by thargas (Deacon) on Jul 28, 2010 at 18:15 UTC
    I second the Net::SNMP note. I've used the non-blocking stuff in production on blocks of 10000 hosts from a single process and usually been able to get all of them in a minute.
Re^4: Why Coro?
by james2vegas (Chaplain) on Oct 28, 2010 at 17:10 UTC
    For the multiple Coro instances situation you might look at Coro-MP which is a Coro wrapper over AnyEvent-MP which lets you co-ordinate multiple AnyEvent loops / Coro instances running in separate processes on one or multiple hosts