meetraz has asked for the wisdom of the Perl Monks concerning the following question:

Monks,
I'm working on a utility which issues roughly 100 SNMP get and getTree requests to each of 5000+ servers over a WAN. This is using Net::SNMP on Win32. I need this utility to run as quickly and accurately as possible. Currently, I'm using a couple scripts simultaneously, each using non-blocking requests with callback routines. The whole process is disappointingly slow.. and not very accurate, either. Sometimes the script reports no response from a server, though I can manually issue a request just fine.

Is it possible that since SNMP is based on UDP, there is some data that gets lost in the flurry of packets? Should I limit the number of non-blocking requests I issue in a certain time frame?

Or, should I attack this another way... and use some kind of threading? Does anybody know if Net::SNMP is thread-safe on Win32? Should I just take the plunge and write this in pure C, for speed & threading robustness reasons?

Replies are listed 'Best First'.
Re: SNMP Scanning method
by AcidHawk (Vicar) on Apr 02, 2003 at 10:08 UTC

    I haven't worked with nearly as many servers as you are talking about with perl and snmp, although I have queried 100-120 servers over the WAN.

    In my experience due to the fact that udp is not guaranteed delivery, querying SNMP over the WAN is NOT advisable. Often these packets get dropped for packets with higher priority (particularly on a WAN link that is heavily burdened.)

    What I had to do was create staging servers, if you will, that do the snmp queries over the lan and store the responses into a repository of some sort. I use XML. I then use a TCP connection over the WAN links to ship the XML data to my central server, where I process the results from teh snmp queries.

     ----            ----             ----
|   |___W_A_N___|   |__S_N_M_P___|   |
|   |    T C P  |   |   U D P    |   |
 ----            ----             ----
Central          Staging         Queried
Server           Server          Server

    [download]

    Obviously the Staging Server can just be a Win32 Service or daemon of some type running on an existing piece of hardware.

    HTH.

    -----
    Of all the things I've lost in my life, its my mind I miss the most.
[reply]
[d/l]
      Yeah, I've thought about this.. but we have servers spread over 26 states. Even if we were to have a "staging server" in every state, we'd still be doing SNMP queries over the WAN in each state. Thanks for the suggestion though.

[reply]

        Have you considered an agent manager type technology.

        This may not be feesible as you are talking about 5000+ servers, but what if you did the SNMP queries with a lightweight agent that you deploy to each server, save the responses in a local repository(eg XML file) and then sent those details via TCP to a Central Server. This would take a while to implement and the admin for doing this is high(although a lot could be scripted), but doing it this way can also cut down the traffic you need to send.(I.e. You only have to send CHANGES to the previous XML data not the entire set of data again.)

        Just a thought. I have done this on a much smaller scale than 5000 servers though.

        -----
        Of all the things I've lost in my life, its my mind I miss the most.
[reply]

Back to Seekers of Perl Wisdom