linxdev has asked for the wisdom of the Perl Monks concerning the following question:

I would like to store the data created by the Net-SNMP Perl Module SNMP.pm during SNMP::initMib();

I want to store %SNMP::MIB as an object, but I'm not sure if it is possible since it is a tied hash into the Net-SNMP library.

The initMib() function takes about 9s to execute since it processes many MIBs. These MIBs do not change so I'm thinking that if I could skip this step and just read in an object created from %SNMP::MIB I could save almost all of those 9 seconds.

I don't know if SNMP::initMib() creates the full structure of %SNMP::MIB of if files are still accessed when I look up values via OID within %SNMP::MIB. Anytime I try to traverse that hash I get an 'out of memory' error. Here is some code where I try to dump it. 
use SNMP;
use Data::Dumper;
use strict;

if(not $ENV{'MIBS'}) {
$ENV{'MIBS'} = 'ALL';
}

if(not $ENV{'MIBDIRS'}) {
$ENV{'MIBDIRS'} = '/usr/share/snmp/private:/usr/share/snmp/mibs';
}

&SNMP::initMib();

$Data::dumper::Maxdepth = 2;
$Data::dumper::Maxrecurse = 2;
$Data::dumper::Deparse = 1;
print Dumper(\%SNMP::MIB);

[download]
Anyone know if it is possible to do what I'm trying to do? If so, what is the best way?

Replies are listed 'Best First'.
Re: Storing a tied hash of SNMP
by beech (Parson) on Feb 24, 2016 at 01:11 UTC
[reply]
      I have started that. In the example I only want to run initMib once. Using mibtree.pl I'm stuck doing it at each execution of the program.

      Looking at SNMP.pm I started at the first node of '.1' and used nextNode to create a very large object of the data. Around 15M. It is much faster for me to do a Storable::retrieve() on this than it is to parse all the MIB files via initMib.

      My problem is that my hash keys are by numeric OID. To search by MIB Name::variable I have to walk a hash that has 60k entries. I need to also search via moduleID::label. If add mib keys for 'moduleID::label' => objectID then that will double the size of the object. Maybe use DBI and create a SQLite3 file instead? I'll need to test the performance of that.

      Like this:

      select oid from mib where moduleID = ? and label = ?

      Object gen like this:

      my $node = $SNMP::MIB{'.1'};
my $t = 0;
while(($node = $node->{'nextNode'})) {
    $t++;
    printf "%-40.40s %s::%s\n",$node->{'objectID'}, $node->{'moduleID'
+},$node->{'label'};
    $data->{$node->{'objectID'}}{'objectID'} = $node->{'objectID'};
    $data->{$node->{'objectID'}}{'moduleID'} = $node->{'moduleID'};
    $data->{$node->{'objectID'}}{'label'} = $node->{'label'};
    $data->{$node->{'objectID'}}{'type'} = $node->{'type'};
    $data->{$node->{'objectID'}}{'description'} = $node->{'description
+'};

}
print "Total: $t\n";
#print Dumper(\%SNMP::MIB);

nstore $data ,'snmp-data.obj';

      [download]
[reply]
[d/l]
[select]

        Maybe use DBI and create a SQLite3 file instead?

        Sure, if you need SQL query abilities, DBD::SQLite is good for that

[reply]
Re: Storing a tied hash of SNMP
by perlfan (Parson) on Feb 24, 2016 at 21:08 UTC
    In the POD for SNMP, it says:

    a tied hash to access parsed MIB information. After the MIB has been loaded this hash allows access to to the parsed in MIB meta-data(the structure of the MIB (i.e., schema)). The hash returns blessed references to SNMP::MIB::NODE objects which represent a single MIB attribute. The nodes can be fetched with multiple 'key' formats - the leaf name (e.g.,sysDescr) or fully/partially qualified name (e.g., system.sysDescr) or fully qualified numeric OID. The returned node object supports the following fields...

    Does that help you?

[reply]
      Unfortunately Storable::nstore(\%SNMP::MIB) 'file' does not work, which is what I was trying to do.

      Instead I've been parsing the SNMP::MIB::NODE objects and converting them to a SQLite DB file. This is working very well.

      100 tests: T1 is using SNMP::MIB after calling SNMP::initMib(). T2 is using a SQLite DB in a squased FS. T3 is using the same file in ram

      Totals    : 685.29 74.02 74.42
Average T1: 6.8529
Average T2: 0.74
Average T3: 0.74

      [download]

      As you can see, using initMib and then converting 60k of SNMP objects to a DB file took me from aprox 7s down to 1s. Storeable may actually be slower.

[reply]
[d/l]
        I was looking at the tie'd interface and it might be worth trying to extend a STORE so that it also writes to a file or database. YMMV, but all of this tie'd stuff seems like terrible core for such a module.
[reply]

Back to Seekers of Perl Wisdom