gabjones has asked for the wisdom of the Perl Monks concerning the following question:

Hi people, My apologies on the somewhat novice question. Please bear with me. We regularly get snmp data from our routers the format is below.
col-1 col-2 col-3 col-4 col-5 col-6 col-7 col-8
23/10/2003 00:00 00:04:19 302 0.26 1.18 3617.13 657.42 2959.71 29
23/10/2003 00:00 00:24:30 315 0.67 0.86 3804.99 1665.32 2139.67 28
col-1 - Sample Time
col-2 - "Delta Time"
col-3 - "Bandwidth Utilization In %"
col-4 - "Bandwidth Utilization Out %"
col-5 - "Bytes/sec Util"
col-6 - "Bytes In" "/sec"
col-7 - "Bytes Out" "/sec"
col-8 - latency
********************************
I currently use the following commands to generate the gnuplot graps of the columns im intrested in. I put this series of commands in a executable file called 'stats' and run it by doing ./stats
**********COMMAND***************
awk '{ print $1, $4, $5 }' RTP*Theale*B* | sed 's/"/ /g' | sed 's/,/ /g' > theale-B; awk '{ print $1, $2 }' theale-B > data-in; awk '{ print $1, $3 }' theale-B > data-out; ./graph

RTP*Theale*B* - file that contains snmp data

theale-B - file to pipe into

data-in and data-out - made the file name generic to save me having to manaully input all the time

graph(Chart:Graph module) - executable that plots graph taken in the data in/out has input

But each time I run this stats script(dont know wether to call it a script or not) I have to manually input the file(RTP*Theale*B*) and the file to pipe into (theale-B).
Is it possible to modify this script to ask for input for the file names without having to open it with vi and modify.
My apologies again but I am quite desperate for help. Any help or advice will be greatly appreciated.
regards,
gab
  • Comment on modifying script to ask for file name as input

Replies are listed 'Best First'.
Re: modifying script to ask for file name as input
by Roger (Parson) on Dec 04, 2003 at 23:17 UTC
    Well, you came to the right place. ;-)

    Perl is excellent at doing this kind of stuff. I have written the following script that does what you wanted. (I could write a perl one-liner-ish that does the job, but I felt a complete program is probably more useful to you as 1) a tutorial on file I/O, command-line parsing. 2) a fully working utility.)

    #!/use/local/bin/perl -w use strict; use IO::File; use Getopt::Long; # Parse command line arguments and assign corresponding variables GetOptions ( 'i|input=s' => \( my $INPUT = undef ), 'o|output=s' => \( my $OUTPUT = "./data-in,./data-out" ), 'g|graph=s' => \( my $GRAPH = "./graph" ), ); unless ( defined $INPUT ) { print <<USAGE SNMP data parser Usage: $0 [option] Options: -i|--input [filename] SNMP router data file -o|--output [in,out] Optional. Default is "datain,dataout". -g|--graph [command] Optional. Default is "./graph" USAGE ; exit(1); } # get the names of the datain and dataout files # note that you can override these names from # the command-line my ($datain,$dataout) = split /,\s*/, $OUTPUT; my $f = new IO::File $INPUT, "r" or die "Could not open input file $INPUT"; my $di = new IO::File $datain, "w" or die "Could not create output file $datain"; my $do = new IO::File $dataout, "w" or die "Could not create output file $dataout"; while (<$f>) { # for each line in the input file next if /^\s*$/; # ignore empty lines my @col = split /,\s+/; # split on , followed by spaces my ($date) = $col[0] =~ /^"(.*)\s.*"$/; # want date only print $di "$date, $col[2]\n"; print $do "$date, $col[3]\n"; } undef $di; # force close of files undef $do; undef $f; # do the graph system($GRAPH) or die "Could not execute $GRAPH"; exit(0); __END__ example: ./script.pl -i snmp.log
    And the output from sample data are -
    data-in ------- 23/10/2003, 1.86 23/10/2003, 2.26 23/10/2003, 0.74 23/10/2003, 1.15 23/10/2003, 1.61 23/10/2003, 1.55 23/10/2003, 1.11 data-out -------- 23/10/2003, 1.91 23/10/2003, 1.53 23/10/2003, 0.46 23/10/2003, 1.53 23/10/2003, 0.72 23/10/2003, 1.08 23/10/2003, 0.52
      Hi Guys,
      Thank you very much for the inputs.
      To Roger:
      The script seems to be doing everything apart from when the ./graph script wants to then graph the data in/out it will error.
      I had this problem that was why I piped the snmp data file to the commands | sed 's/"/ /g' | sed 's/,/ /g' to remove the ' " '
      The problem is it has to be formatted by removing the ' " '
      "23/10/2003 0.85,
      "23/10/2003 0.85,
      I tried to edit the script my self to no avail.Can you modify the script to do this.
      Thanks for the wonderful tutorial.
      Can you also put comments on your script so i can learn and use it to write other similar scripts.
      Thanks reagards,
      gab
        Can you post a fragment of the snmp log file here? So we can have a look at its actual format.

        I have modified my script to remove " and , characters to my best guess since I have not seen the actual data file. And I have added more comments in the code.

Re: modifying script to ask for file name as input
by VSarkiss (Monsignor) on Dec 04, 2003 at 23:26 UTC

    Is it possible to modify this script to ask for input ...
    Yes

    But I think the better question is:

    Is it possible to rewrite this script to simplify my life?
    The answer to which is also Yes.

    Here's your problem, as I see it:

    1. You have an input file whose name changes every time.
    2. You want to extract some columns from it.
    3. You need to reformat the data so your other software can process it.
    So far, you're using awk and sed and the shell to do it, but your suspicions are correct: you can do it all in one Perl program. Here's one possibility:
    open GRAPHIN, '>', 'data-in' or die "Can't open data-in: $!\n"; open GRAPHOUT, '>', data-out' or die "Can't open data-out: $!\n"; while (<>) # You can pass the input file name on the command line { my @f = split /\s/; # Like awk: split at white space print GRAPHIN $f[0], ' ', $f[3], "\n"; print GRAPHOUT $f[0], ' ', $f[4], "\n"; } close GRAPHIN; # Could live without these close GRAPHOUT; system './graph'; # Execute your graph program
    And you're ready to go. Note that awk counts fields starting at 1, and Perl arrays start from 0.

    This is not tested code, it's a guess at what you're trying to do. Unfortunately, my ESP is a little rusty, so I may have misread what you're trying to do. In any case, HTH.

Re: modifying script to ask for file name as input
by waswas-fng (Curate) on Dec 04, 2003 at 22:54 UTC
    Does awkmonks.com redirect to perlmonks? =) To answer your question you can rewrite in perl to do that very easily, or you can escape the awk vars and put the command in a shell script using non escaped $1 and $2 for the two input filesnames. and run like script_name file1 file2


    -Waswas