in reply to Re: Re: Re: comma delimited, syslog parsing
in thread comma delimited, syslog parsing

Once the logs are generated, formatted and sent to that unix server my hands are washed clean of the process, and i have no reason to maintain a DB of these results. It is all being stored and maintained on the Unix server.

Ok - so, let's say something goes down at 8am and comes back up at 11am the same day. What is the process by which you update the relevant record on the Unix box? What is the protocol? How do you tell it "Update THIS event with THIS information."? Once you have that answer, you can answer your question.

I've got a feeling that it's going to (eventually) be something along these lines - you have an event with a given entity. You report to the Unix server "Entity ABCD had an event EFGH at such-and-such a time". It is up to the Unix server (who is the one with all the information) to correlate the various events for the entity ABCD. You should just be reporting "This entity, this event, this timestamp".

------
We are the carpenters and bricklayers of the Information Age.

The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6

Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

  • Comment on Re: Re: Re: Re: comma delimited, syslog parsing

Replies are listed 'Best First'.
Re: Re: Re: Re: Re: comma delimited, syslog parsing
by jeff061 (Initiate) on Oct 13, 2003 at 18:20 UTC
    Basically its like i said. In the example that you mentioned you would send something along the following lines:

    hostname, Oct 13 2003, 8:00:00, service is down, Error, Oct 13 2003, 12:00:00
    The first date and time being the downtime, the second being the uptime.

    Its all really static, i wish the setup on the Unix server was more capable. But you email that line(it can take multiple lines now, after i whined to the Unix guy) to the server. And once it is recieved it is saved to a file specific to that host name, in identical format. Periodically, on the Unix server, scripts are run to create reports on the downtime of each server, and the reason it was down(just the error).

    The Unix system does nothing beyond that, it does not correlate downtimes and uptimes beyond what i send it. I've fought to get this moved to the server to no avail.

      Ok. You need to create some datastore on your side. I would recommend some lite-weight database, like DBD::SQLite. You would then enter something for the downtime event. When you parse an uptime event, you will find the corresponding downtime event, delete it from your datastore, and send the full message to the Unix server. Something like:
      1. Parse the syslog file.
      2. For each event, do:
        1. If downtime event, enter event into local datastore.
        2. If uptime event, do:
          1. Find corresponding downtime event in local datastore.
          2. If it doesn't exist, error out.
          3. If it does, create string to send to Unix server.
          4. Send string to Unix server.
          5. Delete downtime event from local datastore.

      The reason you want to delete the matched downtime event is you want to trap a duplicate message for the uptime event. (It sounds stupid, but there's very little cost associated and you trap a potential issue.)

      As for how to do it ... knowing what you're trying to do is half the battle. Try your hand at coding up the above pseudo-code. Post here with any problems you run into.

      ------
      We are the carpenters and bricklayers of the Information Age.

      The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6

      ... strings and arrays will suffice. As they are easily available as native data types in any sane language, ... - blokhead, speaking on evolutionary algorithms

      Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

        Thanks for the help. Minor change in direction, but thats what i needed.