Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi, Newbie question. I'm writing a script to slot a few thousand records into a MIT kerberos database using the kadmin.local utility. I was pondering simply using system() to make the calls to kadmin.local but this would result in opening serveral thousand system calls, a bit of a waste of resources. Any suggestions as to do this more efficently?

Replies are listed 'Best First'.
Re: Just how efficent is using system() ?
by halley (Prior) on Jul 02, 2003 at 20:19 UTC
    If this is a one-time scratch task, and it will finish in mortal time, and not an every-five-minute cron task, then who cares? If the system is in heavy load production, sprinkle sleep() calls in the main loop. Once it's done, you can move on with your life.

    If it's a common task, or it exceeds rational usage limits, then it's time to see what tools may be better than the existing command-line resources.

    Do a super-search on this site for the phrase "premature optimization" and you might get some related wisdom.

    --
    [ e d @ h a l l e y . c c ]

[reply]
Re: Just how efficent is using system() ?
by grinder (Bishop) on Jul 02, 2003 at 20:50 UTC

    halley speaks the truth. Who cares?

    But your question is "how efficient is using system()". The answer is "not very". The kernel, at least in Unix land, has to allocate all sorts of structures to take the current process in order to fork it and take the child process and replace it with the process you want to run. This takes a non-trivial amount of time, at least as far as a CPU is concerned.

    In Win32 land the cost is similar; CreateProcess (or whatever it's called these days, I haven't touched the Windows API in 10 years) is not something you want to do in a tight inner loop.

    Anyway, you can consider that the upfront cost of performing a system call to be in the range of several hundred milliseconds. The second time you call it, the OS cache has been warmed and most of what you need is in RAM, so the subsequent cost drops by maybe an order of magnitude, so you're looking at possibly sub-100 millisecond start up time.

    What does this mean? Even if, in the worst case, the overall length of a single process run is one second, inserting "several thousand" records is going to take between one, two hours or a bit more. Do a run with the first ten records, ensure that it works, then fire up the rest and go out to lunch. While you're enjoying yourself, the computer is slowly grinding through the workload. Where's the wastage in that? Better than rendering a screen saver in any event.

    The other thing that springs to mind is that if you explain your problem in a bit more detail, some monk is going to pipe up and say "Oh, you just need to install Kerberos::MIT::Bulkloader and you can feed it a comma-delimited file and you're done." But without more information we just can't be sure.

    _____________________________________________
    Come to YAPC::Europe 2003 in Paris, 23-25 July 2003.

[reply]

      Google for "kerberos database perl" and you get this where Dave Steiner proposes a module for parsing a Kerberos database.
      This might do what you want....

      --
      Anthony Staines
[reply]
Re: Just how efficent is using system() ?
by bobn (Chaplain) on Jul 02, 2003 at 20:25 UTC
    I agree w/ halley, with additional thought that any tools that exist will have to do most of the same work in the system that a script would.

    If you use the system('program','parm1','parm2',), you also avoid starting a shell in the call.

    --Bob Niederman, http://bob-n.com
[reply]
[d/l]
Re: Just how efficent is using system() ?
by bunnyman (Hermit) on Jul 02, 2003 at 20:46 UTC

    Both good points above (don't optimize things that are not too slow, don't call the shell).

    I can suggest two possible ways to make this faster. First, if you can get the kadmin program to insert all the records at once, perhaps by reading the records from stdin, then you can just call system() one time (or open() with the pipe feature to feed data into the stdin). If kadmin does not have a way to do that, then you'll be stuck with calling system() many times.

    Second, you could find a way to avoid using kadmin at all and just connect to the database directly in your script.

    Either of these approaches will make it more efficient, but if neither one is possible, then there is no magic way to make system() faster.

[reply]
Re: Just how efficent is using system() ?
by Anonymous Monk on Jul 03, 2003 at 03:19 UTC
    Thanks for the insights guys. Much appreciated. I originally was approaching the problem by making use of the Authen::Krb5:Admin module which looked to have everything I was looking for. However I had difficulty getting it to pass its "make test" on the sparc box and linux box I tried installing it on. Forcing an install yielded very limited functionality and I put it aside for the sake of expendiency. May take a crack at using the Dice::Kadm5 module to do the job when time allows. Kerberos::MIT::Bulkloader sounds like a good tool for my initial loading of info, but it might not be suitable for doing small ongoing updates to the database. I'll be sure to check it out. If anyone has had success with a particular perl module for doing the day-to-day care and feeding of an MIT kerberos KDC I'd love to hear about it.
[reply]
Re: Just how efficent is using system() ?
by fglock (Vicar) on Jul 02, 2003 at 21:03 UTC

    You can make your Perl script use less resources (CPU) by calling it with nice:

    nice -n 19 perl myscript.pl args...
[reply]
[d/l]
[select]

      How does running something more slowly make it run faster?

      nice gives your procecss a low priority, so that it obtains CPU time at a slower rate, so long as higher priority tasks are available to run.

      --
      TTTATCGGTCGTTATATAGATGTTTGCA

[reply]

        That's right. I didn't mean it would run faster. I said it would use less resources.

[reply]

Back to Seekers of Perl Wisdom