Why the fixation on speed? Worry about how long it takes to implement. Only worry about execution time if it turns out to be a problem later.

For your second question, you can slurp the entire file into an array and then search the array, which should speed up the IO (which is easily the slowest part). Example:

open(FH, '<', 'somefile.txt') or die $!;
my @in = <FH>;
close(FH);

my $want_username = 'myusername';
my $hit;
while( (! $hit) && (my $entry = shift @in) ) {
    my $in_user = (split /:/, $entry)[1];
    $hit = ($in_user eq $username);
}
[download]

Update: Typo in variable name.

----
I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
-- Schemer

Note: All code is untested, unless otherwise stated

..also, with the code that hardburn generously provided, how would i put a simple display of 'username exists' or 'username not taken' in there?

This will work if the system is guaranteed to use /etc/passwd, or if it's your system and it does use /etc/passwd, but it's not portable. It won't work if the system uses NIS, LDAP, a database-backed PAM module, etc. A more portable way of doing the same thing is getpwent.

And with either of these techniques, which is faster is determined by the speed of the backend, the number of users, and how many you're looking up. For example, it's probably not worth reading a 50,000-user remote NIS database to find 50 users, but it may be worth reading a 50,000-user /etc/passwd for 50 users.

As usual, the best overall answer is try a few things and benchmark.

The general tone of responses is based on an educated guess of real needs judging from past experience with other people asking those questions.

As for the past experience with people trying to speed things up, well the following essay on code tuning is exactly accurate. Yes, there are times that you need to tune code. I have been there, and done that. But most of the people who think they need that are just shooting themselves in the foot. What they really need to do is learn to write clearer code, get it working, and then be able to step back and do the really valuable kinds of analysis on the overall project.

The image that comes to mind is someone who is so focussed on the need to be moving really fast that they are tripping over their own two feet. Get them to focus on effective walking motions, and they get farther immediately, plus are going to be in a better position to learn how to run later...

I can see your point Tilly, I guess I did not even think about that viewpoint when I posted because I have dealt with the exact same speed issue the person was posting about before. Where a batch check of many names to a silly large /etc/passwd file was taking way to long using getpwnam(). In my instance it changed my average runtime from 14 minutes down to <15 seconds to use my method instead of getpwnam()

BTW, it is great having you back! I missed your insightful posts while you were kept from us.

-Waswas

Hi There, I indeed do have a finished app, but am concerned about how long it is going to take for my script to detect if a user exists on the system when there are 500,000+ users on it.

On a seperate note, how long would you say that 500,000 lines of usernames in a file would take to search?

If it is a local password file you will run into about the same issue I described above where the search can take like 10 seconds per name with the std lib getpwnam. building a hash will take about that same amount of time but the time will be preloaded -- meaning getpwnam of 1000 users against a 500,000 user /etc/passwd file you get 1000 x N (N = number of secs to search the file) hash method you get N + (Y x 1000) where Y is the time it takes to lookup a hash entry in perl (not very long)

-Waswas