...so, what do you preffer? many lines containing few information each, or less lines containing many information each?

To be able to give any useful advice, it would help to know the exact context that this program is going to be used in. For example, are you intending to call it once for every input item? How many input items are you planning to look up? Etc.

Also, as to speed, it's hard to make a precise prediction without knowing all the details (such as hardware being used, complexity of the conjugations, etc.). However, my gut feeling is that precomputing everything will be marginally faster, in particular if you need to compare more than one input item against the full set of words. Generally, if your machine has fast I/O, slurping in the whole precomputed dictionary might not be a problem, while, if you have a fast processor, computing the conjugations / inflections on-the-fly might still be faster after all...   If I were you, I would just benchmark the different approaches, and go with whatever turns out to be faster.

In case you need to do many lookups, it's probably a good idea to use a hash to store all the words (as keys).  If the words aren't unique, i.e. if conjugations of different base words may lead to the same final form (and if you don't want to lose the info where the final form originated from), you might want to accumulate that info in the hash elements' values... In addition to using a hash, it might help to read in the whole data structure once, and keep the process running persistently. I.e., you might consider using some client/server architecture.

Generally, the whole thing very much sounds like the problem a typical spell/grammar checker has to solve (if I understood your task correctly, that is). I'm personally no expert in such algorithms, but I'm sure many people have put much thought into optimising these things... In other words, that's where I would start looking for publications, etc. (e.g. scientific ones).   Good luck! And welcome here, BTW.

Using a hash will speed things up, but unless you are using some persistant environment (mod_perl etc), then reloading the hash everytime will still be quite slow.

If moving your dictionary into a database is not immediatly possible for any reason, then you can speed things up a lot by simply splitting it into several files based on the first letter, so that you have 26 files named dict_A.txt, dict_B.txt, etc. That would (on average) make your program take less than 5% of the time currently takes:

if (length($input)>0){
    print "<p><b>".$input."</b></p>";
    
    my $firstLetter = substr( $input, 0, 1 );

    open DICTE, "dict" . $firstLetter . ".txt" or die $!;

    while (<DICTE>){
        chomp;
        ($english, $lang, $irreg, $clss) = split(/;/,$_); 

        if ($input eq $lang){ 
            print "<p>$english - $lang, $clss</p>";
        } 
        if (conj("$lang;present;1;singular") eq $lang){ 
            print "<p>$english - $lang, $clss</p>";
        } 
        if (conj("$lang;present;2;singular") eq $lang){ 
            print "<p>$english - $lang, $clss</p>";
        } 

    }
}
[download]

And if that is still too slow, then you can divide the dictionary into smaller pieces by using the first two letters. dict_AA.txt, dict_AB.txt etc. which (on average) should reduce the search time to ~ 0.2% of the current.

Though there are probably few words begining with ZZ... or JJ... etc, so the speed up won't be exactly evenly distributed.

Moving your data into a simple tied DB like BerkeleyDB or similar, is a better long term solution, but this might help you out while you get to grips with Perl, hashes, modules and the rest.

If you were doing mutiple lookups per run of the program, then I would expect storing the full lexicon in a hash to help out significantly, since it would only need to read in the full file once and 8M isn't really all that much memory these days. If you're only doing one lookup per run, though, it will probably make things slower, since it would always need to read in the full file rather than stopping once it finds a match.

The earlier comment regarding spell/grammar checkers was spot-on. If you can find any information on how they function, it would probably be highly relevant to your problem.

For more general solutions, this seems to me like a database would be your best bet, whether a 'real' database (Postgres, MySQL, etc.) or just a tied/dbm hash.

If you really need to work directly off of a plain text file for some reason, you could index it to get at least some of the improvement that a database would bring: Sort the text file (it's probably already sorted, being a dictionary, but I mention it just to be sure) and then build a separate index file containing the offset in the dictionary for the first word beginning with each letter. By seeking to that position in the file before reading and processing lines and stopping when you hit a line that starts with a different letter, you can avoid searching through any words that start with the wrong letter, effectively reducing your dictionary size substantially.

Just to elaborate a bit on the hash approach, here's a very simple example of how you would populate the hash and then look up some value(s):

#!/usr/bin/perl

my %dict;  # the hash

while (my $line = <DATA>) {
    chomp $line;
    $dict{$line}++;  # instead of ++ you could also assign some value.
+..
}

my @inputs = qw(
foo
fooed
fooen
prefoo
postfoo
);

for my $input (@inputs) {
    print "found '$input' in lexicon\n" if exists $dict{$input};
}

# I'm using the special DATA filehandle here to be able to inline it..
+.
# That would be your DICTE handle supplying all the precomputed 385090
+ lines

__DATA__
foo
prefoo
bar
baz
...
[download]

BTW, am I understanding you correctly, that what you mean by 'analyse' is essentially to check whether some given $input is found in the lexicon?

the problem with this code is that, as the possible conjugated forms get larger, and also if its necessary to check for prefixes or suffixes, the analysis takes very long if the dictionary is too big (specifically, 16008 words).

You may wish to look into a concept called 'stemming'. Basically, the program reads in terms, it removes prefixes, suffixes, numeric quantifiers, etc, to get just the 'stem' or base of the word. You then compare just the base word, rather than all of the possible permutations.

This may also reduce the number of terms that you're tracking in your dictionary. As you're trying to be language agnostic, this may be more difficult, as the rules for stemming will be dependant upon the language. Also, English is a notoriously difficult language to do this in, as its terms come from multiple languages, and tend to keep the rules for the original language.

#!c:\perl\bin\perl
use Storable; #use this for calling the hash storing functions.

my %dict;  # the hash

while (my $line = <DATA>) {
    chomp $line;
    $dict{$line}++;  # instead of ++ you could also assign some value.
+..
}
store(\%dict, "hash.txt"); #store the hash in the file.

%dict = %{retrieve("hash.txt")}; #retrieve the hash from the file "has
+h.txt". thus, it only needs to be generated once.
my @inputs = qw(
foo
fooed
fooen
prefoo
postfoo
);

for my $input (@inputs) {
    print "found '$input' in lexicon\n" if exists $dict{$input};
}
[download]

just for informing: this message above by "Anonymous Monk" is ours, we just had forgotten to log in when we sent it.

#!c:/perl/bin/perl.exe
print "Content-type: text/html\n\n";
use BerkeleyDB;
   my $filename = "dict";
   my $db = new BerkeleyDB::Hash
                -Filename => $filename,
                -Flags    => DB_CREATE
        or die "Cannot open file $filename: $! $BerkeleyDB::Error\n";
[download]

$db->db_get("word", $lang); #the word is stored in the $lang variable.
[download]

Part of the utility of a database is that you can do updates to it, rather than regenerating it every time there's change. If your language changes, say from Spanish to Russian, then yes, you'd have to swap out the database. If you're working in one language, though, most of the words I would suspect stay the same, and you'll only be making additions and corrections over time.

I would suggest learning to leverage the advantages using databases gives you. Whether you stick with Berkeley or use something SQL-centered, you should be looking to do updates to the database instead of regenerating it on a regular basis.

db_load -c duplicates=1 -T -t hash -f dict.txt dict.db
[download]