Re: Performance issues

The script sa-learn (which your above bash script calls) is written in Perl and is part of the spamassassin distribution. You won't achieve anything by converting your little bash script to Perl. You should simply "nice" the sa-learn process and let your operating system do the rest.

cat spamlearn_old.sh
#!/bin/sh
for f in `ls /var/lib/amavis/virusmails/spam*.gz`; do
  echo Learn Spam-Mails from File $f ...
    gzip -cd $f | sudo -u amavis -H nice sa-learn --spam --showdots;
    done
[download]

All dogma is stupid.

Comment on Re: Performance issues Download Code

Replies are listed 'Best First'.
Re^2: Performance issues by matze77 (Friar) on Nov 28, 2008 at 16:21 UTC
Thanks. I tried already to start it nice (e.g. reniced after i noticed) it, it wont help much it is very "consuming" anyway. As a practice i wanted to convert the bash script and insert the "sleep things" combined with watching CPU load factor, that i cant convert sa-learn i know, but maybe something could be won by making the whole thing a bit more "intelligent" . I forgot to mention that i tried nice earlier. Sorry if i didnt describe my goals accurately, didnt give all infos. So i give "breathe beyond nice" a try.	[reply]
Re^3: Performance issues by tirwhan (Abbot) on Nov 28, 2008 at 16:42 UTC
I tried already to start it nice (e.g. reniced after i noticed) it, it wont help much it is very "consuming" anyway. Then you're either a.) on a crap operating system or b.) not CPU-bound. I'd guess b.), especially with sa-learn and a huge bayesian database it's quite likely that you're either running out of RAM (and thus starting to swap) or just using up all the systems IO. Try doing a system call trace on the process (`strace` on Linux), also see what your general resource usage ist (`vmstat`). Stopping and restarting the process is an extremely crude tool and likely to do more harm and good IMO (especially since AFAICR sa-learn actually locks the bayesian database while inserting, which means spamassassin has to wait for the process to restart if you stop and start when the lock is being held). All dogma is stupid.	[reply] [d/l] [select]