As stated above, you're good to go as long as max % prime != 0:

use strict;
use warnings;

my ($seed, $prime, $max, $input, $result);

$seed = 8;
for $prime (2,3,5,7,11,13,17,19,23) {
    for $max (20..30) {
        next if $max % $prime == 0;
        my %hash;
        for $input (0..($max-1)) {
            $result = ($prime * $input + $seed) % $max;
            $hash{$result}++;
        }
        print "$prime $max : ";
        print join '', values %hash, "\n";
    }
}
[download]

All you need to guarantee this is a sufficiently large prime so that it's always going to be larger than sqrt(max) (but not equal to max itself).

use strict;
use warnings;

my (@arr, $seed, $max, $prime, $input, $result);

@arr = 1..10;
$seed = 5;

$max = $#arr+1;
$prime = 65521;

for $input (0..($max-1)) {
    $result = ($prime * $input + $seed) % $max;
    print $arr[$result], " ";
}
[download]

The hard part though, since I assume you're doing this in assembly or a low level language, will be making sure that none of the basic mathematical operations go out of bounds.