On Ubuntu-18.04.5 the manual (man which) states:

DESCRIPTION
which returns the pathnames of the files (or links) which would be exe
+�-
cuted in the current environment, had its arguments been given as  com
+-
mands  in a strictly POSIX-conformant shell.  It does this by searchin
+g
the PATH for executable files matching the names of the  arguments.  I
+t
does not canonicalize path names.
[download]

I assume that I have a POSIX-conformant shell:

$ echo $SHELL
/bin/bash
[download]

I don't know what "It does not canonicalize path names" means. Is that relevant to this issue ?

Pretty clearly, the manual tells me that which perl returns the path of the perl that will be executed when, in my bash shell, I enter a command beginning with the word "perl".
Just as clearly, that assertion is false in my particular situation.
Here's a demo, starting with a freshly opened bash shell:

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/game
+s:/usr/local/games:/snap/bin

$ which perl
/usr/bin/perl

$ perl -le 'print $];'
5.026001

$ /usr/bin/perl -le 'print $];'
5.026001

# Select another perl, and hide the
# perl executable by renaming it
$ mv ~/perl528-d/bin/perl ~/perl528-d/bin/perl_hide

# Prepend the location of that
# "hidden" perl to $PATH
$ export PATH=~/perl528-d/bin:$PATH

# That change to $PATH clears the cache,
# so run a perl command that will again
# set the cached perl to /usr/bin/perl.
# The problem does not arise if this 
# step is omitted. 
$ perl -le 'print $];'
5.026001

# Make the hidden perl visible again
$ mv ~/perl528-d/bin/perl_hide ~/perl528-d/bin/perl

$ which perl
/home/sisyphus/perl528-d/bin/perl

$ /home/sisyphus/perl528-d/bin/perl -le 'print $];'
5.028000

$ perl -le 'print $];'
5.026001
[download]

The only thing happening that's "wrong" is that final output. Everything else is sane and as expected.

Of course, in my original case, I didn't run a command aimed at caching perl to /usr/bin/perl. (I just ran perl -V, expecting to see the configuration of the newly installed 5.33.1 - and getting the configuration of the system perl instead.)
And I didn't begin by renaming the "perl" executable to something else.
It was installed already named as something else (namely "perl5.33.1").
All I did was to rename "perl5.33.1" to "perl" - which would have been sufficient if "perl" was not by then cached as "/usr/bin/perl"

I do think the "which" documentation is way too glib.
Sure, it specifies the way that "which" determines the pathname of its argument(s), but it should not be unconditionally asserting that this will match the pathname used when that argument is given as a command in a strictly POSIX-conformant shell. That is (demonstrably) not always true - and the documentation should at least point this out.

Cheers,
Rob

PS
Following on from where I left off the above one liners, and just to demonstrate clearly that a change to $PATH is enough to reset the cache:

$ perl -le 'print $];'
5.026001
$ export PATH=~/nonexistent:$PATH
$ perl -le 'print $];'
5.028000
[download]

But I doubt that needed to be demonstrated, anyway ;-)