Were I the ISP I would probably encrypt by taking the
password, the name, and some sort of secret key, then
taking the MD5 hash of that and truncating to the length I
want. This is a well-known and simple procedure.
That procedure has the following properties:
You can verify a login.
The key stored does not help anyone find the password.
The keys generated for one user give no information on
whether anyone else has the same password.
Without the secret key there is no possible
dictionary attack. Therefore the ISP is in a position
to track any brute force attacks.
Now I cannot prove that the ISP's programmers settled on
the solution that I would. But I guarantee you that if
they did then there is no way you are going to break their
encryption mechanism...