I'm not sure if this conversion actually happens

Yes, it does. A more precise wording of what happens is: "The Perl literal 0xFF is converted to the numerical value 255 by the parser before oct ever sees it."

oct, requiring a string, converts the numerical value 255 to the string value 255. Doing it's job, oct then converts the string value 255 to the numerical value 173.