That's because the original sign on the telephone was NOT a # but ⌗ (Unicode: U+2317 viewdata square). Often inclined it looks like a rhombus ⌗ °

The # is only a replacement symbol, which made it to the computer keyboard because of it's American use for numbering things.

The official German name I use for # is Doppelkreuz (literally "double cross")

Doppelkreuz is also a name for other symbols like the Patriarchal_cross but that's hardly relevant in an IT context.

°) look at that pic of an old phone, to discover a rhombus with overshots at the corners https://commons.m.wikimedia.org/wiki/File:AT%26T_push_button_telephone_western_electric_model_2500_dmg_black.jpg