UPDATE2: I think there is indeed a need to syntactically distinguish the two forms of dereferences, maybe by obliging the programmer to enclose symbols in double quotes:

${"$sym"}
[download]

But this should be done equally for package and lexical vars, IMHO no need to break orthogonality.

UPDATE3: But anyway you're bringing a syntax-argument into a semantic-discussion.