Coincidentally enough, I've been working on a lexer/parser combo (in C++) for one of my classes, so I guess I'll take a stab at answering this.

The lexer has to be smart enough to recognize quoted strings as individual elements, consequently it would return something along the lines of

('my', '$x', '=', 'my $x = \"my $x\""', ';')
[download]

Some lexers might even be smart enough to classify tokens, and return something like this:

([ 'MY' => 'my'], ['VAR' => '$x'], ['OPERATOR', '='], ['STRING' => 'my
+ $x = \"my $x\""'], [TERMINATOR => ';']).
[download]

To clarify, the lexer needs to understand the nature of the tokens that it generates, at a minimum knowing things such as which characters can be used in an identifier, and what the quoting rules are. (Quick, what does perl -e 'for qw(foo bar) { print $_ }' do?) The parser's knowledge relates to what order tokens can legally come in, and what different combinations of tokens mean.

If you're interested in reading more about specifically parsing perl, you should read merlyn's On Parsing Perl.

--
Ryan Koppenhaver, Aspiring Perl Hacker
"I ask for so little. Just fear me, love me, do as I say and I will be your slave."