There's Re: Simple primality testing which links to an older node prime factorization using base 1.

Then there's the amazing dc.sed script which implements arbitrary precision numeric calculations in decimal base in sed – you can find its source here or inside the tarball of gnu sed as a test. I don't really understand how it does the multiplication and division, but I could do the addittion on my own in a slightly more complicated way in Re^2: --- adding a column of integers (it's a loop of four substitutions, dc.sed has two and the teasing comment "could be done in one s/// if we could have >9 back-refs..." which I don't really believe).

How about fibonacci numbers? Some snippets like Re: Fibonacci golf with one state variable use regex substitutions but then they're not really using the power of the regex engine. There must be some way to actually use the regular expression engine to generate fibonacci numbers though. Searching yields fibonacci regex and Fibonacci Regex. These test for fibonacci numbers rather than generating them but there's probably some way to convert them. Another idea is to use something like this but probably there's some nicer way to phrase it:

perl -le'$==1,(1x$_)=~/(^)(1|11\1)*(?{$=++})^/,print$=for 0..20'
[download]

See also Explain one liner for fibonacci.

Update: Explain Fibonacci one-liner ?? too.

Update 2010-07-21: Try this for Catalan numbers:

perl -le'$==0,(1x$_)=~/(^)(|1(?2)(?2)(?(?{})))$(?{$=++})^/,print$=for 
+0..13'
[download]

Update 2010-11-21: A nicer version for the Catalan numbers:

perl -le'$==0,(1x$_)=~/^(|()1(?1)(?1)\2)$(?{$=++})^/,print$=for 0..13'
[download]