I've found that when writing complex regexs it helps to use /x and to say exactly what you want. This may be slightly incorrect, I don't have a perl intrepeter to check the syntax of everything with, nor do I have any of my regex refrences, since I'm at work, but here it is:

/
    (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) # IP Address
    \s+
    (\S+)                # ident user
    \s+
    (\S+)                # auth user
    \s+
    \[            # Square brace
        ([^\]]+)    # date
        \s        # space
        ([^\]]+)    # time zone
    \]            # The closing square brace
    \s+
        "            # Opening quote
            ((?:[^\\]
            |        # method
            \\.)+)
        \s
            ((?:[^\\]
            |        # url
            \\.)+)
        \s
            ((?:[^\\]
            |        # protocol
            \\.)+)
        "            # Closing quote
    \s+
    (\d+)                # status
    \s+
    (\d+)                # bytes
    \s+
        "
            ((?:[^\\]
            |        # refer
            \\.)+)
        "
    \s+
        "
            (?:[^\\]
            |        # platform
            \\.)+
        \s
            (?:[^\\]
            |        # extended info
            \\.)+
        "
/x
[download]

Good luck. I'm not sure you really want a regex, other people have given pretty good ideas. I just figured I could throw in a more complete regex. It's probably a lot more sparce than necessary.
-Ted