in reply to Re: Is PERL right for parsing logs
in thread Is PERL right for parsing logs

It's long, but roughly looks like this:
I am interested in keying on "Thread_ID:XXX" 

2006-03-08-19:27:59.844+00:00I----- thread(168) trace.pdweb.debug:2 /p
+roject/amweb410/build/amweb410/src/wand/wand/log.c:253: 
----------------- Browser ===> PD -----------------
Thread_ID:148
POST /fedtradeqa/FedTradeWeb/FedTradeTraderDispatcherServlet?873 HTTP/
+1.1

user-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Mozilla
+/4.0 (compatible; MSIE 6.0; Windows NT 4.0; -D-H1-MS318089))

Cookie: PD-S-SESSION-ID=2_EjfxinmS3wkiJDKN2ZPmz1xaSYY4s-ttf8FBJvSXALiH
+v1S6

Request-GUID: 6e9c2c1b0c3ef35b:208b20d4:109db19f683:-7c98

Host: fedtrade.frb.org

Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2

Connection: keep-alive

Content-type: application/x-www-form-urlencoded

Content-Length: 663


---------------------------------------------------


2006-03-08-19:27:59.861+00:00I----- thread(168) trace.pdweb.debug:2 /p
+roject/amweb410/build/amweb410/src/wand/wand/log.c:253: 
----------------- PD ===> BackEnd -----------------
Thread_ID:148
POST /FedTradeWeb/FedTradeTraderDispatcherServlet?873 HTTP/1.1

e-mail: NOT_FOUND

aba-num: NOT_FOUND

via: HTTP/1.1 p0p489:443

request-guid: 6e9c2c1b0c3ef35b:208b20d4:109db19f683:-7c98

host: 172.30.122.202:3443

user-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Mozilla
+/4.0 (compatible; MSIE 6.0; Windows NT 4.0; -D-H1-MS318089))

content-type: application/x-www-form-urlencoded

authorization: ****************

content-length: 663

iv-groups: "starrwi_GPI","FedTrade_Trader_GQI"

name-last: Joedecke

name-first: Christine

accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2

login-name: b1cxjmg

institution-name: New York / New York

connection: close

iv-user: b1cxjmg

iv-remote-address: 172.30.122.196



---------------------------------------------------


2006-03-08-19:27:59.933+00:00I----- thread(181) trace.pdweb.debug:2 /p
+roject/amweb410/build/amweb410/src/wand/wand/log.c:253: -------------
+---- Browser ===> PD -----------------
Thread_ID:211
GET /

---------------------------------------------------


2006-03-08-19:27:59.943+00:00I----- thread(168) trace.pdweb.debug:2 /p
+roject/amweb410/build/amweb410/src/wand/wand/log.c:253: -------------
+---- PD <=== BackEnd -----------------
Thread_ID:148
HTTP/1.1 200 OK
request-guid: 6e9c2c1b0c3ef35b:208b20d4:109db19f683:-7c98
content-type: application/octet-stream
cache-control: no-cache="set-cookie,set-cookie2"
date: Wed, 08 Mar 2006 19:27:59 GMT
content-language: en-US
transfer-encoding: chunked
server: IBM_HTTP_Server/2.0.47.1-PQ94086 Apache/2.0.47 (Unix) DAV/2
connection: close
expires: Thu, 01 Dec 1994 16:00:00 GMT
---------------------------------------------------


2006-03-08-19:27:59.943+00:00I----- thread(168) trace.pdweb.debug:2 /p
+roject/amweb410/build/amweb410/src/wand/wand/log.c:253: -------------
+---- Browser <=== PD -----------------
Thread_ID:148
HTTP/1.1 200 OK

request-guid: 6e9c2c1b0c3ef35b:208b20d4:109db19f683:-7c98

content-type: application/octet-stream

cache-control: no-cache="set-cookie,set-cookie2"

date: Wed, 08 Mar 2006 19:27:59 GMT

content-language: en-US

connection: close

expires: Thu, 01 Dec 1994 16:00:00 GMT

Set-Cookie: LtpaToken=fZADztK8UKrD9mST6zCU4UTV2wR/ekUiDaSz3N8F7ordI5Bs
+rz2/Gv9pAm3qvt3/V0rRPCeR3A7Xo0qZ/wDk9VAyLMdVOvgPnMSxcQ64734yZZ5SW3yjy
+kpd8HxTjGFZPc3E8A4cieXIsgAABYQZ+IUcJqDj+6UP4t8UATI7VKm/989Lcl4d0wGolJ
+NgsjtPxIUTh9WbfgT4tYHUPgMUb2JumIJLOGI2CjnQIJJuYDPX0Tg6H8T3N2vnmnTz3Sb
+CmOp8zvAZRRCYI7tCoKomhLgGEayMdZ4i6LFr624CJa4C/KjSSlcdO1pVrSuQhY1YlKWC
+jDzEuCAN/GX/GvdpUw==; Path=/fedtradeqa/

Set-Cookie: PD-S-SESSION-ID=2_EjfxinmS3wkiJDKN2ZPmz1xaSYY4s-ttf8FBJvSX
+ALiHv1S6; Path=/; Secure



---------------------------------------------------


2006-03-08-19:27:59.998+00:00I----- thread(26) trace.pdweb.debug:2 /pr
+oject/amweb410/build/amweb410/src/wand/wand/log.c:253: --------------
+--- Browser ===> PD -----------------
Thread_ID:137
POST /fedtradeqa/FedTradeWeb/FedTradeTraderDispatcherServlet?664 HTTP/
+1.1

user-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Mozilla
+/4.0 (compatible; MSIE 6.0; Windows NT 4.0; -D-H1-MS318089))

Cookie: PD-S-SESSION-ID=2_9dBTinQh4+rQaLrhQwkK+opYt-ap3QXJVhVCkPgsC2se
+pXJ8

Request-GUID: f621dbbb1b2e8502:21d5b7ea:109db272287:-7d69

Host: fedtrade.frb.org

Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2

Connection: keep-alive

Content-type: application/x-www-form-urlencoded

Content-Length: 311


---------------------------------------------------


2006-03-08-19:28:00.011+00:00I----- thread(157) trace.pdweb.debug:2 /p
+roject/amweb410/build/amweb410/src/wand/wand/log.c:253: -------------
+---- Browser ===> PD -----------------
Thread_ID:89
POST /fedtradeqa/FedTradeWeb/FedTradeTraderDispatcherServlet?272 HTTP/
+1.1

user-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Mozilla
+/4.0 (compatible; MSIE 6.0; Windows NT 4.0; -D-H1-MS318089))

Cookie: PD-S-SESSION-ID=2_MPhP64vidkNHce1zvETkKqHP9F4KDD3Ln9OqlV3DdPLI
+zsLl

Request-GUID: 79cc4c61386630b2:-34f1cd9a:109db47de3e:-7ef1

Host: fedtrade.frb.org

Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2

Connection: keep-alive

Content-type: application/x-www-form-urlencoded

Content-Length: 314

[download]

---------------------------------------------------

Each entry is seperated by "----"

20060322 Janitored by Corion: Put data in between <code>...</code> tags

Edited by planetscape - added readmore tags

Replies are listed 'Best First'.
Re^3: Is Perl right for parsing logs
by jdporter (Paladin) on Mar 22, 2006 at 14:26 UTC

    Surely Perl is well suited for such a parsing task. But I have some questions first.

    Some of that data seems broken, i.e. inconsistently formatted. In the first two records, the 

    ----------------- Browser ===> PD -----------------

    [download]
    and 
    ----------------- PD ===> BackEnd -----------------

    [download]
    strings are on their own lines (the second line of the record); in the subsequent records, those strings are at the end of the first line of the record. I'm also suspicious of all those empty lines within each record. Perhaps you made a copy-and-paste error?

    We're building the house of the future together.
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom