Re: Accessing and storing information from another website's database

Most probably it doesn't really matter what you do in the browser window, just try and see what URL is used to retrieve the information and tamper with it to do the queries you want.

It should be something along the lines of

http://www.somewhere.com/something.asp?class=someClass&mode=Basic
[download]

. Now you can just fake this kind of request constructing it with LWP::UserAgent. In general, this is not hard.

Good luck, -gjb-

Comment on Re: Accessing and storing information from another website's database Download Code

Replies are listed 'Best First'.
Re: Re: Accessing and storing information from another website's database by sprakash (Acolyte) on Dec 23, 2002 at 13:39 UTC
gjb, Hi! Thanks for reply. I did try what you advise, but it seems that the website doesn't accept any direct URL ..... (1) it's HTTPS and (2) somehow its blocking for external refs to its information. Anyways, thanks for your help. Take care, Surya	[reply]
Re: Re: Re: Accessing and storing information from another website's database by hardburn (Abbot) on Dec 23, 2002 at 14:40 UTC
For blocking external refs, that is done by checking the HTTP Referer header. Try setting that portion to the header to whatever page the search normaly comes from. As for HTTPS, it looks like you'll need to do some tricks combining LWP::UserAgent and Crypt::SSLeay (I've never done HTTPS in perl, so I can't help you with the specifics).	[reply]