Re^2: HTTP request

Yes, i'm trying to figure out my own code for now. And seems i'v bit more than i can chew.

Anyways my efforts on trying to learn some socket programming through perl have been less than appreciated in my academy and I am forced to focus on other topics. Though im still gonna consult you people for advices under cover:].

Comment on Re^2: HTTP request

Replies are listed 'Best First'.
Re^3: HTTP request by marto (Cardinal) on Mar 14, 2011 at 12:23 UTC
As you've discovered there is more to a crawler than socket programming. You don't seem to understand the basics or appreciate what is involved in implementing something like this properly, respecting robots.txt (or even learning what robots.txt is), not hammering servers etc. I suggest you take the time to learn the basics, review the advice people have given to you on these topics, see Network Programming from the tutorials section.	[reply]

Replies are listed 'Best First'.

Re^3: HTTP request
by marto (Cardinal) on Mar 14, 2011 at 12:23 UTC

As you've discovered there is more to a crawler than socket programming. You don't seem to understand the basics or appreciate what is involved in implementing something like this properly, respecting robots.txt (or even learning what robots.txt is), not hammering servers etc.

I suggest you take the time to learn the basics, review the advice people have given to you on these topics, see Network Programming from the tutorials section.

[reply]