What you are talking about might be handled in one of two ways:

1. a web proxy, that intercepts the web request and makes it on your behalf, returning either the working page or the search results. As McDarren noted earlier, this could be accomplished via a Squid proxy, among other methods.
2. a DNS proxy, that intercepted the DNS query and made it on your behalf, returning the appropriate data if it existed, or the DNS information for the preferred search engine if such a site did not exist. Such a method would likely start with a look at the Net::DNS::Nameserver module.

Of the two options, option 1 seems the more accurate, as it can handle the situation of a missing page, whereas option 2 can only handle the case of a site that does not exist or is not responding. Consider the scenario in which you send requests to sites that do not exist over to a search engine found at a site called "www.mypreferredsearchengine.net", and a user attempting to go to a site called "www.mymissingsite.org", which does not exist.

Using option 1 (a web proxy), the following steps are likely to occur:

Using option 2 (a DNS proxy), the following steps are likely to occur:

I hope that helps clear up the difference in the scenarios. For further reading (and where I obtained some of the data I adapted for the examples above-although any errors in the scenario's adaptation are my own),

• Chapter 3: Learning HTTP, from "Web Client Programming with Perl: Automating Tasks on the Web" by Clinton Wong (O'Reilly and Associates, 1997)
• Section 5: HTTP/1.1: Request, from RFC 2616: Hypertext Transfer Protocol - HTTP/1.1

Update: 18 Jan 2006: Added readmore tags around process descriptions because of length.

Hi atcroft, Corion, ikegami, jesuashok, McDarren and All,

Thank you all for your kind wisdom :)

atcroft, thanks. I am much much more clear at where I am heading now.

I do agree with you all that the proxy way is the best way, but since the resources I've been given for this work is scarce (one IBM xseries 220 windows2k server with 256MB ram and 34GB scsi harddisk as a search portal + 2 existing DNS Sun Sparc stations), I think may be option 2 gives the IBM server less load. please correct me if I am wrong.

On the web server side, I guess I have the choice of either using the existing IIS 5.0 server or the Apache server for my portal. Do I have to code using an extension/API of some sort to achieve what I want to do?

Cheers and Best Regards,

Wilson

You are quite off-topic here, however....

If you are using squid to proxy all your web traffic, you can achieve what you want by customising the ERR_DNS_FAIL error file.

cheers,
Darren :)

Please note that not all DNS requests originate from a browser. You are mixing DNS and HTTP, so intercepting and changing DNS will only aggravate your non-browser users and won't produce the process flow you want, hijacking the users browser input and sending it off to a search engine.

I think the HTTP proxy idea is the best, turning all 4xx responses into search queries. But I think that this will confuse users much more than just setting their homepage to a search engine.

Sorry if this has been posted twice coz I'm not sure if it has been posted.

We need to approve a post before it becomes visible to everyone. You check it even before it has been approved by visiting your home node and clicking on "writeups". You won't see the other post now, since it's been deleted.

is it possible that we could intercept the invalid result from the DNS server, send the user typed url as keywords to a search engine, and display the search results to this user?

Firefox already does that, kinda. If you type in a bad URL, it does a google search and loads up the first match.