Search Engine Bot Verification Woes
Tuesday, November 6th, 2007Along with many of you, I have read several articles about "proxy hacking" and other issues related to the need to verify the source of user agents purporting to be a known search engine spider. For those of you unfamiliar with such problems, "proxy hacking" involves routing a bot through a proxy server in order to get to your site. Page ranking associated with your site could then be transferred to the URL of the proxy server. Other issues involve browsers or bots masquerading as a known search engine spider for any purpose.
After reading some of these articles, I decided to do something to protect my site. What is there to fear? Proxy hacking can be a legitimate concern. As for the matter of simple masquerading; if you enter my establishment while flashing a fake I.D., it makes me think you’re up to something.
