Quote:
Originally Posted by jelv
I now know the user agent that Phorm is looking for in robots.txt and its time to get Google's opinion methinks. I have received confirmation from BT that Phorm is explicitly looking for the Googlebot's permission in robots.txt - not any search engine:
I posed the following question in an email to Emma Sanderson:
(Note I whoopsied and got the name of the Google agent wrong)
This evening I have received a reply which included the following:
The fact that they have corrected it to Googlebot confirms that it is specifically looking for the Googlebot permission.
|
Perhaps aconcerted effort to Google about this might get them on our side and in the fight.
If I remember correctly when I first built a website I had to apply for google bot to visit which would give them consent then. So google have conscent from me due to me applying.