Quote:
Originally Posted by jelv
I now have absolute confirmation that it is googlebot and googlebot only that Phorm looks for in robots.txt:
Email from me to Emma Sanderson of BT:
Reply from Emma Sanderson:
|
There is still wriggle room. She's said "because with the exception of msn no OTHER bots are allowed". She hasn't said - "the only thing we look at is whether googlebot is allowed".
It's like trying to wrestle with a mud covered eel.