Was just taking a look at the robots.txt files of some major websites and came across this for ebay.co.uk:
Code:
### BEGIN FILE ###
#
# allow-all
#
#
# The use of robots or other automated means to access the eBay site
# without the express permission of eBay is strictly prohibited.
# Notwithstanding the foregoing, eBay may permit automated access to
# access certain eBay pages but soley for the limited purpose of
# including content in publicly available search engines. Any other
# use of robots or failure to obey the robots exclusion standards set
# forth at <http://www.robotstxt.org/ wc/ exclusion.html> is strictly
# prohibited.
# v3
#
User-agent: *
Disallow: /help/confidence/
Disallow: /help/policies/
Disallow: /disney/
### END FILE ###
The paragraph at the top explicitly says no to phorm but the machine readable part mostly says come on in.
Also on the subject of robots.txt - Googlebot's full user agent string is something like:
Code:
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
The interesting part is the url pointing to information about google bot and what it does. If phorm fake that then they are certainly committing some offense IMO.