Quote:
Originally Posted by rryles
With regards to user agents and robots.txt it is important to note that there are 3 user agent strings involved!
|
And this is where BT/HO idea of implied consent for web sites breaks down.
Quote:
|
1. The user agent string sent in headers for normal page requests (likely to be unaltered from the users)
|
In which case there's no implied consent for cookie authenticated sessions and no way for web masters without access to the robots.txt to opt out of Phorms profiler.
Quote:
|
2. The user agent string sent in the headers for the request of robots.txt (no idea what this will be)
|
And until we're told what this will be, there's effectively no opt-out for site operators most of whom want publicly accessible pages to be indexed by google. Few want a 3rd party performing unauthorized interception.
Quote:
|
3. The user agent (if any) used to match lines in the robots.txt file. (Maybe googlebot but who knows??)
|
I'm pretty sure Google would action that.
BT's hand waving about opting out is exactly that, those already in contact should start pressing them on it
