Re: Virgin Media Phorm Webwise Adverts [Updated: See Post No. 1, 77, 102 & 797]
Quote:
Originally Posted by pseudonym
As they will be caching robots.txt, wouldn't it be better to deliver phorm a massive list of disallow statments, including for paths that do not exist - storage may be cheap, but searching it would make a bit more work for Phorm's profilers every time a phorm victim requested a page from your site?
|
They are only taking the page that you look at not searching the entire site ala google. That would not slow them down.
|