View Single Post
Old 31-07-2008, 21:53   #13025
SMHarman
Inactive
 
Join Date: Jun 2003
Services: Cablevision
Posts: 8,305
SMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronze
SMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronzeSMHarman is cast in bronze
Re: Virgin Media Phorm Webwise Adverts [Updated: See Post No. 1, 77, 102 & 797]

Quote:
Originally Posted by pseudonym View Post
As they will be caching robots.txt, wouldn't it be better to deliver phorm a massive list of disallow statments, including for paths that do not exist - storage may be cheap, but searching it would make a bit more work for Phorm's profilers every time a phorm victim requested a page from your site?
They are only taking the page that you look at not searching the entire site ala google. That would not slow them down.
SMHarman is offline