View Single Post
Old 02-04-2005, 18:58   #53
mr-b
Inactive
 
Join Date: Jan 2004
Location: Warwickshire
Posts: 53
mr-b is an unknown quantity at this point
Re: Poplar Proxies don't reach 3 meg speed

Maybe I should have said more specific in my last post...
Anyway I carried out some testing on the actual example that was given in the first post of this thread.

I downloaded this same 1.5MB object from www.supernews.com several times:
- via my local transparently-redirected group of proxies (Bham)
- via specified proxies, both local and remote in different areas.
- via my work's VPN connection via a VPN server in the US

I downloaded the object using the free GNU uility 'wget' on XP: http://gnu.paradoxical.co.uk/software/wget/wget.html

It is a simple command line 'web get' tool which should eradicate any browser discrepancies. It can specify a proxy, indicate the object's download speed, and display the http headers which are useful in determining whether it's the same object and whether it's been obtained from cache. (Unfortunately I did have some small background file transfers going on which might explain some variance in the results, but it wasn't very significant.)

Results (long)

The speed for the first object fetch was broadly similar (70-100KB/s) for all locations, irrespective of whether it was from transparently redirected caches or manually specified - and the same for local vs remote. It was even slower via my US VPN connection (which does have to encrypt/decrypt and encapsulate every packet).
Then for any subsequent hits from the same cache, the object downloaded at full speed (300KB/s).

So from this admittedly quick and dirty test I can't see anything much different about any of the proxies that I tried, and it appears that www.supernews.com is doing rate limitation for every connection (fairly understandable given the site) - unless anyone can download this object at a significantly faster rate (and show that is hasn't been cached).

Full results:

In the below http headers, note that if the 'Age:' header is >= a few seconds then the object is a cache hit i.e. it has been served from the cache.
(http://rfc.net/rfc2616.html#s14.6)

#1 Transparently redirected via Bham cache group (Miss)

Destination is www.supernews.com - the network latency via ping seem to be about 200ms (so it's a US site).

D:\temp\wget>ping www.supernews.com

Pinging www.supernews.net [216.168.3.20] with 32 bytes of data:

Reply from 216.168.3.20: bytes=32 time=175ms TTL=107
Reply from 216.168.3.20: bytes=32 time=177ms TTL=107
Reply from 216.168.3.20: bytes=32 time=269ms TTL=107
Reply from 216.168.3.20: bytes=32 time=191ms TTL=107

Ping statistics for 216.168.3.20:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 175ms, Maximum = 269ms, Average = 203ms

#1 First object fetch via local transparent proxies (Bham)

D:\temp\wget>wget http://www.supernews.com/stats/active.txt
--14:21:23-- http://www.supernews.com/stats/active.txt
=> `active.txt'
Resolving www.supernews.com... 216.168.3.20
Connecting to www.supernews.com[216.168.3.20]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1,442,929 [text/plain]

100%[====================================>] 1,442,929 94.32K/s ETA 00:00

14:21:40 (83.78 KB/s) - `active.txt' saved [1442929/1442929]

#2 Again (Hit)

D:\temp\wget>wget -S http://www.supernews.com/stats/active.txt
--14:26:20-- http://www.supernews.com/stats/active.txt
=> `active.txt.1'
Resolving www.supernews.com... 216.168.3.20
Connecting to www.supernews.com[216.168.3.20]:80... connected.
HTTP request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 297
3 Accept-Ranges: bytes
4 Date: Sat, 02 Apr 2005 13:21:23 GMT
5 Content-Length: 1442929
6 Content-Type: text/plain
7 Connection: keep-alive
8 Proxy-Connection: keep-alive
9 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
10 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
11 ETag: "cb9610-160471-424de09b"

100%[====================================>] 1,442,929 240.05K/s ETA 00:00

14:26:25 (277.44 KB/s) - `active.txt.1' saved [1442929/1442929]


#3 Again (Hit)

D:\temp\wget>wget -S http://www.supernews.com/stats/active.txt
--14:32:10-- http://www.supernews.com/stats/active.txt
=> `active.txt.3'
Resolving www.supernews.com... 216.168.3.20
Connecting to www.supernews.com[216.168.3.20]:80... connected.
HTTP request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 647
3 Accept-Ranges: bytes
4 Date: Sat, 02 Apr 2005 13:21:23 GMT
5 Content-Length: 1442929
6 Content-Type: text/plain
7 Connection: keep-alive
8 Proxy-Connection: keep-alive
9 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
10 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
11 ETag: "cb9610-160471-424de09b"

100%[====================================>] 1,442,929 313.85K/s ETA 00:00

14:32:15 (319.67 KB/s) - `active.txt.3' saved [1442929/1442929]

#4 Use Poplar-1 proxy (Miss)
>set http_proxy=popl-cache-1.server.ntli.net:8080

D:\temp\wget>wget -S -Yon http://www.supernews.com/stats/active.txt
--14:39:05-- http://www.supernews.com/stats/active.txt
=> `active.txt.4'
Resolving popl-cache-1.server.ntli.net... 62.255.64.4
Connecting to popl-cache-1.server.ntli.net[62.255.64.4]:8080... connected.
Proxy request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 0
3 Date: Sat, 02 Apr 2005 13:39:07 GMT
4 Content-Length: 1442929
5 Content-Type: text/plain
6 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
7 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
8 ETag: "cb9610-160471-424de09b"

100%[====================================>] 1,442,929 120.91K/s ETA 00:00

14:39:20 (104.51 KB/s) - `active.txt.4' saved [1442929/1442929]

#5 Same again (Hit)

D:\temp\wget>wget -S -Yon http://www.supernews.com/stats/active.txt
--14:39:57-- http://www.supernews.com/stats/active.txt
=> `active.txt.5'
Resolving popl-cache-1.server.ntli.net... 62.255.64.4
Connecting to popl-cache-1.server.ntli.net[62.255.64.4]:8080... connected.
Proxy request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 51
3 Accept-Ranges: bytes
4 Date: Sat, 02 Apr 2005 13:39:07 GMT
5 Content-Length: 1442929
6 Content-Type: text/plain
7 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
8 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
9 ETag: "cb9610-160471-424de09b"

100%[====================================>] 1,442,929 306.08K/s ETA 00:00

14:40:02 (306.26 KB/s) - `active.txt.5' saved [1442929/1442929]

#6 try from Poplar-cache-2 (Miss)

D:\temp\wget>wget -S -Yon http://www.supernews.com/stats/active.txt
--14:41:10-- http://www.supernews.com/stats/active.txt
=> `active.txt.6'
Resolving popl-cache-2.server.ntli.net... 62.255.64.5
Connecting to popl-cache-2.server.ntli.net[62.255.64.5]:8080... connected.
Proxy request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 0
3 Date: Sat, 02 Apr 2005 13:41:11 GMT
4 Content-Length: 1442929
5 Content-Type: text/plain
6 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
7 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
8 ETag: "cb9610-160471-424de09b"

100%[====================================>] 1,442,929 121.03K/s ETA 00:00

14:41:23 (116.68 KB/s) - `active.txt.6' saved [1442929/1442929]

#7 try from local Bham-5 cache (Miss):

D:\temp\wget>set http_proxy=brhm-cache-5.server.ntli.net:8080

D:\temp\wget>wget -S -Yon http://www.supernews.com/stats/active.txt
--14:42:33-- http://www.supernews.com/stats/active.txt
=> `active.txt.7'
Resolving brhm-cache-5.server.ntli.net... 62.255.32.13
Connecting to brhm-cache-5.server.ntli.net[62.255.32.13]:8080... connected.
Proxy request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 1
3 Date: Sat, 02 Apr 2005 13:42:34 GMT
4 Content-Length: 1442929
5 Content-Type: text/plain
6 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
7 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
8 ETag: "cb9610-160471-424de09b"

100%[====================================>] 1,442,929 71.01K/s ETA 00:00

14:42:51 (83.84 KB/s) - `active.txt.7' saved [1442929/1442929]

#8 local Bham-5 cache again (Hit)

D:\temp\wget>wget -S -Yon http://www.supernews.com/stats/active.txt
--14:47:37-- http://www.supernews.com/stats/active.txt
=> `active.txt.9'
Resolving brhm-cache-5.server.ntli.net... 62.255.32.13
Connecting to brhm-cache-5.server.ntli.net[62.255.32.13]:8080... connected.
Proxy request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 304
3 Accept-Ranges: bytes
4 Date: Sat, 02 Apr 2005 13:42:34 GMT
5 Content-Length: 1442929
6 Content-Type: text/plain
7 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
8 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
9 ETag: "cb9610-160471-424de09b"

100%[====================================>] 1,442,929 319.13K/s ETA 00:00

14:47:41 (320.54 KB/s) - `active.txt.9' saved [1442929/1442929]

#9 From Manc-2 (Miss)

D:\temp\wget>wget -S -Yon http://www.supernews.com/stats/active.txt
--14:44:39-- http://www.supernews.com/stats/active.txt
=> `active.txt.8'
Resolving manc-cache-2.server.ntli.net... 62.252.192.5
Connecting to manc-cache-2.server.ntli.net[62.252.192.5]:8080... connected.
Proxy request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 0
3 Date: Sat, 02 Apr 2005 13:44:41 GMT
4 Content-Length: 1442929
5 Content-Type: text/plain
6 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
7 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
8 ETag: "cb9610-160471-424de09b"

100%[====================================>] 1,442,929 104.05K/s ETA 00:00

14:44:54 (98.48 KB/s) - `active.txt.8' saved [1442929/1442929]

#10 local Bham-6 cache (Miss)

D:\temp\wget>wget -S -Yon http://www.supernews.com/stats/active.txt
--14:48:23-- http://www.supernews.com/stats/active.txt
=> `active.txt.10'
Resolving brhm-cache-6.server.ntli.net... 62.255.32.14
Connecting to brhm-cache-6.server.ntli.net[62.255.32.14]:8080... connected.
Proxy request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 1
3 Date: Sat, 02 Apr 2005 13:48:25 GMT
4 Content-Length: 1442929
5 Content-Type: text/plain
6 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
7 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
8 ETag: "cb9610-160471-424de09b"

100%[====================================>] 1,442,929 91.69K/s ETA 00:00

14:48:42 (79.62 KB/s) - `active.txt.10' saved [1442929/1442929]


#11 - via work VPN (US connection)

D:\temp\wget>wget -S -Yoff http://www.supernews.com/stats/active.txt
--14:50:26-- http://www.supernews.com/stats/active.txt
=> `active.txt.11'
Resolving www.supernews.com... 216.168.3.20
Connecting to www.supernews.com[216.168.3.20]:80... connected.
HTTP request sent, awaiting response...
1 HTTP/1.0 200 OK
2 Age: 0
3 Date: Sat, 02 Apr 2005 07:26:44 GMT
4 Content-Length: 1442929
5 Content-Type: text/plain
6 Connection: keep-alive
7 Proxy-Connection: keep-alive
8 Server: Apache/1.3.31 (Unix) DAV/1.0.3 mod_perl/1.29 mod_ssl/2.8.19 OpenSSL/0
.9.7d
9 Last-Modified: Sat, 02 Apr 2005 00:00:27 GMT
10 ETag: "cb9610-160471-424de09b"
11 Via: 1.1 corpse (xxxxxxxxx)

100%[====================================>] 1,442,929 52.42K/s ETA 00:00

14:50:51 (59.17 KB/s) - `active.txt.11' saved [1442929/1442929]
mr-b is offline   Reply With Quote