Simple view on how proxies work (inc DNS)

[mr-b]

Originally Posted by danielf

Ah, that makes sense, as I was wondering if the proxy had to query for a more recent page, which would increase traffic and defeat the purpose of the proxy unless you only have to retrieve the time stamp of the most recent page.

I presume this expiry time/date is set differently depending on the type of page? (very low for a forum like this) and high(er) for less dynamic pages)?

Or is it just based on recent 'refresh rates'?

One important thing to note is that a web page is made up of several objects, each of which can have different cacheability HTTP headers:

- Expires (the best since they are explictly set by the content publisher)

- Cache-Control

- Last-Modified

So a forum page should have the frequently updated part as having either a very recently modified date, a short expiry time or else marked as non-cacheable and the relatively static parts (logos, banners etc.) as being older or having longer expiry times.
This will make the page much more responsive than making the whole page uncacheable. Note that these settings are what your local browser cache uses as well, so it's not just for the benefit of proxies).

In fact checking this page via the Cacheability Engine shows this to be true (takes some time to load):
http://www.web-caching.com/cgi-web-c...php%3Ft%3D9354

There's some more info about how this all works (without requiring knowledge of the RFCs) and how to optimise web servers at:
http://www.mnot.net/cache_docs/

Originally Posted by Chrysalis

well right now pem I am running through my own proxy on port 3333, but the ntl transperent proxy is still intercepting my pages, I will look for the post that explains this and paste it here.

Running a proxy on your own machine is a complete waste of time - it is certainly not going to bypass NTL's caches because as far as the above explanation goes, your proxy is still the local client.

[Florence]

The bottom line is these transparent proxies slow down your surfing as if they havent got the page you want it now has to look it up which you had already done on the proxy. In real time doubling the time till the page arrives if its one not stored.

If I have it wrong then I am willing to learn so come on techies teach me the ideas behind proxies and what benefit they give customers!

[danielf]

I presume (but I might be wrong), the reason for having proxies is not to speed up browsing but to reduce the bandwith used outside the NTL network, thereby reducing cost (and as a result reducing what we need to pay for our connection)?

[Chrysalis]

pem you might have misread my previous posts, the proxy is not on my local machine, it is on a remote box. It is a pass through proxy it doesnt cache pages.

this link no longer shows any ntl proxy servers.

http://romulas.zmnt.co.uk/forum/vars.asp

Originally Posted by Chrysalis

pem you might have misread my previous posts, the proxy is not on my local machine, it is on a remote box. It is a pass through proxy it doesnt cache pages.

this link no longer shows any ntl proxy servers.

http://romulas.zmnt.co.uk/forum/vars.asp

Fair enough, where is this box located then ?

[Chrysalis]

dedicated box I got in a datacentre in the us.

I got some minor issues with cookies and ssl at the moment needs some tweaking, but overall I am pleased with the improvements.

http://www.cisco.com/warp/public/732...witching/wccp/

hohum

[Florence]

Originally Posted by John Doe

http://www.cisco.com/warp/public/732...witching/wccp/

hohum

Thank you for the link unfortunately what that says isn't happening.. I was quicker when proxies was off that now on my general surfing I.E. when researching for college course visiting websites I have never been to before..

Also the one website I visit the most the forums I moderate is slower this is hosted in Germany. I have more page errors on proxies even though the address is correct to get to these pages I have to go via Google.

Infect when the proxies were turned off that was the best internet I had and the DNS propagated quicker. I have had to waste downloads as I have sometimes started a download and itâ₠¬ÃƒÂ¢Ã¢â‚¬Å¾Ã‚¢s stopped partway through and would not restart. Yet if I started again from it worked.

Suppose its the proxies but I find the information on that link very limited about proxies and not at all reasuring after all the problems with mail servers how can we be sure the proxies will stay up. The more different ways we have to ask to get to our final website the more bandwidth we end up using.... IMPO........

[mr-b]

Originally Posted by Kitty

The bottom line is these transparent proxies slow down your surfing as if they havent got the page you want it now has to look it up which you had already done on the proxy. In real time doubling the time till the page arrives if its one not stored.

If I have it wrong then I am willing to learn so come on techies teach me the ideas behind proxies and what benefit they give customers!

Not sure exactly what the above is saying, but if your question is about the latency that proxies add to internet requests, then (assuming that the proxy is working correctly, running within specification and there are no network problems) it is negligible compared to the response time via the internet and remote web server. Caches are designed to reduce the latency of internet requests as a whole.
A cache 'hit' (where it satisfies the request from local disk) should take tens of milliseconds (ms). A 'miss' (where it has to fetch the object from the remote web server via the internet) takes several hundreds to thousands of ms.

A slightly older doc on the problems of latency (but the principles still hold true today) is:
http://rescomp.stanford.edu/~cheshir...s/Latency.html

[Chrysalis]

well ntl rang me up today, concerning my proxy complaint, I ran some tests for him downloaded a file from download.com and it was fine. I told him of the hotmail problems and on the firefox site as well as my browsing slowdown, he told me what I believed to be true tho, these proxies were turned on after ntl decided nationwide they are all to be on, he said its ok for me to use my own proxy, so I will carry on using it.

[Florence]

Originally Posted by mr-b

Not sure exactly what the above is saying, but if your question is about the latency that proxies add to internet requests, then (assuming that the proxy is working correctly, running within specification and there are no network problems) it is negligible compared to the response time via the internet and remote web server. Caches are designed to reduce the latency of internet requests as a whole.
A cache 'hit' (where it satisfies the request from local disk) should take tens of milliseconds (ms). A 'miss' (where it has to fetch the object from the remote web server via the internet) takes several hundreds to thousands of ms.

A slightly older doc on the problems of latency (but the principles still hold true today) is:
http://rescomp.stanford.edu/~cheshir...s/Latency.html

Thank you for that link I will read it later tonight when I have peace. I must say that the proxies was working ok recently but last night they blew it big time. As soon as it was too late to call in the proxies for me died completely. Unable to access any website.. MSN, Yahoo, ICQ and AIM alsom MIRC chatroom all still worked but anything that needed the http was dead.... this lasted about 30 mins...

Need some sort of way to communicate these problems when faults is closed or 24 hour cover..

[SMHarman]

Originally Posted by danielf

Ah, that makes sense, as I was wondering if the proxy had to query for a more recent page, which would increase traffic and defeat the purpose of the proxy unless you only have to retrieve the time stamp of the most recent page.

I presume this expiry time/date is set differently depending on the type of page? (very low for a forum like this) and high(er) for less dynamic pages)?

Or is it just based on recent 'refresh rates'?

Http also has a HEAD command and a GET command. GET asks for the whole page, HEAD asks for the headers, which can be used to compare to the cached headers. Head uses a lot less bandwidth as a result.

[iadom]

Originally Posted by Chrysalis

http://forum.nthellworld.co.uk/showp...2&postcount=89

shows the info

but right after i retrieved this post the ip shown in my sig is no longer ntl proxy server.

I think it is working after all

The proxies have only just been turned on again in this area and I have just noticed that your sig is now giving out my proxy and not my true IP.

[Chrysalis]

yeah I got this sig from someone else I added it purely as a diagnostic to see if using a ntl proxy.