Cable Forum - View Single Post - NTL proxies alter download file size if it's too large

Legorol · 13-06-2006, 18:09

This is an issue that came up for me as I was trying to download the just-released public beta of Microsoft Windows Vista 64-bit version. It appears that NTL's proxy servers mess up the Content-Length field of an HTTP response if it is too large to be represented in 32-bits. In layman's terms: if you are trying to download a file that is over about 4 gigabytes, the downloaded file's size (and therefore the download itself) gets messed up. This affects you even if you use a download manager.

Here follows a description of the issue and how I found out about it.

Symptoms: when I attempted to download the single gigantic ISO image from MS's official download site, then rather than receiving a file that is 4 Gb in size, I ended up with 13.7 Mb. No matter what browser or download manager I tried.

I have done some investigating by capturing network traffic to and from MS's download server, both when going through NTL's proxy as well as a 3rd party proxy server. The result was that when going through the 3rd party proxy, the Content-Length field returned from the web server is 4309368832 bytes (0x100DBC000 in hexadecimal), which is a number that is a little bit over what can be represented in 32 bits. When going through NTL's proxy, the Content-Length field returned is 14401536 bytes (0x00DBC000 in hexadecimal, missing the leading 1). Hence my download ends up at 13.7 Mb.

In other words, it appears that at the moment NTL's proxies are unable to handle a file download that is larger than 4 gigabytes. I have alerted NTL to the problem and i was promised that their server support team will look into this issue.

The real problem with this is NTL's use of transparent proxies. Even if you are using a download manager program, as long as you are downloading something from a web server via HTTP, you are affected by this issue. If you dont' have access to a 3rd party proxy server that allows you to use large amounts of bandwidth, you are in trouble.

13-06-2006, 18:09	#1
Legorol Inactive Join Date: Jun 2006 Posts: 5	NTL proxies alter download file size if it's too large This is an issue that came up for me as I was trying to download the just-released public beta of Microsoft Windows Vista 64-bit version. It appears that NTL's proxy servers mess up the Content-Length field of an HTTP response if it is too large to be represented in 32-bits. In layman's terms: if you are trying to download a file that is over about 4 gigabytes, the downloaded file's size (and therefore the download itself) gets messed up. This affects you even if you use a download manager. Here follows a description of the issue and how I found out about it. Symptoms: when I attempted to download the single gigantic ISO image from MS's official download site, then rather than receiving a file that is 4 Gb in size, I ended up with 13.7 Mb. No matter what browser or download manager I tried. I have done some investigating by capturing network traffic to and from MS's download server, both when going through NTL's proxy as well as a 3rd party proxy server. The result was that when going through the 3rd party proxy, the Content-Length field returned from the web server is 4309368832 bytes (0x100DBC000 in hexadecimal), which is a number that is a little bit over what can be represented in 32 bits. When going through NTL's proxy, the Content-Length field returned is 14401536 bytes (0x00DBC000 in hexadecimal, missing the leading 1). Hence my download ends up at 13.7 Mb. In other words, it appears that at the moment NTL's proxies are unable to handle a file download that is larger than 4 gigabytes. I have alerted NTL to the problem and i was promised that their server support team will look into this issue. The real problem with this is NTL's use of transparent proxies. Even if you are using a download manager program, as long as you are downloading something from a web server via HTTP, you are affected by this issue. If you dont' have access to a 3rd party proxy server that allows you to use large amounts of bandwidth, you are in trouble.