an idea for VM on traffic management

[Chrysalis]

As I predicted I think VM have dug themselves a hole.

previously the situation was the majority of UBR ports were not congested and as such gave people good performance. So the only complaints were price related or from congested areas.

Now we in a situation where in these area's people are been shaped when isn't actually needed, and I would imagine VM's complaints could well have quadrupled overnight. It previously had a medal for resisting traffic shaping which it has now lost.

On the other side of the coin the shaping is probably ineffective in the worst areas (not doing enough due to simply not enough capacity). So on top of low speed complaints they now gettng complaints in relation to shaping.

Of course finally even to customers who dont bother with speedtest's and downloading they may find their game has broke eg. WoW due to traffic shaping false positives and another source of new complaints from previously happy customers.

It is clear using a one size fits all approach to congestion is not a good idea for VM. I still have entanet's ALT system in my head and I would suggest this.

1 - scrap STM in its current form. Similiar to the shaping either not doing enough or throttling when not needed.
2 - scrap the shaping.
3 - add a global anti packet loss script to each UBR so it reserves unused capacity to maintain jitter, so if eg. 80% upstream utilisation causes jitter then cap entire utilisation to that level.
4 - add a script that throttles users based on following rules.
(a) check if anti packet loss tool is in use, ie. limit hit.
(b) look for top 5% of users on daily usage (last 24 hours, not from start of day), and decrease available throughput BOTH ways by 25%, this is regardless of protocol used.
(c) if limit still been hit thottle the same users by a 2nd 25% to make 50%, and repeat to 75% if required.
(d) if limit still been breached repeat for next 5%.
(e) run it 24/7 not just on peak.

This should be more effective for congested areas (throughput low anyway), and not kick in for good areas and as such keeping customers happier.

Interested in thoughts.

[AbyssUnderground]

Interesting idea, but think of the cost of implementing it...

[Chrysalis]

probably less complex than traffic shaping configuration.

[KangoChad]

I have no idea on how much this would cost VM but it's one of the best posts I have read on this forum about the new 'policy'.

It makes sense to temporarily punish the heavy users, not everyone!

Well done and hopefully they could change and implement this system - if they don't change, I'm off

[Ignitionnet]

I think I've discussed things like this a few times in the past and pointed to a system implementing something similar.

Something similar was deployed by Comcast in the United States, using a combination of in-house developed software and Sandvine hardware.

To accomplish this on the VM network would require considerable integration work, it would be a long way from being a trivial exercise.

The current system is largely out of the box, as is its' configuration looking at the balls up that has been made of it

It could be fine if implemented properly, which it has not been. Shaping should be monitored very closely during implementation. This looks like it was configured by an IP engineer or Sysadmin who went on a couple of courses and knows how to configure the stuff but has no idea why he's configuring it that way as he's not an actual application network engineer and is not familiar with the applications he's trying to throttle.

If he were people wouldn't be able to dodge shaping by changing port they run applications on.

Pretty weak to be honest and not a patch on Plusnet's intricate, generally accurate and well controlled systems.

[Chrysalis]

yep that looks pretty much like what I said, very similiar.

it throttles based on utilisation level of the ubr port and how heavy the user is compared to average.

really VM should have done the same as comcast then.

[Jayster]

The question is will they be bothered to sort it out? Or if they do decide to will it be to late?

[pabscars]

Originally Posted by Ignitionnet

I think I've discussed things like this a few times in the past and pointed to a system implementing something similar.

Something similar was deployed by Comcast in the United States, using a combination of in-house developed software and Sandvine hardware.

To accomplish this on the VM network would require considerable integration work, it would be a long way from being a trivial exercise.

The current system is largely out of the box, as is its' configuration looking at the balls up that has been made of it

It could be fine if implemented properly, which it has not been. Shaping should be monitored very closely during implementation. This looks like it was configured by an IP engineer or Sysadmin who went on a couple of courses and knows how to configure the stuff but has no idea why he's configuring it that way as he's not an actual application network engineer and is not familiar with the applications he's trying to throttle.

If he were people wouldn't be able to dodge shaping by changing port they run applications on.

Pretty weak to be honest and not a patch on Plusnet's intricate, generally accurate and well controlled systems.

Sounds like they could do with the help of someone in the know ,

re Plusnet; you didnt work on it did ya by any chance

VM do have a history of rushing things out half arsed and I see there traffic shaping is no exception.

[Chrysalis]

plusnet's system is finely tuned but its worth noting it took them a while to get to where they are now, it was a mess for quite a few months. I do commend them tho that they are much more open with their customers, eg. I read that their shaping equipment itself hit saturation, the gig port on an ellacoya got maxed out and this was told in public and they split the load with a new ellacoya to fix it.

Generally tho I think traffic shaping is more trouble then its worth, things can and will go wrong since its reliant on heuristics that are ever changing, the hardware itself can cause latency and slow downs due to processing loads and of course it can be abused changing ports etc.

What comcast use may be hard to integrate but generally is far less complex and wont have heuristics that keep need updating and heavy users wont be able to evade it by masking their traffic.

[Chrysalis]

Now for my 2nd idea. Probably much more complex than the first.

VM create virtual docsis interface ran in software.

So a real doscis3 upstream and downstream 4 channels bonded on each so 4xdocsis2 72mbit upstream port. Plus 4 down ports for 200+mbit downstream.

Within this 200mbit down and 72mbit upstream up capacity create virtual docsis interfaces that customers connect to, these virtual interfaces been smaller. This software would allow fine tuning where specific customers connect so although the same hard connection limitations exist (individual customers cannot be moved) they can be moved around to different virtual ports, and as such VM can do things like seperate serial users in an area from normal users.. This sort of thing is becoming more common in server hosting where a physical hardware kit will host many virtual servers.

The virtual docsis can be 1.x, 2.x or 3.x so to maintain compatability with customer equipment.

Yes I can vouch for PN's traffic management system but it's a shame it doesn't flex itself if the network is quiet, i.e. relax the throttles - saying that, peer2peer is untouched now all day until around 6-7pm which is better than what they advertise, now they've actually took the plunge and bought some decent chunks of bandwidth and have some spare

[Ignitionnet]

Originally Posted by Chrysalis

Now for my 2nd idea. Probably much more complex than the first.

VM create virtual docsis interface ran in software.

So a real doscis3 upstream and downstream 4 channels bonded on each so 4xdocsis2 72mbit upstream port. Plus 4 down ports for 200+mbit downstream.

Within this 200mbit down and 72mbit upstream up capacity create virtual docsis interfaces that customers connect to, these virtual interfaces been smaller. This software would allow fine tuning where specific customers connect so although the same hard connection limitations exist (individual customers cannot be moved) they can be moved around to different virtual ports, and as such VM can do things like seperate serial users in an area from normal users.. This sort of thing is becoming more common in server hosting where a physical hardware kit will host many virtual servers.

The virtual docsis can be 1.x, 2.x or 3.x so to maintain compatability with customer equipment.

It's not complicated particularly but it's a duplication of existing DOCSIS capabilities. Virgin could produce bad boy pipes right now if they chose to without messing with the DOCSIS segementation or reducing the bandwidth available to other customers.

[Chrysalis]

Originally Posted by Ignitionnet

It's not complicated particularly but it's a duplication of existing DOCSIS capabilities. Virgin could produce bad boy pipes right now if they chose to without messing with the DOCSIS segementation or reducing the bandwidth available to other customers.

Oh dear, not sure if I wanted to hear that. Because now I know VM can do it then it says to me they dont consider high jitter etc. a problem. It is a sad thing that the same problem exists on VM network now that I suffered from 5 years ago. The incapability to keep jitter low and performance high in high takeup/usage areas and now you tell me they already can move off the heavy users. (before you told me they couldnt? when I asked if could be split from them).

here is how well the current traffic shaping and STM is working.

Pinging bbc.co.uk [212.58.224.138] with 32 bytes of data:
Reply from 212.58.224.138: bytes=32 time=38ms TTL=119
Reply from 212.58.224.138: bytes=32 time=17ms TTL=119
Reply from 212.58.224.138: bytes=32 time=30ms TTL=119
Reply from 212.58.224.138: bytes=32 time=72ms TTL=119

Ping statistics for 212.58.224.138:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 17ms, Maximum = 72ms, Average = 39ms

although VM can do bad boy pipes now, I guess my 2nd idea has the advantage the overall shared capacity is larger so lower jitter on high utilisation. If VM could get 36mbit docsis2 going then it would be over 140mbit of upstream which is much harder to jitter than 9mbit. I tested on a 10mbit lan connection, I only had to utilise about 60% to get jitter, but on 100mbit I could hit almost 98%,

[Ignitionnet]

Originally Posted by Chrysalis

Oh dear, not sure if I wanted to hear that. Because now I know VM can do it then it says to me they dont consider high jitter etc. a problem. It is a sad thing that the same problem exists on VM network now that I suffered from 5 years ago. The incapability to keep jitter low and performance high in high takeup/usage areas and now you tell me they already can move off the heavy users. (before you told me they couldnt? when I asked if could be split from them).

Nah I didn't say they could move them off, they could just change use the QoS mechanisms within DOCSIS 1.1/2.0/3.0 to deprioritise their traffic. This is the mechanism Comcast use to enforce their FairShare system.

The downside of course being games don't like that, nor do streaming services. People who have been deprioritised don't get to send or transmit unless there's no traffic of a higher priority waiting.

---------- Post added at 18:26 ---------- Previous post was at 18:23 ----------

Originally Posted by Chrysalis

although VM can do bad boy pipes now, I guess my 2nd idea has the advantage the overall shared capacity is larger so lower jitter on high utilisation. If VM could get 36mbit docsis2 going then it would be over 140mbit of upstream which is much harder to jitter than 9mbit. I tested on a 10mbit lan connection, I only had to utilise about 60% to get jitter, but on 100mbit I could hit almost 98%,

It's 26.4Mbit per upstream, and sadly DOCSIS isn't Ethernet. Regardless of how wide the pipe is once it gets so full jitter will arise. The bonus is that more people need to be uploading at once to fill a wider pipe. Statistical contention is a good thing.

---------- Post added at 18:36 ---------- Previous post was at 18:26 ----------

I love Comcast's openness on this matter.

http://www.isoc.org/isoc/conferences...dth_woundy.pdf

http://customer.comcast.com/Pages/FA...ork-Management

EDIT: Ah also for more serious technical information - http://downloads.comcast.net/docs/At..._Practices.pdf

[Ignitionnet]

You did give me an idea Chrysalis, I've written about the Comcast solution in the official VM forums to guage responses there and had a think about how difficult it would be to implement.

The costs in terms of hardware wouldn't actually be that high, development would be the rub.