If you can't limit the speed, can you limit the number of physical connections?
I have noticed that Azureus is built in Java. Java is slow at the best of times, You might want to try BitTorrent++ to see if the same things happen?
Edit: Blimey... Just checked the screenshots... Looks erm.. comprehensive
