I think youll find if its on your property you can disconect the other conections

i wud assume people have hooked themselves up and helped them selves, so maybe you might find there are using a starview box etc to gain free tv, As the black tape bodge job defo inst a virgin media instalation lol.
Personally i would cut all other people off that line and make sure that yourself and Next door if he is using it, that you both have a conection. sod everyone else.
That will be thier problem and Vm@S problem to give them a proper instalation
I bet then youll find most of your probs dissappear.
What you also might find is that with so many people on the BB freq are conflicting with each other, resulting in major packet loss, or as ur experiencing very slow data trasnfers.
Disconect them

is ya best bet.