VoIP can be affected by the smallest of issues, possibly additional noise on the line related to weather conditions or exchange VP capacity issues.
You can check the capacity in the exchange by typing the PSTN number into
http://usertools.plus.net/exchanges/.
This will show you if the VP is highly contended.
Issues could also be caused by other non-ADSL equipment attached to the line (i.e. Fax and redcare system)
As for the VPN access, if you have a windows XP machine on the LAN side of the router you could use a VPN pass-through to that PC and temporally terminate your VPN there. Or even better set yourself up a free account at
http://www.logmein.com and make the PC available for remote access.