I'd go along with that train of thought too.
I had a Belkin router too, and it kept cutting out to both wired and wireless computers just as you described. One way to tell if it's definitely the router (although it not doing it doesn't mean the router is innocent) is check the colour of the lights on the router when it cuts out. If it's green that's good, if it's orange mans your router is definitely up the creek!
Hope that helps.