I think a simple switch as linked to by Graham is the quickest and easiest solution, as it's a cheap option with proper dedicated hardware to "split the signal".
I have seen ethernet splitters, but from what I understand, they're not really designed, and don't work too well for what you want.
i.e. I think they're good if you've got your wall socket in centre of room, computer A in 1 corner, computer B in another corner so you want to plug both in permanently but you'll never have them both switched on at same time, as the splitter has no intelligence to direct the signal properly, so I think you'd have problems with both machines on at once. A switch knows where to send what info.
I may have oversimplified that a bit too much, or got it a bit wrong, so some-one please correct me if I'm wrong, I just know switches and routers are best tool for job as they're intelligent, routers more so than switches, as they can also connect to internet
