How are you testing the speed of the router? What other applications are running on the machines it serves? What machine and OS are you using when connecting direct to the internet?
Have you taken a look at
this forum and
this other forum?
I use the older WRT54G (equivilant of the WRT54GL you have) with DD-WRT and have no problems at all - it runs as fast as my net connection will let it.