http://www.apachefriends.org/en/ have Windows/Linux/Mac Apache,PHP,MySQL installs which make it all a breeze, there is also information regarding making your site secure if you scroll down the download screen or visit their forum (English section is near the bottom of the forum list).
If you download the Windows installation file (as opposed to the zips) set the install directory to Drive:\ [space] you need to add a space for the install button to become active.
No directory changes are made to your system so you can just delete all the files to get rid.
The Windows version comes with the batch files to run Apache and MySQL as a program or a service, plus batch files to switch between PHP4 and 5.
PHP 5 installs by default.
The German guy and girl who put these compilations together are really dedicated, always including the latest versions.
