Still not sure whats the best way to do this....
But I have got some great results :
What I have done so far is to convert videos to flv(riva flv) and play them with this :
http://flowplayer.sourceforge.net/
Streaming of the file is done using a PHP script and so far so good.
Weakest link is 'flow player' which seems to like firefox but not IE.