Go FISH?

[Nicholas Leippe]

Rather than do a full remote desktop, you can avoid installing the X-server by 
simply installing the client xlib libraries, and then whatever X11 apps you 
want, and send the display to any X server you want.  There are X11 servers 
available for both mac and windows as well.

The client and server were recently (a year or so?) split into separate 
packages to make this easier.  Dunno about other distros, but this is at 
least done for gentoo. (I'm speaking of xorg's X11, not xfree86.)

Nick