Hi - I have gotten two XServe G5's to install / configure and
experienced the same problems. I found this worked,
Steps:
1. Assuming you're on a network with a DHCP server,
set your DHCP server to assign an IP address to the
headless XServe via it's Ethernet ID for port 1 printed on the back,
and connect to the network to port 1.
2. Download OSXvnc.app to the desktop of a machine with a monitor
3. Open Terminal and copy the application to the XServe via scp, eg.
cd Desktop
scp -r OSXvnc.app root@ip-address-of-xserve:~/.
[password]
The password is the first 8 letters of the XServe's serial number,
or 12345678 for old XServes
4. log in to the server with ssh by typing from Terminal
ssh root@ip-address-of-xserve
[password]
5. Start the VNC server by running with these commands on the XServe
cd OSXvnc.app
./storepasswd MyPassword the_password_file
./OSXvnc-server -rfbauth the_password_file &
The server will print some status text and wait for connections.
6. Download a VNC Viewer, eg. I found "Chicken of the VNC" to work
and connect to the XServe at the IP address and with the password you stored.
You need your firewalls, etc. set up to allow you to connect to port 5900
on the XServe from the machine that's running the VNC Viewer.
Hope that's useful
Richard