Hi there!
I am also trying to setup dialing into a Mac OS X server machine.
Here is what I've figured out:
In the /dev directory there are 2 devices related to the internal modem: tty.modem and cu.modem.
tty.modem is for incoming calls
cu.modem is for outgoing calls
To verify the...