Yup, you can do it with just Fink. Basically, there are at least four or five ways to get XFree86.
Probably the easiest way is to get it through the MacGimp project. The real problem with that is that noone outside of the MacGimp project knows what version of XFree86 it is (all we know is that it wasn't a stable release), and they sure aren't telling.
You can also get it through XTools. That costs money, though.
You can get it through the XonX project. I got it this way once, so I know it works.
You can compile it yourself, like you were trying to do. If I didn't have Fink, that'd be the option I'd pick.
Finally, you can get Fink. If you ever in the smidgen think that you'd like to look at some other open-source software out there later, I'd say go this route. Like I said, once you've installed Fink, the world is your oyster. It gets the source code for you, compiles it, and installs it for you. The added bonus is that if you're dealing with a lot of programs, all you need to do is update Fink, and the new version of Fink can check to see if you're up-to-date. Honestly, I'd say check it out.