There are several problems with this proposal:
1) X11 is not just a graphics library
2) If I were porting an X11 app, the first thing I would do is throw away all the X11 code even if I could retain it. 90% of X11 code is drawing stuff which is taken care of by various Views like tables, text, images, buttons, sliders, etc. The exceptions to this rule are mostly cheesy 2D games not using SDL.
3) There is absolutely no point to porting an app to OS X if you're not going to change the interface. Writing a library allowing X11 programmers to recompile their aps will result in X11 apps. X11 code results in X11 apps. You may as well run rootless X11.
4) X11 apps generally suck ass and aren't worth porting. Those which do not are for the most part commercial and ought to be ported properly.
>80 column hard wrapped e-mail is not a sign of intelligent