Sorry, definitely DON'T store all your user fonts (non system fonts) there because your mac will want to load them all for every application. Defeats the purpose of using a font manager. :)
I mean to say put your MY FONTS folder directly in your hard drive but not in any other folders...