I would highly recommend turning off FileVault unless you store credit card numbers in your home folder or work for the government and have Top Secret clearance.
There's no need to use FileVault in a home or casual business environment unless people are out to get you -- it's overkill, plus I guarantee you will have problems with it in the future, possibly even including data loss.
While FileVault is a great feature and appeals nicely to the paranoid types, it's just not necessary: no one wants to read your resume or your recipies, they couldn't care less about your family photos nor your music. And, if those kinds of things fell into the "wrong hands," big deal -- hackers want the good stuff, not a family photo album and some Phil Collins MP3s.
On the other hand, if you do work in a sensitive environment (nuclear secrets, FBI, CIA, IRS, credit company) and have sensitive documents on your hard drive that could cause physical/monetary/mental damage if they fell into the wrong hands, by all means leave FileVault on. If you're a casual home user that stores a few business documents on their hard drive, turn it off -- it's more of a risk than a safety net.