On Unix OSes like OS X, niceness referes to a process's priority. Processes with high niceness take a low priority, allowing other processes to do their work before they takes up CPU time. Processes with low niceness will hog the CPU whenever they need it, putting normal processes on hold.
By default, all apps have a neutral niceness level. Some background system processes are made nicer so that they won't bog down the system while you're trying to work, and instead will only make use of the CPU when you're idle.
The "% nice" item in Activity Monitor tells you the amount of CPU time used by processes with high niceness levels.
If you have high CPU usage, usually you'll suffer very poor performance if you try to do much until whatever is using the CPU finishes. But if a high percentage of the CPU usage is under "% nice", then you know that anything you do will take priority, so you'll get good performance.
