Open Activity Monitor (in /Applications/Utilities), and go to the System Memory tab. There are a few things you should check here: the "free" memory, the "swap used" and the "page outs". If you have high free memory, low swap, and low page-outs, that means your system has plenty of RAM, and adding more will not help much. If the page-outs and swap used are high, then it's likely to help.
But more important than those numbers, which are hard to interpret well, is your experience. Have you noticed slowdowns in any particular cases? For example, when switching applications? When you have too little RAM, you'll often have excessive delays when switching apps, sometimes over a full minute in extreme cases (5-10 seconds is more typical, in my experience). With more RAM that delay can be reduced to nothing, so it's a huge difference.