The system has cache files, as well, and trashing these sometimes will help the speed of the system, but not to any real perceivable degree, unless you have something wrong with your system.
I would use Cocktail, like bobw does -- that'll get all of them.
I wouldn't do it but once every month or so, they don't hurt the system, slow it down, or take up much room unless your hard drive is already cramped for space.