Replacing the hard drive on the newest iMacG5 is not simple at all. Open the case, remove the display panel, make sure to transfer the temp sensor to your replacement drive. Got your credit card handy? (it's one of the tools used to open the case ) The drives that Apple uses for replacement have the temp sensor already attached. I would not recommend anyone tackle trying to replace the hard drive without good directions first.
As fryke says, replacing memory is simple. Open the small panel on the computer bottom, and the memory slot is right there.