If you installed a hard drive that came from a Windows PC, then that needs to be formatted, and have some version of a Mac operating system installed.
Those disks that you have labeled for PowerBook G4, or iBook G4, might work to boot your PowerMac, but you can't do much else, as those are the grey restore disks that came with specific Macs. You can't install anything from those disks to your PowerMac G4.
The flashing ? that you see, means that the Mac can't find a bootable operating system. Your hard drive has not been formatted for your Mac, and no Mac operating system, so the ? is normal.
Make sure that hard drive that you installed has the jumpers set for a master drive.
You _should_ be able to boot and install OS 9.
Here's how you can try...
insert the OS 9 CD in your drive.
Restart, while holding the letter C on your keyboard.
If it boots to the OS 9 CD, go into the Utilities folder, run the Drive Setup, and try to initialize the hard drive. If that is successful, you should then be able to install OS 9.
You will need to find a Mac OS X installer disk to install OS X. The best OS X version that you should look for is 10.4 (Tiger), but 10.3 will be OK, too, if you can find that.
That installer disk will NOT be grey, but will be black, with a big X across the label.