That drive icon on your Desktop is the mounted disk image file. When you opened the disk image (ends in .DMG), you created the little drive on your Desktop because it's mounted as if it were a disk. You can eject it by pressing Command+E, or dragging it to the trash, and then you can delete the disk image file you downloaded.
All that said, you need to open that drive, just like you would your hard drive on your desktop, and you will find the Firefox application icon in there. Drag that file into your Applications folder. You cannot (and should never) drag the disk image into a folder on your drive, because it will eject when you log out or restart your computer (or when you eject it).
By the way, that disk image mounted because Safari mounts them automatically. Firefox will not, and, when you download disk images, you will need to double-click them, or click them and press Command+O to open/mount them.