It sounds like you dragged the disk image of FireFox into your applications folder.
In order to install Firefox correctly, do this:
1) Download FireFox.
2) Double-click the downloadede FireFox disk image.
3) Another FireFox icon should appear on your desktop -- this is the mounted FireFox disk image, containing the FireFox application. Also, a window may (or may not) automatically appear as well.
4) Double-click the FireFox mounted disk image, which will open a window displaying the FireFox application icon (or, if this window has already appeared, skip to 5).
5) Drag the FireFox application icon from within this window to your Applications folder.
Next, "eject" the disk image of FireFox by dragging the mounted FireFox disk image to your trash icon in the dock. The trash icon should be an "eject" symbol if you do this correctly.
After that, drag the downloaded FireFox disk image to the trash. The trash icon should be the normal trash basket icon if you do this correctly.
Optionally empty the trash after this last step.