That's a favorites icon.
Make your icon in your favorite icon app., roughly 16x16 at 4bits. You can do up to 32x32 at 8 bit, and multi resolution icons, but I've had some roblems in the past with those types. 16x16 should be more than enough.
You have to save it as a .ico file with this name: favicon.ico and it has to be 1k or less in size. Now, just upload to your main (root) html directory, or to every directory (I did this and it seemed to eliminate displaying problems on pages in subdirectories). You're done, but you may have to clear your cache to see it the first time.
There's also a way to provide a link in the head portion of your HTML document if you wish (this allows you to put the .ico file in any directory with any name...
<link rel="shortcut icon" href="http://www.yoursite.com/icons/tk4two1.ico">