Do you have any other computer that is also accessing your internet connection to see if it is having problems as well? Is the problem consistent after some time has passed or does it happen intermittenty? I suspect that there might be an issue with your internet connection from your ISP. It's happened to me with Comcast where sometimes the internet connection stalls for a while for reasons unknown.
You could also try and empty the cache of both Safari and Firefox to see if maybe that resolves some problems.
• Apple iMac G5 17" (2 GHz G5) - Mac OS X 10.5.8/Ubuntu 10.04
• Asus Eee PC 901 (1.6 GHz Atom N270) - Fedora 13
• Apple Macintosh Quadra 650 (33 MHz MC68040) - Mac OS 8.1
• "JHVH-1" (2 GHz AMD Athlon XP 2400+) - Slackware 13.1