GNU wget: what exactly is it?

T

themacko

Barking at the moon.
Greetings UNIX folk! I've been told (by a couple people now) that I need to use wget when downloading stuff off the net. I've been lead to this site http://www.gnu.org/software/wget/wget.html but I'm still not sure exactly what this software does nor do I know how to set it up.

Would someone mind filling me in on this thing? I'm quite interested as I've never really used the UNIX part of OS X .. but I'm a total newb so talk to me like a 6-year-old.

I appreciate your help, in advance. I know the last thing most of you want to do is explain to a newb how to use the Terminal. ;)

By the way, I'm using OS 10.3 if that makes any difference.
 
themacko said:
Greetings UNIX folk! I've been told (by a couple people now) that I need to use wget when downloading stuff off the net. I've been lead to this site http://www.gnu.org/software/wget/wget.html but I'm still not sure exactly what this software does nor do I know how to set it up.

Would someone mind filling me in on this thing? I'm quite interested as I've never really used the UNIX part of OS X .. but I'm a total newb so talk to me like a 6-year-old.

I appreciate your help, in advance. I know the last thing most of you want to do is explain to a newb how to use the Terminal. ;)

By the way, I'm using OS 10.3 if that makes any difference.
Click to expand...

wget is simple application that grabs items off the web and saves them to files. It can also be used to copy the entire contents of a website. I don't know why someone would say you "need to use wget". Anything you can get using wget you can get using a web browser, unless you're looking to recursively download an entire site.

Brian
 
btoneill said:
wget is simple application that grabs items off the web and saves them to files. It can also be used to copy the entire contents of a website. I don't know why someone would say you "need to use wget". Anything you can get using wget you can get using a web browser, unless you're looking to recursively download an entire site.

Brian
Click to expand...

which, by the way, is all kinds of fun...
 
If you're downloading files and get disconnected, with wget, you can just resume the download and not have to redownload the whole thing.
 
Download an entire website???
Damn, I coulda used that information YESTERDAY!!! :p

How would one do that (download an entire website)?
 
Sogni said:
Download an entire website???
Damn, I coulda used that information YESTERDAY!!! :p

How would one do that (download an entire website)?
Click to expand...


at the command prompt, type;
$ wget --help
that'll tell you exactly how to download an entire Website.
This is useful if you want to mirror the entire website.

A Better alternative to wget in MacOSX is curl
at the command prompt, type;
$ curl --help
It comes bundles in MacOSX and is arguably more powerful.

I'm not an expert on curl or wget though. My use is restricted to bulk downloading of an image sequence;

$ for i in 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20;
do
curl -O http://www.<repository of your images>/<path>/$i.jpg;
done;
 
To get a whole site I do wget -r <site>, but depending on the server configuration it doesn't always work -- so mess around with it a little.
 
Mr K. - Set the -k flag as well as -r will correct links, images, etc. to point to the locally downloaded copies whenever the originals point to something on the server.

Also, you might want to set -l <depth> to limit how deeply it will follow links. All you need is to miss a link to the 'my 500 favourite links' page...
 
You must log in or register to reply here.
Back
Top