wget

davidbrit2

Licensed Computer Geek
This is driving me nuts... Whenever I try to use wget for some sort of mirroring type recursive download, it just downloads the specified url and quits if I give it ANY arguments with comma separated lists. If I don't give it any lists, it works, but since I couldn't give it a list, it doesn't do what I want. I even suspected a bug with version 1.5.3 (which is included with OS X,) so I downloaded version 1.6 and compiled it. I still didn't have any luck. Can some wget expert point me in the right direction? What am I doing wrong?
 

Red Phoenix

Registered
I am not sure at what you are getting at here, I think. Are you specifically trying to do a recursive download, or are you trying to do a list? Or maybe it is somewhere in between? While I haven't messed around too much with either recursive downloads or downloading a list of files, it seems to support these in a logical fashion (this is based on the help file in "file:///Library/Documentation/Commands/wget/". Since I don't quite understand the question, I hope this helps. It details the "-I" command, which might be what you are looking for:

`-I list'
`--include list'
`include_directories = list'
`-I' option accepts a comma-separated list of directories included in the retrieval. Any other directories will simply be ignored. The directories are absolute paths. So, if you wish to download from `http://host/people/bozo/' following only links to bozo's colleagues in the `/people' directory and the bogus scripts in `/cgi-bin', you can specify:

wget -I /people,/cgi-bin http://host/people/bozo/

Another source for help with wget is typing "wget --help"

Finally, you can get wget to download a file containing URLs with the command "wget -i file". This might be the best option, since the file need not be comma delimited.

-Bruce Adcock
 

davidbrit2

Licensed Computer Geek
It's like this... When I give an option a comma separated list (which it's supposed to be able to handle,) it doesn't do any recursion.
e.g.
wget -r -D domain.com http://www.domain.com/index.html
would work fine, but...
wget -r -D domain.com,otherdomain.com http://www.domain.com/index
doesn't work.
When any option is given a list like that, it completely ignores all recursion.
 
Top