Automated backup (download) via FTP/HTTP

[michaelsanford]

I'm looking for a *NIX app, PERL app or script suite that will let me copy my remote web site to a local folder/NFS mount (this is to run from my Slackware machine).

Ideally, of course, I'd like only to download changed files but if that's not possible I don't really mind that much...I don't pay for bandwidth overload-usage and my site's not very big.

I don't have shell access, sadly, so I can't just Tar-GZ the folder and download that, which would be a simple wget.

N.B. Why would I want to pull my site down if I have to upload the designs from my computer in the first place ? My site is dynamic and uses a few wikis and other dynamic-content engines so I'd have to download the versions changed on the server...

 

[scruffy]

wget --recursive --convertlinks --page-requisites http://hostname.tld/dir/index.html

Or some variant of the above.

 

[scruffy]

Oh - note that wget will respect robots.txt, so some stuff might still be excluded.

 

[michaelsanford]

So will files that have no links, like my mirror and private folders.

 

[ksv]

Perhaps you can use rsync? Check its manual page

 

[michaelsanford]

Hey I've never considered that...I'll take a look at it.