Re: curl vs. wget (was Re: script browsing)
Re: curl vs. wget (was Re: script browsing)
- Subject: Re: curl vs. wget (was Re: script browsing)
- From: Gary Lists <email@hidden>
- Date: Wed, 07 Jan 2004 06:52:17 -0500
Walter Ian Kaye wrote [01-07-04 5:23 AM]:
>
> wget will read the pages it's downloaded to find more links to
>
> download, and can relink them after downloading. It's great for
>
> things like downloading all the thumbnails on a page, downloading
>
> all the linked images on a page, or even downloading an entire web
>
> site to your local drive complete with directory structure.
>
>
Yuck... now I think I'd like to find out its UA string so I can block
>
wget from my site. Don't want no site-suckers visiting. (Anyone got a
>
list of those?)
<
http://www.robotstxt.org/wc/robots.html> as a start...
--
Gary
_______________________________________________
applescript-users mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/applescript-users
Do not post admin requests to the list. They will be ignored.