Re: curl vs. wget (was Re: script browsing)
Re: curl vs. wget (was Re: script browsing)
- Subject: Re: curl vs. wget (was Re: script browsing)
- From: Sander Tekelenburg <email@hidden>
- Date: Wed, 7 Jan 2004 16:34:54 +0100
At 02:23 -0800 UTC, on 2004/01/07, Walter Ian Kaye wrote:
>
At 09:26p +1300 01/07/2004, Tom Robinson didst inscribe upon an
>
electronic papyrus:
[...]
>
>wget will read the pages it's downloaded to find more links to
>
>download, and can relink them after downloading. It's great for
>
>things like downloading all the thumbnails on a page, downloading
>
>all the linked images on a page, or even downloading an entire web
>
>site to your local drive complete with directory structure.
>
>
Yuck... now I think I'd like to find out its UA string so I can block
>
wget from my site.
Don't fool yourself. Just like many other http user-agents wget can send any
ID string (and cookies and http referers, etc.). Browser sniffing as an
exclusion mechanism will at best _appear_ to work.
>
Don't want no site-suckers visiting.
The Web is a publishing system. If you don't want something published, don't
put it on a Web server. If you do want it published, who cares what
user-agent people use?
From a scripter's perspective this is what's useful about wget/curl. No need
to GUI-script some popular browser. Use do shell script to 'create' the
user-agent you need.
--
Sander Tekelenburg, <
http://www.euronet.nl/~tekelenb/>
_______________________________________________
applescript-users mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/applescript-users
Do not post admin requests to the list. They will be ignored.