HTTP defines a number of status codes used to redirect one request to another, for example when the requested URL has been moved to another location, or when the URL can't be exactly matched, but similar locations are available. By default, WebCopy will following internal redirects only.
Option | Notes |
---|---|
Don't follow redirects | Specifies that no redirects should be followed |
Follow internal redirects | Follow redirects that point to the site currently being crawled |
Follow all redirects | Follows any redirect, regardless of destination. Note that crawl modes and rules should be used to control if any actions should be taken on the external URL |