HTTP defines a number of status codes used to redirect one request to another, for example when the requested URI has been moved to another location, or when the URI can't be exactly matched, but similar locations are available. By default, WebCopy will following internal redirects only.
Option | Notes |
---|---|
Don't follow redirects | Specifies that no redirects should be followed |
Follow internal redirects | Follow redirects that point to the site currently being crawled |
Follow all redirects | Follows any redirect, regardless of destination. Note that crawl modes and rules will then control if any actions should be taken on the external URI |