As it is impractical for WebCopy to crawl the Internet, by default the crawling is limited to only the domain of the crawl URI, but this can be changed as required.
|Only crawls URI's that match the host name specified in the crawl URI
|Includes any sub domains of the host URI
|Includes both sub domains and sibling domains of the host URI
|Will crawl any HTTP or HTTPS URI detected
Use of the Everything option is not recommended and should only be used on sites which are self contained or where rules are used to explicitly exclude addresses. Use of this option may cause WebCopy to become unstable.
While you may not wish to crawl external sites, it is still possible to download any files directly linked from the site you are crawling. When the Download all resources option is set, WebCopy will automatically download any external file, as long as the reported content type is not
text/html. The downloaded file will not be crawled, allowing easy downloading of linked images, sounds and other files.