Crawling additional hosts

By default, WebCopy will only scan the primary host you specify, for example http://example.com. By setting the sub domains option, you can include sub-domains, for example http://demo.example.com. In additional to this, you can instruct WebCopy to include other hosts with completely different domain names, for example if the site you are copying makes use of a CDN.

Some project settings are ignored when crawling additional hosts, for example the crawling above the root URI

Configuring additional hosts

1. From the Project Properties dialog, click Additional Hosts
2. Enter each additional host you want to crawl, one host per line. Do not enter protocol or path information, only include the domain name
3. Click OK to save your changes. When you next crawl this website, any URI's belonging to the hosts you specify will no longer be skipped, but will be crawled as though they were part of the primary project URI.