Crawling above the root URL

If you crawl an URL such as http://www.cyotek.com/help/, by default WebCopy will not crawl above this URL. So it will match /help/index.htm, but will not match /products/index.htm. In some circumstances it may be desirable to allow this without changing the root URL to be at a higher level.

To enable or disable crawling above the root URL

1. From the Project Properties dialog, click the Advanced category
2. Check or uncheck the Crawl above the root URL option