If you crawl an URL such as http://www.cyotek.com/help/, by default WebCopy will not crawl above this URL. So it will match /help/index.htm, but will not match /products/index.htm. In some circumstances it may be desirable to allow this without changing the root URL to be at a higher level.
To enable or disable crawling above the root URL
- From the Project Properties dialog, click the Advanced category
- Check or uncheck the Crawl above the root URL option
See Also
Customizing Projects
- Customizing Projects
- Specifying the web site
- Specifying the save folder
- Remapping extensions
- Specifying how a website is crawled
- Specifying default documents
- Crawling additional root URL's
- Updating local time stamps
- Scanning custom attributes
- Fixing sites using mixed prefixes
- Extracting inline data