Crawling multiple URLs

If there are sections of your site which do not have any inbound links, but which you still wish crawled, you can specify additional seed URLs for the crawl process. The additional list of URLs can also be read from an external file.

To set up multiple URLs for crawling

• From the Project Properties dialogue, select the Additional URLs category

Defining the URL list within the WebCopy project

1. Select the Load additional URLs from list option
2. Enter the URLs to process, one per line

Storing the URL list within an external text file

1. Select the Load additional URLs from file option
2. Enter or select the name of the external file name