By default, the Sitemap Creator will ignore HTTP status codes that indicate errors, such as 503. You can configure the crawl process to abort when it hits a matching status code. For example, if the website is down and returns 503 or 504, then you probably don't want to continue! Alternatively, if you are testing links, you may wish to abort on 404's so you can correct the link before continuing.

To configure HTTP cancellation codes

  1. From the Project Properties dialog, click the Advanced option.
  2. Enter a comma separated list of HTTP status codes that will cancel a crawl in the Status Codes field.