HTTP persistent connection, also called HTTP keep-alive, or HTTP connection reuse, is the idea of using a single TCP connection to send and receive multiple HTTP requests/responses, as opposed to opening a new connection for every single request/response pair. This can improve crawl performance but can sometimes cause requests to fail with a protocol violation error.

To disable keep-alive

  1. From the Project Properties dialog, select the Advanced category
  2. Uncheck the Keep alive option


It is not recommended this option unless you are having specific problems with a given website.

See Also

Configuring the Crawler

Working with local files

Controlling the crawl



Modifying URLs

Creating a site map


Deprecated features