Many web servers can compress data prior to sending it, using a variety of different compression methods. When a client makes a request of a server, it includes the Accept-Encoding header which informs the server which encoding methods the client supports. Sitemap Creator supports the following compression methods

  • Compress (Legacy)
  • Deflate
  • GZip
  • Brotli (Non-standard)
  • BZip2 (Non-standard)
  • Identity (no compression, not directly selectable)


Disabling compression will cause the Identity value to be sent for the Accept-Encoding header, informing the web server not to compress content before serving it.


Encoding options generally only apply to static content, such as HTML, CSS and JavaScript. Other files such as Pdf or Zip are already compressed and normally won't be recompressed by a server.

To enable or disable compression

  1. From the Project Properties dialogue, select the HTTP Compression option group.
  2. Check or uncheck the types of compression methods you wish to support.


It is recommended to always ensure that at least Deflate and GZip options are enabled as this helps makes downloads of HTML and static content smaller and faster.


Some servers may not support all available compression options. If you receive 406 (Not Accepted) errors when trying to copy a website, specifying an unsupported encoding could be the cause.

See Also

Configuring the Crawler

Working with local files

Controlling the crawl



Modifying URLs


Deprecated features

© 2010-2023 Cyotek Ltd. All Rights Reserved.
Documentation version 1.2 (buildref #81.15583), last modified 2023-04-09. Generated 2023-04-29 14:30 using Cyotek HelpWrite Professional version 6.19.1