Every HTTP request is made of one or more headers. Usually, Sitemap Creator will take care of determining the correct set of headers to send. However, you can also define your own headers to include with requests.

Opening the custom headers editor

  • From the Project Properties dialogue, expand the Advanced category and select Custom Headers

Adding a new header

  1. Click the Add button to add an empty header definition
  2. Enter the header information into the Name and Value fields
  3. Optionally, if you only wish the header to be sent with certain requests, enter the URL into the URL Expression field. This field supports regular expressions.


If your expression includes any of the ^, [, ., $, {, *, (, \, +, ), |, ?, <, > characters and you want them to processed as plain text, you need to "escape" the character by preceding it with a backslash. For example, if your expression was application/epub+zip this would need to be written as application/epub\+zip otherwise the + character would have a special meaning and no matches would be made. Similarly, if the expression was example.com, this should be written as example\.com, as . means "any character" which could lead to unexpected matches.

Editing an existing header

  1. Select the header to edit from the list
  2. Modify the header properties using the Name, Value and URL Expression fields

Deleting a header

  1. Select the header you want to remove from the list
  2. Click the Delete button to remove the header

See Also

Configuring the Crawler

Working with local files

Controlling the crawl



Modifying URLs


Deprecated features

© 2010-2023 Cyotek Ltd. All Rights Reserved.
Documentation version 1.2 (buildref #81.15583), last modified 2023-04-09. Generated 2023-04-29 14:30 using Cyotek HelpWrite Professional version 6.19.1