You can configure your project to automatically apply one or more cookies before crawling commences. This can be used to provide authentication cookies in the event that the built in authentication features of Sitemap Creator are not sufficient.


Cookies stored in a Sitemap Creator project must conform to the Set-Cookie header syntax. The minimum required is <cookie-name>=<cookie-value>. <cookie-value> should be URL encoded if appropriate, Sitemap Creator does not perform any automatic encoding.


If you use Forms, Passwords or Cookies to authenticate with a website, you should consider adding a custom rule to exclude any logout pages. Otherwise, if Sitemap Creator detects this page, eventually it will access it and your session will be logged out, potentially affecting the remainder of the crawl.

To customise cookies

  • From the Project Properties dialogue, expand the Advanced category and select the Cookies sub-category
  1. Click the Add button
  2. Enter the the cookie data into the Data field
  1. Select one or more cookies that you wish to remove
  2. Click the Delete button
  1. Select the cookie to edit from the list. The Data field will be updated to contain the value of the cookie
  2. Enter new value for the cookie data

Reading cookies from an external file

To read cookies from an external file, enter the file name in the Read Cookies From field, or click the Browse button to select a file.


Only cookies in the Netscape cookie file format are supported.

Discarding session cookies

To discard any session cookies from the external file, ensure the Discard session cookies option is checked. When set, any cookies without an expiry date will be skipped.

Writing cookies to an external file

Once a copy operation has complete, Sitemap Creator can optionally write all cookies into a file, using the Netscape cookie format. To write cookies to a file, enter the file name in the Write Cookies To field, or click the Browse button select a file.


Cookies are only written when performing a copy, not when performing a read-only scan.

See Also

Configuring the Crawler

Working with local files

Controlling the crawl



Modifying URLs


Deprecated features

© 2010-2023 Cyotek Ltd. All Rights Reserved.
Documentation version 1.2 (buildref #81.15583), last modified 2023-04-09. Generated 2023-04-29 14:30 using Cyotek HelpWrite Professional version 6.19.1