HTTP provides a general framework for access control and authentication. The most common HTTP authentication is based on the Basic schema, but others such as Digest and Bearer are also available. Sitemap Creator directly supports Basic schema, although Bearer can be used via custom headers.

In addition, you can use an embedded web browser to log into a site manually prior to Sitemap Creator crawling the site. This can be useful for more complex authentication systems that are not directly supported, such as multi-factor.


Sitemap Creator is unable to automatically log into web sites using two-factor authentication (2FA), multi-factor authentication (MFA), or that require client side JavaScript to be executed as part of the login process. You can use External Authentication or cookies to manually log into a target website prior to copying.


If you use Forms, Passwords or Cookies to authenticate with a website, you should consider adding a custom rule to exclude any logout pages. Otherwise, if Sitemap Creator detects this page, eventually it will access it and your session will be logged out, potentially affecting the remainder of the crawl.

Click the links below to learn more about working with passwords.

© 2010-2023 Cyotek Ltd. All Rights Reserved.
Documentation version 1.2 (buildref #81.15583), last modified 2023-04-09. Generated 2023-04-29 14:30 using Cyotek HelpWrite Professional version 6.19.1