While forms can have myriad purposes, in terms of crawling a website using Sitemap Creator they are normally used to authenticate a user.

Sitemap Creator is able to post forms to websites. As well as specifying values for user supplied parameters, the crawl engine will automatically include other parameters present on the original form, such as session tokens or anti-forgery tokens, without you needing to define these yourself.

Forms can be manually defined, or automatically captured. Once you have defined a form, you can test that it works correctly before initiating a website copy.


Sitemap Creator is unable to automatically log into web sites using two-factor authentication (2FA), multi-factor authentication (MFA), or that require client side JavaScript to be executed as part of the login process. You can use External Authentication or cookies to manually log into a target website prior to copying.


If you use Forms, Passwords or Cookies to authenticate with a website, you should consider adding a custom rule to exclude any logout pages. Otherwise, if Sitemap Creator detects this page, eventually it will access it and your session will be logged out, potentially affecting the remainder of the crawl.

Click the links below to learn more about working with forms.

© 2010-2023 Cyotek Ltd. All Rights Reserved.
Documentation version 1.2 (buildref #81.15583), last modified 2023-04-09. Generated 2023-04-29 14:30 using Cyotek HelpWrite Professional version 6.19.1