HTTP provides a general framework for access control and authentication. The most common HTTP authentication is based on the Basic schema, but others such as Digest and Bearer are also available. WebCopy directly supports Basic schema, although Bearer can be used via custom headers.

In addition, you can use an embedded web browser to log into a site manually prior to WebCopy crawling the site. This can be useful for more complex authentication systems that are not directly supported, such as multi-factor.

Important

WebCopy is unable to automatically log into web sites using two-factor authentication (2FA), multi-factor authentication (MFA), or that require client side JavaScript to be executed as part of the login process. You can use External Authentication or cookies to manually log into a target website prior to copying.

Important

If you use Forms, Passwords or Cookies to authenticate with a website, you should consider adding a custom rule to exclude any logout pages. Otherwise, if WebCopy detects this page, eventually it will access it and your session will be logged out, potentially affecting the remainder of the crawl.

Click the links below to learn more about working with passwords.

© 2010-2024 Cyotek Ltd. All Rights Reserved.
Documentation version 1.10 (buildref #185.15779), last modified 2024-03-31. Generated 2024-03-31 14:04 using Cyotek HelpWrite Professional version 6.19.1