JavaScript enabled websites

While WebCopy will download JavaScript enabled websites and detect any linked JavaScript in HTML tags, it currently does not parse and execute JavaScript. This means that if a website makes use of JavaScript to construct the page client side, it is unlikely WebCopy will be able to make a true copy if it is unable to discover all of the website.

For example, if a website uses JavaScript to build the entire page then WebCopy will not find anything to crawl. If a website uses JavaScript to build page links (for example a dynamic navigation menu), then WebCopy will be unable to crawl to those pages unless they are referenced elsewhere.

Future Plans

While Cyotek hopes to include some basic JavaScript support in future versions of WebCopy, such support will be very conservative. For example, if a button had a click event, even if WebCopy had JavaScript support it would never trigger that click due to the potential for unexpected consequences (for example the button could be used to trigger the delete of a resource). Another example would be infinite scrolling - it would be quite difficult to try and mimic this and WebCopy would not try. WebCopy is not a web browser and does not pretend to be one.

Downloading the raw source code for web sites

WebCopy does not download the raw source code of a web site, it can only download what the HTTP server returns. This means you cannot, for example, use WebCopy to download the PHP source code of a web site or any backend databases - unless that website is incredibly badly configured!

Caveat emptor

While WebCopy will do its best to create an offline copy of a website, advanced data driven websites may not work as expected once they have been copied as the underlying source code and databases for a website can never be downloaded unless expressly offered by website owners.