Auto Crawler Guide

Auto crawler setting

• The basic settings of Auto crawler
• How to crawl a site with minimal impact on the contents

Specifications to which Auto crawler scanning is applicable/not applicable

• Is it possible to crawl websites using Basic authentication, Digest authentication, NTLM authentication, or client authentication
• Is it possible to crawl websites crossing multiple domains?

Troubleshooting

• The target URL is not shown in Crawling result but in Crawling exclude URL