Aspose.3D Bot is the name of the crawler that gathers data for the Aspose.3D Search app.
There're actually two kinds of Aspose.3D Bot:Aspose.3D Bot looks for directives in your robots.txt file. You can use the Crawl-delaydirective to tell Aspose.3D Bot how quickly it can request pages from your website. Our default value for the Crawl-delay is 3 seconds if it's undefined in your robots.txt. One index task per 3 seconds should not hurt or overburden your web server; however, we'll honor whatever setting you choose.
Yes. All you have to do is add some directives in your robots.txt file. The directives determine which paths we're allowed to access, if any, and which paths we cannot.
For example, if you don't want Aspose.3D Bot to access any pages under /models, you can add following directives:
User-agent: Aspose3DBot Disallow: /models
If you don't want Aspose.3D Bot to access any portion of your site, then you can add following directives:
User-agent: Aspose3DBot Disallow: /
If you offer 3D models to the public, then please consider allowing Aspose.3D Bot to crawl your website. We're trying to share your website with new users and bring traffic to your website. We do not steal your IP, data, traffic or any rights to your content.
If we find Aspose.3D Bot has been disallowed, then we will respect your decision. Our bot will check your robots.txt at a fixed period, usually once per day. If we find we're not welcomed, then we will not access your web site anymore.
Currently we only have one app that will access external websites, and it's user agent is: Mozilla/5.0 (compatible; Aspose3DBot; +https://products.aspose.app/3d/bot)
If you would like any links to or content regarding your website removed from our search results, then you can send an email to dmca@aspose.com. We will:
For more details regarding IP removal, please check our DMCA Policy.