About the WholesaleX crawler
To list a supplier, the software of WholesaleX reads the public website of that company. This page tells the operator of a website what that software does.
How it identifies itself
The crawler sends the name WholesaleXDirectoryBot and the address of this page in every request. It does not pretend to be a browser.
How it behaves
- It follows the rules in the robots.txt of your website.
- It waits at least 2 seconds between two requests to the same website.
- It takes at most 5 products of a company from its own product page.
How to stop it
Add a rule for WholesaleXDirectoryBot to your robots.txt, or write to us. We remove the data of a company on request.
Contact: global@wholesalex.de