If you run SEMrush Audits, you might have experienced SEMrush being blocked from auditing a website.
You will see an error code like this:
Sorry, auditing domain.com has failed
We encountered an error that stopped us from crawling your website:
Network connection was interrupted.
We couldn't crawl your homepage due to connectivity issues on your site.
How to fix it
This issue is typically caused by your hosting provider blocking our crawler. Please download a log file and contact your website administrator to fix this error. See how to whitelist SEMrushBot on different hosting providers.
How to fix SEMrush bot from being blocked
- Check your error code by downloading the log.
- Whitelist SEMrush's IP range in your Hosting, or remove any Robots.txt blockages.
- Purge your cache
As you can see above, Cloudflare is blocking my SEMrush Audit scan, find out how to fix it below:
How to whitelist SEMrushBot in Cloudflare:
Log in to your Cloudflare account.
Select your domain.
Click the Firewall >> Tools sub-menu settings.
Under IP Access Rules, enter the following details:
- Enter the SEMrush Values. (184.108.40.206, 220.127.116.11, 18.104.22.168)
- Select an Action >> Allow.
- Select whether the rule applies to This website or All websites in the account.
- Add a note (i.e., SEMrush).
Purge your cache and wait at least an hour before trying to audit your website again.
This can be done by clicking on "purge everything" under the caching tab.