Is it okay to block 'ahrefs.com/robot/' from accessing my sites

[Aleks]

Is it okay to block 'ahrefs.com/robot/' from accessing my sites, they are flooding my sites and slowing it down trying to reply to comments on the blogs.

Thanks for the advice

[TheButcher]

I have had quite a bit of trouble with bots since Christmas, I think I might have mentioned here at some of those times.

here is what I discovered works, I added it to my .htaccess file.

Code: Select all

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^.*(Ahrefsbot|Claudebot|Yandex|Bytespider|Petalbot).*$ [NC]
RewriteRule .* - [F,L]

You can add more bots by the name in the list

I hope this helps.

[viking]

You really blocked Ahrefs?

[TheButcher]

You really blocked Ahrefs?

Yes and Semrush and a total of 11 so far, watching the bots as I see them I add them to the .htaccess file.

I only care about Google and Bing,

[Webby]

You really blocked Ahrefs?

Yes and Semrush and a total of 11 so far, watching the bots as I see them I add them to the .htaccess file.

I only care about Google and Bing,

Semrush is important you might want to unblock the spiders.

[TheButcher]

You really blocked Ahrefs?

Yes and Semrush and a total of 11 so far, watching the bots as I see them I add them to the .htaccess file.

I only care about Google and Bing,

Semrush is important you might want to unblock the spiders.

I rather sleep at night than get an alert from downnotifier that the site is down. Get online check CPanel, open FTP, logs and open tech support tickets. I honestly want Google and Bing traffic and can live without all the rest. My learned friend gotonudecom has been telling me for years to get Google Anyltics on the board, with all due respect to him I really have had it with bots.

[tacnet]

Yes and Semrush and a total of 11 so far, watching the bots as I see them I add them to the .htaccess file.

I only care about Google and Bing,

Semrush is important you might want to unblock the spiders.

I rather sleep at night than get an alert from downnotifier that the site is down. Get online check CPanel, open FTP, logs and open tech support tickets. I honestly want Google and Bing traffic and can live without all the rest. My learned friend gotonudecom has been telling me for years to get Google Anyltics on the board, with all due respect to him I really have had it with bots.

You really should get analytics on here. All it takes is to register with google, then paste a bit of code into your template and you will be done

[TheButcher]

You really should get analytics on here. All it takes is to register with google, then paste a bit of code into your template and you will be done

I will once I get a few nights sleep, last few months has been an on going battle with the mysql database being over worked with hundreds of bots trying to register and post. I really believe today was a bog day figuring out how to block all these bots.

[BrasSmonkey]

don't add google it fucks things up....


Does Google Analytics Affect the Performance of My Website? The answer is not a simple yes or no. While the initial implementation of the tracking code itself does not significantly slow down a website, certain features and settings within Google Analytics can have an impact on website performance.Jun 21, 2023


https://www.igvinc.com/blog/can-google- ... y-website/

[tacnet]

don't add google it fucks things up....


Does Google Analytics Affect the Performance of My Website? The answer is not a simple yes or no. While the initial implementation of the tracking code itself does not significantly slow down a website, certain features and settings within Google Analytics can have an impact on website performance.Jun 21, 2023


https://www.igvinc.com/blog/can-google- ... y-website/

Dont listen to this advice. This is only when you are using the advanced event tracking or e-commerce tracking. As it says on that page "However, when properly configured and optimized, the impact of Google Analytics on website speed is typically minimal."

Also, its possibe to load the script a few seconds AFTER the page has completed loading. We do this on all our pages on TAC so it discounts people who immediately close down the page (eg if it has been popped) and also has no impact on page load speed

[BrasSmonkey]

don't add google it fucks things up....


Does Google Analytics Affect the Performance of My Website? The answer is not a simple yes or no. While the initial implementation of the tracking code itself does not significantly slow down a website, certain features and settings within Google Analytics can have an impact on website performance.Jun 21, 2023


https://www.igvinc.com/blog/can-google- ... y-website/

Dont listen to this advice. This is only when you are using the advanced event tracking or e-commerce tracking. As it says on that page "However, when properly configured and optimized, the impact of Google Analytics on website speed is typically minimal."

Also, its possibe to load the script a few seconds AFTER the page has completed loading. We do this on all our pages on TAC so it discounts people who immediately close down the page (eg if it has been popped) and also has no impact on page load speed

ok...... don't do it~~~

[born2bewild]

.htaccess is the sure way when I started blocking certain bots \using robots.txt it was not as effective,some bots did not pay attention to that file but htaccess sure did stop them.

[saucyadele]

I see this in dozens on my logs for one of the blogs, what is it.I mean dozens in the past 24 hours.

"Guest
Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/10.0.2"

[razor]

Block all Chinese and Indian search bots. The traffic is not going to convert so what do you care if they list your site.

[TheButcher]

Just an update, after blocking the useless bots the sever is running 0.3 load max, I wish i had done this a long time ago.