Bots are hitting Sponsorchat real hard

[alsporn]

The problem seems to be mysql queries which slows down the site, is there a way to prevent anyone not registered going a search or any function that requires mysql?

It won't be searches so much. On a site like this (where new content is being added all the time) every page is dynamic and will run dozens of MySql queries with every page view. You could block all guests from being able to view anything but that's not really a good idea as users want to see what the site is about plus you would also block google, bing etc etc

But if you have a site where new content is added say once or twice a day, then its very easy to setup a cron job to read your high traffic pages every hour or so and store them as static .html files. That way the sql queries only have to run once an hour rather than every page load.

I can not see how that will work for a forum, posts have to be real time or else how will people reply. They will only get to see updates once an hour. This other forum i have been a member on since 2022 has this same problem. I wish i can post the url but it's not an adult forum and I do not want to get banned there for sharing the link on an adult board.

[tacnet]

The problem seems to be mysql queries which slows down the site, is there a way to prevent anyone not registered going a search or any function that requires mysql?

It won't be searches so much. On a site like this (where new content is being added all the time) every page is dynamic and will run dozens of MySql queries with every page view. You could block all guests from being able to view anything but that's not really a good idea as users want to see what the site is about plus you would also block google, bing etc etc

But if you have a site where new content is added say once or twice a day, then its very easy to setup a cron job to read your high traffic pages every hour or so and store them as static .html files. That way the sql queries only have to run once an hour rather than every page load.

I can not see how that will work for a forum, posts have to be real time or else how will people reply. They will only get to see updates once an hour. This other forum i have been a member on since 2022 has this same problem. I wish i can post the url but it's not an adult forum and I do not want to get banned there for sharing the link on an adult board.

Yep your exactly right. It's no good on a site like this (or any social media site) where users want to see updates as they happen

It works for us because we add updates daily and so our high traffic pages (index page etc) are updated once an hour which keeps the load off the Sql server

The only thing you can do on a social media site like this is create MySql indexes. MySql runs a LOT faster if it can use indexes but you really need to have a good understanding of MySql and know what queries are being run in order to do that.

[TheButcher]

One thing I noticed bots do not start out at the index page.

[webcamhostess]

They even go for white label sites, I have a few dozen and noticed they get hit by bots, I asked a few of the sponsors of my WL's and they all said it's bots they are trying to manage the attacks.

[TheButcher]

I just registered at Cloudflare and changed the name servers to them,there is a thread if anyone has issues but it all went very smoothly and no down time.

Peace

[Cars-n-Chicks]

Bots are hitting all sites this year including mine.

[jag]

2026 is going to remembered for the bot attacks across the net

[tacnet]

I think this is where you've got to look at converting some of your high traffic pages to static html so the bots don't bring down your SQL server

We've got pages that update every minute using a simple cron job that reads the pages and stores them as static html files.

[TheButcher]

I think this is where you've got to look at converting some of your high traffic pages to static html so the bots don't bring down your SQL server

We've got pages that update every minute using a simple cron job that reads the pages and stores them as static html files.

Will it work with phpbb?

[TheButcher]

Not to jinx things but for the last 2 days I have not had to turn on the "Under attack mode" in Cloudflare the most bots I see with just Cloudflare protection is 230 at a time max

[tacnet]

I think this is where you've got to look at converting some of your high traffic pages to static html so the bots don't bring down your SQL server

We've got pages that update every minute using a simple cron job that reads the pages and stores them as static html files.

Will it work with phpbb?

Yes. I guess your main traffic pages will be the index page and the main topic list viewforum.php?f=74. You could easily have a cron job produce static html versions of both those pages that update once a minute.

Then in your .htaccess, you simply add a line to redirect traffic to those pages. Something like this :-

Code: Select all

RewriteEngine On
RewriteRule ^viewforum.php?f=74$ /topics.html [NC,L]
RewriteRule ^/?$ /index.html  [NC,L]

Those may not be the pages the bots are hitting, but offloading them to HTML will certainly reduce your server load. These are all standard techniques for scaling up for large traffic sites

[TheButcher]

I guess your main traffic pages will be the index page and the main topic list viewforum.php?f=74. You could easily have a cron job produce static html versions of both those pages that update once a minute.

Then in your .htaccess, you simply add a line to redirect traffic to those pages. Something like this :-

Code: Select all

RewriteEngine On
RewriteRule ^viewforum.php?f=74$ /topics.html [NC,L]
RewriteRule ^/?$ /index.html  [NC,L]

Those may not be the pages the bots are hitting, but offloading them to HTML will certainly reduce your server load. These are all standard techniques for scaling up for large traffic sites

The htaccess part is a simple copy n paste but I have no idea where or how to write a cron job boss.

[tacnet]

I guess your main traffic pages will be the index page and the main topic list viewforum.php?f=74. You could easily have a cron job produce static html versions of both those pages that update once a minute.

Then in your .htaccess, you simply add a line to redirect traffic to those pages. Something like this :-

Code: Select all

RewriteEngine On
RewriteRule ^viewforum.php?f=74$ /topics.html [NC,L]
RewriteRule ^/?$ /index.html  [NC,L]

Those may not be the pages the bots are hitting, but offloading them to HTML will certainly reduce your server load. These are all standard techniques for scaling up for large traffic sites

The htaccess part is a simple copy n paste but I have no idea where or how to write a cron job boss.

ok this is quick and dirty. Cron job will look something like this :-

Code: Select all

<?php 
//   First write the index to index.html
$html = file_get_contents('https://www.sponsorchat.com/index.php');
$filename = 'index.html';
$handle = fopen($filename,"w");
fwrite($handle,$html);
fclose($handle);

//   Now write topic list to topics.html
$html = file_get_contents('https://www.sponsorchat.com/viewforum.php?f=74');
$filename = 'topics.html';
$handle = fopen($filename,"w");
fwrite($handle,$html);
fclose($handle);
?>

Then, if your not familiar with editing your crontab file then I would get your hosts to add this line to it

Code: Select all

* * * * * php /full/path/to/your/php/file/mycronfile.php

That will run your cron file once every minute

Hope that helps

[TheButcher]

I guess your main traffic pages will be the index page and the main topic list viewforum.php?f=74. You could easily have a cron job produce static html versions of both those pages that update once a minute.

Then in your .htaccess, you simply add a line to redirect traffic to those pages. Something like this :-

Code: Select all

RewriteEngine On
RewriteRule ^viewforum.php?f=74$ /topics.html [NC,L]
RewriteRule ^/?$ /index.html  [NC,L]

Those may not be the pages the bots are hitting, but offloading them to HTML will certainly reduce your server load. These are all standard techniques for scaling up for large traffic sites

The htaccess part is a simple copy n paste but I have no idea where or how to write a cron job boss.

ok this is quick and dirty. Cron job will look something like this :-

Code: Select all

<?php 
//   First write the index to index.html
$html = file_get_contents('https://www.sponsorchat.com/index.php');
$filename = 'index.html';
$handle = fopen($filename,"w");
fwrite($handle,$html);
fclose($handle);

//   Now write topic list to topics.html
$html = file_get_contents('https://www.sponsorchat.com/viewforum.php?f=74');
$filename = 'topics.html';
$handle = fopen($filename,"w");
fwrite($handle,$html);
fclose($handle);
?>

Then, if your not familiar with editing your crontab file then I would get your hosts to add this line to it

Code: Select all

* * * * * php /full/path/to/your/php/file/mycronfile.php

That will run your cron file once every minute

Hope that helps

Thanks will give it a shot Monday so I don;t bug the host guys over the weekend.

By the way bots were having a field day today so turned on the "Under attack" mode in Cloudflare it's a great tool they provide.