Rate limiting server access

Introduction and miscellaneous that we haven't created categories for, yet
Post Reply
User avatar
johu
Site Admin
Posts: 5682
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Rate limiting server access

Post by johu »

Starting yesterday I noticed an excessive number of accesses from a specific IP address. They seemed to be downloading the entire forum or whatnot.

By the time I noticed it had been going on for 12 hours, the access log file had swollen to 700 MB instead of the usual 20 MB or so!

I have now implemented rate limiting. So if a given IP address makes too many requests within a certain time it will be completely blocked after a while.
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
mjc506
Posts: 343
Joined: Wed Sep 09, 2020 9:36 pm
Location: Wales, United Kingdom
Has thanked: 30 times
Been thanked: 28 times

Re: Rate limiting server access

Post by mjc506 »

Not a twitter/Elon IP was it? :)
User avatar
johu
Site Admin
Posts: 5682
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Rate limiting server access

Post by johu »

Unless he moved the head quarters to Darmstadt, Germany, no ;) Was thinking maybe TOR exit node?
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
mjc506
Posts: 343
Joined: Wed Sep 09, 2020 9:36 pm
Location: Wales, United Kingdom
Has thanked: 30 times
Been thanked: 28 times

Re: Rate limiting server access

Post by mjc506 »

User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Re: Rate limiting server access

Post by janosch »

Slight tangent:

Are forum & wiki dumps available actually (w/ user table reduced to nickname & PMs stripped out obviously)?

If it's easy to setup I would be interested to keep some local backups, otherwise we have to rely on archive.org if something ever happens to this website.

I noticed recently that for certain short search terms the forum search isn't producing the best results, so I have gone to google:
"site:openinverter.org BMS" to find short or specific terms.
User avatar
johu
Site Admin
Posts: 5682
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Rate limiting server access

Post by johu »

I do a more or less weekly backup of the entire data base and web server content. It sits here on a raid5 array (3 disks)

Current configuration wouldn't allow a dump to be pulled externally, at least not without rate limit from the client side.
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Scrappyjoe
Posts: 82
Joined: Fri Feb 25, 2022 3:16 pm
Has thanked: 22 times
Been thanked: 9 times

Re: Rate limiting server access

Post by Scrappyjoe »

Johu, I think it’s worth considering distributing backups between a small number of trusted members, who are known to the community. I have been in situations where forums just disappear when the admin loses interest or just ghosts. It’s a terrible loss, and sometimes the community never recovers.
User avatar
TheSilverBuick
Posts: 187
Joined: Mon Jun 08, 2020 5:01 pm
Location: Nevada, United States
Has thanked: 3 times
Been thanked: 1 time

Re: Rate limiting server access

Post by TheSilverBuick »

Probably someone paranoid that with Damien's departure/hiatus that this forum will vanish off the internet.
Need to learn. Starting from the basics.
Ammar
Posts: 4
Joined: Fri Dec 31, 2021 6:33 pm
Has thanked: 2 times

Re: Rate limiting server access

Post by Ammar »

Please do not use RAID5 for anything you care about. RAID6 cost you just one more disk, and will save you a lot of agony.
User avatar
johu
Site Admin
Posts: 5682
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Rate limiting server access

Post by johu »

Why?
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Post Reply