I meant to post a summary about the server downtime once it's been resolved, but since it's taking much longer than I anticipated, I'll write it now to keep everyone in the loop. We've been pushing frequent updates on our Discord server, but I understand that not everyone uses Discord. I'll use this article to post updates regarding the downtime.

We're currently down because of a large scale DDOS attack. Or attacks, rather - we're dealing with a fight on a couple different fronts. Some of the attacks are volumetric (we've peaked between the 100-200 gbps range so far), some are application specific. Simply said, the malicious traffic is made to look much like a regular player, doing regular things. It becomes difficult to distinguish what's real and what's not for our filters, which lets the malicious traffic through.

We've dealt with many attacks in the past and I was happy to say that we weren't really down because of a DDOS in a couple years now. That's why I didn't think much of it at the beginning, I figured our host (OVH at the time) would take care of it as they usually do. But they didn't at first, so I started reaching out to them for help, to see if they could spare a technician to work on a solution with us. Despite dozens of e-mails and calls, I was hugely unsuccessful and was stuck trying to frankenstein a solution together with other members of our team.

Somewhere at this point, I've made a critical mistake of not communicating continuous updates to our leadership team, I kept my efforts to the discussion between me and OVH at the time, updating them with what I've already tried. And I did try all that I really could - there's only so much you can do to deal with an attack of this scale and precision without having direct access to the hardware that the protection runs on. Regardless, I assume the lack of frequent updates during the prolonged issues is what made Surreal, Davis, Jakey and Apophis quit. I say I assume, because I was never told or given a notice of more than one hour before they did. There's plenty I disagree with here and dislike the way it was handled, but I don't see any benefit in diving much deeper into it. Hey, shit happens and I hope they succeed in whatever they focus on now.

So, that's the summary of the lead-up to last week. About a week ago, after trying several different hosts with no positive results, I did more reserach and decided to give two more hosts a try, the first one of which we're dealing with right now - Clouvider. They've gone above and beyond with their support and with the help of their partner who provides DDOS protection we've been able to analyze the multitude of attacks that are currently deployed against us. We've applied several solutions hand-crafted for our situation, with limited success. While there's been significant progress made and the server was playable for some amount of time, we're still facing frequent timeouts and not everyone is able to connect. This is in part because of some players being in the IP ranges that the provider blocked (blocking certain IP ranges is only one of our methods of countering the attack). We've a solution currently in the works, but because of the holidays, it's taking a bit longer to deploy than we've anticipated, but once it is, we should be good to move on from this whole mess.

As a side note, please avoid conspiracy theories about who the attacker is. We don't know, and you don't know. Nobody reached out to us and all of the rumors floating around are exactly that - rumors.

I'd like to address everyone who's offered their help in this situation with different tips on how to protect ourselves from this. While I'm very thankful for your insights, we're working with professionals who deal with this every day and have been for a long time now. There's a big chance that what you're suggesting was already tried, or doesn't make sense in our case.

To address the leadership changes - while this will definitely have large impact on our community, it's nothing the server hasn't seen and we'll be able to move on from that. Bennemus is stepping up to help out with the community side of LS-RP and we'll be working on filling vacant positions (think department leaders, not necessarily a bunch of lead admins right away) as soon as the server is stable. The forum will also be back once the server's more stable - we're working on making some changes there. We've also gotten rid of a bunch of toxic topics about this whole situation.

Speaking of which, I understand that during times such as these, emotions run high. But beware that toxic behavior won't be tolerated. Everyone's doing their best right now and while we won't stop anyone from expressing frustration, snarky remarks and toxicity will be removed from our communication channels. It's not helping anyone.

In contrast, I appreciate all messages of support we've received - It's really great to see how many of you care. I also appreciate the staff members who stuck around, because we aren't going anywhere. We'll figure out a solution and move on like we always do. I'm nowhere ready to give up on LS-RP and will not run away or crumble under pressure from criminals.

Premium packages won't expire until the server's stable again. Everyone will be reimbursed with additional time for the server downtime.

TL:DR - We're under attack, and it's difficult to deal with. We're working around the clock to figure out a solution. Most of lead admins left. Communication could've been better. Forums up when the server is stable. Updates will be on this post. LS-RP is not shutting down.

Thank you for your attention and Happy Holidays, everyone!

- Martin