Today, like the past few days, we have had some downtime. Apparently some script kids are enjoying themselves by targeting our server (and others). Sorry for the inconvenience.
Most of these ‘attacks’ are targeted at the database, but some are more ddos-like and can be mitigated by using a CDN. Some other Lemmy servers are using Cloudflare, so we know that works. Therefore we have chosen Cloudflare as CDN / DDOS protection platform for now. We will look into other options, but we needed something to be implemented asap.
For the other attacks, we are using them to investigate and implement measures like rate limiting etc.
Anything we can do as “users” to help, other than donating?
Hmm, best would be if those kids find a real hobby so they stop bothering us. On the other hand, it helps us understand Lemmy better and secure it.
That’s true. Free stress testing the system I guess? Still they need to touch grass lol
If it’s the same people, they’ll probably get tired of it and move on. But the more we talk about it, the more likely it is that new people want to get in on the “fun”. I’d say to not make memes about the downtime and pretty much act like it doesn’t exist (as users, obviously the admins should take action as necessary to mitigate it and post to be transparent).
deleted by creator
This would probably be less useful than you think.
Firstly, changing the default doesn’t matter - the attackers will just switch to targeting whichever URL causes the most pain.
Secondly, in comparison to prerendered pages of live content, js files are incredibly cheap and easy to chuck on a CDN. They don’t change often, so you don’t need to worry about cache invalidation, and even at a server level they’re probably hosted by a simple file server rather than hitting the DB.
deleted by creator
Do you run a reverse proxy infront? Eg. nginx is pretty performant at dropping unwanted traffic.
That doesn’t help with volumes of otherwise legit looking traffic right? The problem that Cloudflare and Akamai etc address is usually content that is otherwise static that can be cached. Say the front page of hot lemmy.world is updated every few minutes with the newest hot item. That page is otherwise distributed by the CDN so the CDN can just direct the traffic to access it, and no requests are made to lemmy.world.
nginx would be helpful for any attacks located from a single address trying to making large numbers of connections, but without reading more into the attack I can suspect that this isn’t what the attacker did.
That’s for for always keeping everyone up date. Sucks that you have these people wanting to DDOS a free community of people, I don’t get it.
Either way thank you. Now to just somehow find a decentralized version of CloudFlare so we don’t have to deal with there trackers that they have.
Tagging @ernest in case instance owners don’t have a larger community in which they share news like this with each other.
deleted by creator
Growing pains. This server and the platform will be better for it. If not for these script kids, some other attacker would eventually be motivated to try it.
Thank you! I will donate tomorrow
Be aware that you use another server so you might consider donating to them instead.
I have an account on yours too, but I might split it between both indeed :)
I’m curious, why bother with multiple accounts? It seems counterintuitive when taking federation into account
I do it for the following reasons:
- The big main lemmy servers can and do go down regularly. having accounts on other instances still gives me the ability to log in and participate on the communities I care about. I sure do wish the underlying federation logic allowed for associating selected logins on various instances together somehow.
- Testing: often, I will create a post or comment into a community hosted on a different lemmy server and not see it update. When this happens, I’ll log in on the other instance to see what that post or comment looks like from there.
- Insurance against defederation: I participated/followed some beehaw.org communities before they defederated, and then I was forced to open an account there to continue participating in those communities.
Well, I can’t answer for them but this situation in particular makes it nice to have accounts on different instances. If I can’t log on to/load my lemmy.world account then I can switch to my lemm.ee account and load content there.
Taking some load from the biggest servers such as LW. I still have a community on LW however, and mod with my local account
I have accounts on both lemmy.world and outpost.zeuslink.net. I have had a lot of problems with seeing posts from the smaller instance, while lemmy.world has been overloaded due to the massive Reddit exodus and script kiddies.
I created the account on the smaller instance first, but I’m still having a ton of issues with federation on that one (many communities still come up blank there, while lemmy.world has enough users that I can see everything when it isn’t down).
since 0.18.2 I havn’t noticed any federation issues with my own personal instance, nothing obvious anyway.
So far federation seems to be a lot better since the fix from last week - I can’t say that I’ve seen anything out of place or out of sync myself, but I am trying to keep a vigilant eye out for anything that’s been overlooked!
deleted by creator
Cloudflare isn’t bad per se, but having huge amounts of the public internet behind a centralized provider is bad for the flexibility and resiliency of the internet as a whole.
Obviously cloudflares ddosing lemmy just to get some extra money
Well I signed today and I got an error saying rate limit earlier for using these types of symbols “î¦âö)ééäë((ºÜݨ¿ã¿ï” I’m assuming It has nothing to do with this but just In case I’m making a comment about it edit:also just realized It may have been from how long the password was (33 characters)
We was behind Cloudflare since day one 😀 And even on Cloudflare there is not the origin IP its again reverse proxied, and we are small site compared to lemmy.world 😜
Whats the motivation to DDOS? How mutch is specific malice to lemmy or lemmy.world itself and how much is genaric.
they have a bad habbit of getting Brain bleeds