The site has been down a lot lately, and I mean, a lot. Our automated systems would usually bring it back up within half an hour only for it to go down again in maybe five minutes.
Piper finally had to take time off from her house move to look into it. Turns out AI spider bots had found a vulnerability and were exploiting it to scan the site over and over. We were getting hit by these bots hundreds of thousands of times a second, over and over and over. The bots had no purpose but to visit the site and explore all the connections. With thousands of stories and hundreds of authors, and an assload of tags, we had a lot. Hence, all the 503 failure to fetch, and 504 timeout errors. The site wasn't actually down, it just couldn't keep up.
Closing some vulnerabilities and strengthening our firewalls stopped the visits by the plethoras, those ravening fishes.
Anyway, Piper saved us again. Hooray for Piper, and I hope the rest of your move goes smoothly tomorrow! :)
Hugs,
Erin
Comments
Bots
I've been wondering how well the protections against these parasites are holding up.
I despise the ideas of them using our stuff for training data. Fuck AI.
Problem is they are going to violate the copyrights of the various authors here from what they had scooped up.
We really need better proactive protections.
Courts already ruled it's
Courts already ruled it's fair use to rip anything they want for training data despite all the proof it's illegal. Weird that they sided with the billionaire /s.
Keeping the bots out
Isn't this something that Cloudflare should be handling anyway? Or do you have to ask for it to be added for your website(s)?
They found a flaw
There are exceptions for Cloudflare, they exploited one. It's been closed.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.
Big thank you!
Big thank you to Piper for getting this fixed! And thank you, Erin, for filling us in on what’s going on “behind the curtain.” I’ll confess I was getting pretty worried, and to be honest, while copyright violations are annoying, they don’t even make my top 50 things to worry about . . . not these days.
PS - Wouldn’t you know — when I tried to post this, I got a “this site can’t be reached” message?
— Emma
i think BC has been hurt like this before, Erin
You and you elves do a great job but the jerks are always out there.
John in Wauwatosa
That explains a lot
I'm glad Piper was able to fix it. Please extend our thanks to Piper. We all know the trials and tribulations that she's been through lately and understand that life can throw a monkey wrench into one's daily routine. All we can do is deal with it the best we can.
Hugs
Patricia
Happiness is being all dressed up and HAVING some place to go.
Semper in femineo gerunt
Ich bin ein femininer Mann
Can anyone tell me?
What's a spider bot and how is this scenario being used for training?
As you can see, I'm a complete ignoramus when it comes to things like this!
Stay safe
T
Spider bots and training
A spider bot is a program that crawls the web, collecting data of some kind. AI spider bots collect content to feed their AI with info they can use to make content that resembles the stuff they collected, the way Frankenstein's monster resembles a piano teacher, a hog farmer, a stewardess, a lumberjack and an optometrist.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.