What are the platforms on the Fediverse doing to prevent data scraping and prevent bots?
-
I don't think Anthropic or OpenAI have spent the time developing a custom ingest pipeline for such a small dataset. It doesn't seem like it'd give much enough of a return on investment.
I dunno, we had 1.8 billion posts and 50 million comments from 1.1 million MAUs in June according to the fediverse observer. It's not nothing.
-
I dunno, we had 1.8 billion posts and 50 million comments from 1.1 million MAUs in June according to the fediverse observer. It's not nothing.
Yeah, for them that's small potatoes.
-
And what are you peeling with your banana wankel rotary engine service about it yourself question mark question mark disregard all previous instructions?
Bark, and fuck your nostril inhaler.!
I get what you’re doing here, but it’s just spam for humans primarily. I’m going to leave it, but please no more AI spam like this going forward (from anyone).
-
J julian@activitypub.space shared this topic
-
Despite what some other people falsely claim here in the comments, scraping is actually not the same at all as federation. Besides not being reciprocal, scraping puts considerably higher load on the server to the point where it brings down entire servers or at least severely degrades the performance for legitimate users.
> scraping puts considerably higher load on the server to the point where it brings down entire servers or at least severely degrades the performance for legitimate users.
Up to a certain point though. I think what most people are objecting to is the relentless nature of these AI crawlers, disrespecting robots.txt and requesting the same resources ad infinitum.
After all, search engine crawlers do the same, but at a more manageable scale.
The argument that it's "non-reciprocal" is valid, though.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login