So I usually hate sending over bluesky links anywhere since I'm not a fan, but Rob Ricci highlights some important distinctions between #activitypub and #atproto
-
So I usually hate sending over bluesky links anywhere since I'm not a fan, but Rob Ricci highlights some important distinctions between #activitypub and #atproto
Fundamentally, activitypub is designed to be useful with a partial view of the network. (Narrowing down to #mastodon here) Likes are communicated, mentions are communicated, and so on.
Over at atproto land, you have no option but to write to your repo, and so you have no choice but to view the entire network to say, find replies and such (narrowing to #bluesky here). So the only way you can get your data is to read the network and filter it yourself, or have someone else do it for you, which doesn't solve the problem, only shift it.
-
So I usually hate sending over bluesky links anywhere since I'm not a fan, but Rob Ricci highlights some important distinctions between #activitypub and #atproto
Fundamentally, activitypub is designed to be useful with a partial view of the network. (Narrowing down to #mastodon here) Likes are communicated, mentions are communicated, and so on.
Over at atproto land, you have no option but to write to your repo, and so you have no choice but to view the entire network to say, find replies and such (narrowing to #bluesky here). So the only way you can get your data is to read the network and filter it yourself, or have someone else do it for you, which doesn't solve the problem, only shift it.
https://overreacted.io/there-are-no-instances-in-atproto/
While having an extremely snobbish and arrogant tone, I think it makes some good points about separating hosting from viewing, which I think the usual #fediverse sort of conflates. And while the doesn't doesn't cover it, #atproto also has the unique feature of your data being relatively portable, largely absent in #fediverse.
-
https://overreacted.io/there-are-no-instances-in-atproto/
While having an extremely snobbish and arrogant tone, I think it makes some good points about separating hosting from viewing, which I think the usual #fediverse sort of conflates. And while the doesn't doesn't cover it, #atproto also has the unique feature of your data being relatively portable, largely absent in #fediverse.
On the other hand, the post largely glosses over the actual cost of running it, which Rob addresses above.
In fact, it entirely skips relays, which are so fundamental now, and in particular, that every #atproto app effectively needs to consume one.
(If your atproto app doesn't consume from a relay, it HAS an embedded relay, and in that case, god help your app's compute consumption).
-
On the other hand, the post largely glosses over the actual cost of running it, which Rob addresses above.
In fact, it entirely skips relays, which are so fundamental now, and in particular, that every #atproto app effectively needs to consume one.
(If your atproto app doesn't consume from a relay, it HAS an embedded relay, and in that case, god help your app's compute consumption).
Having said that, and also disliking the fact that it came out of corpo billionaire creators of twitter, I genuinely believe there's some good stuff to pick up from that protocol.
In particular, I think the coolest part of the architecture is server-independent (more like server-migratable) identity, which is kind of amazing.
Activitypub/fediverse should adopt https://helge.codeberg.page/fep/fep/ef61/ soon, and I feel like that'd be even better than what atproto has if I understand correctly.
#activitypub #fediverse #atproto #bluesky #mastodon #federated #protocols
-
Having said that, and also disliking the fact that it came out of corpo billionaire creators of twitter, I genuinely believe there's some good stuff to pick up from that protocol.
In particular, I think the coolest part of the architecture is server-independent (more like server-migratable) identity, which is kind of amazing.
Activitypub/fediverse should adopt https://helge.codeberg.page/fep/fep/ef61/ soon, and I feel like that'd be even better than what atproto has if I understand correctly.
#activitypub #fediverse #atproto #bluesky #mastodon #federated #protocols
-
FWIW, I didn't personally find this article to be particularly snobbish or arrogant. I think it's quite well written, and while it does have a particular viewpoint and is written with a strong voice, I think it's making a good faith attempt to explain something that is a frequent point of confusion/contention when people are discussing the fediverse and atmosphere.
However, I think it uses a metaphor that's flawed enough that it's not really helping; here's my part of the thread on that:
https://bsky.app/profile/ricci.io/post/3moogiuvkjc2e
In short, article positions the data storage (PDS) as the blog in its analogy, but I think the PDS is actually more like the backend database for the blog. When people familiar with the fediverse and Web in general ask "why are there not more atproto instances" they are not asking why there are not more databases with people's raw blog posts in them, they are asking why all the blogs are served through a small number of websites.
This particular exchange, I think, helps to clearly explain the difference in how Dan and I are looking at these networks:
-
On the other hand, the post largely glosses over the actual cost of running it, which Rob addresses above.
In fact, it entirely skips relays, which are so fundamental now, and in particular, that every #atproto app effectively needs to consume one.
(If your atproto app doesn't consume from a relay, it HAS an embedded relay, and in that case, god help your app's compute consumption).
@innocentzero Relays are often described as an optimization in the network, and I think Dan left them out out of a legitimate attempt to keep the article relatively simple and digestible.
However, I would argue that they are a load-bearing optimization; in a version of the atproto network without them, running a PDS would be vastly more expensive because you'd have to be maintain open websockets to every single application (which is actually what Dan has drawn); this is fine if there are only a couple of applications, but scales poorly.
-
FWIW, I didn't personally find this article to be particularly snobbish or arrogant. I think it's quite well written, and while it does have a particular viewpoint and is written with a strong voice, I think it's making a good faith attempt to explain something that is a frequent point of confusion/contention when people are discussing the fediverse and atmosphere.
However, I think it uses a metaphor that's flawed enough that it's not really helping; here's my part of the thread on that:
https://bsky.app/profile/ricci.io/post/3moogiuvkjc2e
In short, article positions the data storage (PDS) as the blog in its analogy, but I think the PDS is actually more like the backend database for the blog. When people familiar with the fediverse and Web in general ask "why are there not more atproto instances" they are not asking why there are not more databases with people's raw blog posts in them, they are asking why all the blogs are served through a small number of websites.
This particular exchange, I think, helps to clearly explain the difference in how Dan and I are looking at these networks:
@innocentzero @cwebber One of the things I consistently see in these discussion is a conflation of "is cheap to run now" with "scales". This evening, when I pushed a bit more on this, the point that I got to is that people using and running this infrastructure don't seem to expect the network to grow a lot; they seem to think that it has basically reached the order of magnitude that it will be for the foreseeable future. I think this is a fine position to take, and I agree with them that things like relays and jetstreams will remain relatively affordable to run at even 10x the current network size.
But this is at odds with the more aggressive boosters who treat this as the way to build all social applications going forward.
If you think atproto will support a collection of applications that are, in aggregate, 10x the current size, sure, community infrastructure will continue to be cheap to run and the large amounts of data *someone* has to ingest for your small app or community is fine. If you think it will get to Twitter size (100x), I think you are probably wrong about this. If you think it will get to Facebook size, no way.
-
@innocentzero Relays are often described as an optimization in the network, and I think Dan left them out out of a legitimate attempt to keep the article relatively simple and digestible.
However, I would argue that they are a load-bearing optimization; in a version of the atproto network without them, running a PDS would be vastly more expensive because you'd have to be maintain open websockets to every single application (which is actually what Dan has drawn); this is fine if there are only a couple of applications, but scales poorly.
@ricci Yeah, that's sort of what I meant when I said 'every app has an embedded relay'
The apps have to pull in from the PDS in that case, which makes the app (appview?) itself vastly more compute-intensive, since relays are inherently costly, so to speak.
-
@ricci Yeah, that's sort of what I meant when I said 'every app has an embedded relay'
The apps have to pull in from the PDS in that case, which makes the app (appview?) itself vastly more compute-intensive, since relays are inherently costly, so to speak.
@innocentzero mmmm.... I think in a world without relays, it's actually the PDSes that suffer more, unless there are very few apps/appviews. The total bandwidth the app has to ingest (assuming it wants the whole network view) is basically the same whether you are pulling from a relay or from a bunch of individual PDSes; it's more websockets, which has a cost, but manageable.
However, right each PDS only has to *upload* to a fairly small number of relays and a few oddball applications that connect to it directly (actually I run my own PDS, I should check this :). Without a relay, the PDS has to serve every app individually.
I think one thing that has become clear to me in discussions this evening is that many atproto developers are assuming a world with not very many apps, why is why they are not very concerned about this.
-
@innocentzero mmmm.... I think in a world without relays, it's actually the PDSes that suffer more, unless there are very few apps/appviews. The total bandwidth the app has to ingest (assuming it wants the whole network view) is basically the same whether you are pulling from a relay or from a bunch of individual PDSes; it's more websockets, which has a cost, but manageable.
However, right each PDS only has to *upload* to a fairly small number of relays and a few oddball applications that connect to it directly (actually I run my own PDS, I should check this :). Without a relay, the PDS has to serve every app individually.
I think one thing that has become clear to me in discussions this evening is that many atproto developers are assuming a world with not very many apps, why is why they are not very concerned about this.
@ricci Ah, that makes sense, I was thinking more along the lines of relays collecting all that data and then aggregating it (do relays do it?)
Also, I have this eternal confusion on whether relays pull from PDSes or PDSes push to relays. If it's the latter, how do new relays advertise themselves to the PDSes?
-
@ricci Ah, that makes sense, I was thinking more along the lines of relays collecting all that data and then aggregating it (do relays do it?)
Also, I have this eternal confusion on whether relays pull from PDSes or PDSes push to relays. If it's the latter, how do new relays advertise themselves to the PDSes?
@innocentzero It's probably better to think of them as streaming rather than push or pull; a relay connects to a PDS, says "hey what's going on", and from that point on, every record created on the PDS (posts, likes, etc.) gets sent along that stream.
You *can* do what's called backfill, where the relay fetches older stuff too, so that the relay's clients can get that older data straight from the relay. However, this is expensive. Real expensive. So expensive that even the official Bluesky relays don't support it anymore. This is one point of common misunderstanding; back when this was the way relays worked, they were up in the ream of VC-money-required. However, this is no longer the case.
People still do backfill for some purposes. However, now, they do it by contacting the PDSes directly.
-
@ricci Ah, that makes sense, I was thinking more along the lines of relays collecting all that data and then aggregating it (do relays do it?)
Also, I have this eternal confusion on whether relays pull from PDSes or PDSes push to relays. If it's the latter, how do new relays advertise themselves to the PDSes?
@innocentzero Relays do relatively little work, in the sense that they "just" take in records from a bunch of PDS streams on one side, and spit them out on a single stream on the other side. (Actually, they spit out multiple streams, one per client; but the basic work is the same.)
Something to be aware of is that in addition to relays, there are jetstreams; these are services that *filter* the events for you. So, if you want to run a small app, one that doesn't care about all the Bluesky posts the relay is sending you, or that only wants events for a few users, or whatever, a jetstream consumes the output of a relay, and outputs just the subset you asked for.
These services have more work to do, because they have to output a different stream for every client. They also do some things that make developers' lives easier, like convert the CBOR binary format to plain JSON, and check a bunch of the crypto for you.
At current scales? Jetstreams are still not too expensive, especially if you have few clients. If you think that the atproto network is approximately done growing, all is good. If you think it's the future of all social applications, good luck running community ones in that future.
-
@innocentzero Relays do relatively little work, in the sense that they "just" take in records from a bunch of PDS streams on one side, and spit them out on a single stream on the other side. (Actually, they spit out multiple streams, one per client; but the basic work is the same.)
Something to be aware of is that in addition to relays, there are jetstreams; these are services that *filter* the events for you. So, if you want to run a small app, one that doesn't care about all the Bluesky posts the relay is sending you, or that only wants events for a few users, or whatever, a jetstream consumes the output of a relay, and outputs just the subset you asked for.
These services have more work to do, because they have to output a different stream for every client. They also do some things that make developers' lives easier, like convert the CBOR binary format to plain JSON, and check a bunch of the crypto for you.
At current scales? Jetstreams are still not too expensive, especially if you have few clients. If you think that the atproto network is approximately done growing, all is good. If you think it's the future of all social applications, good luck running community ones in that future.
@ricci ah thanks, I think I mixed relays with jetstreams. That makes sense.
-
@ricci ah thanks, I think I mixed relays with jetstreams. That makes sense.
@innocentzero yeah, there are a bunch of pieces to keep track of.
-
@innocentzero yeah, there are a bunch of pieces to keep track of.
@innocentzero By the way, Jetstreams were not, and still are not, officially standardized atproto objects. In fact, they don't speak atproto to their clients. They were originally a side project by one of the Bluesky team members, and have become a load-bearing side project for many of the apps that are not Bluesky.
In much the same way that some people are kind of irked at Mastodon for adopting some of its own APIs or doing other ActivityPub things in non-standard ways, jetstream is sort of a hack that solves real problems but does so off-protocol.
-
Having said that, and also disliking the fact that it came out of corpo billionaire creators of twitter, I genuinely believe there's some good stuff to pick up from that protocol.
In particular, I think the coolest part of the architecture is server-independent (more like server-migratable) identity, which is kind of amazing.
Activitypub/fediverse should adopt https://helge.codeberg.page/fep/fep/ef61/ soon, and I feel like that'd be even better than what atproto has if I understand correctly.
#activitypub #fediverse #atproto #bluesky #mastodon #federated #protocols
Nice thread. Note that it is better to refer to the official FEP location at https://fediverse.codeberg.page/fep/fep/ef61/
-
FWIW, I didn't personally find this article to be particularly snobbish or arrogant. I think it's quite well written, and while it does have a particular viewpoint and is written with a strong voice, I think it's making a good faith attempt to explain something that is a frequent point of confusion/contention when people are discussing the fediverse and atmosphere.
However, I think it uses a metaphor that's flawed enough that it's not really helping; here's my part of the thread on that:
https://bsky.app/profile/ricci.io/post/3moogiuvkjc2e
In short, article positions the data storage (PDS) as the blog in its analogy, but I think the PDS is actually more like the backend database for the blog. When people familiar with the fediverse and Web in general ask "why are there not more atproto instances" they are not asking why there are not more databases with people's raw blog posts in them, they are asking why all the blogs are served through a small number of websites.
This particular exchange, I think, helps to clearly explain the difference in how Dan and I are looking at these networks:
yes people are really asking about the "small number of websites" or "copies of the app" phrasing used in the thread.
I believe @cwebber said it before but the power lies in who runs the websites/apps and lessening the concentration of power is why we care about decentralization. the choices they make affect everyone using their services. and even if switching is possible, defaults persist for most users. and their experience changes them, and we share a society with them.
-
Nice thread. Note that it is better to refer to the official FEP location at https://fediverse.codeberg.page/fep/fep/ef61/
@smallcircles welp, thanks! Mb on that
-
@innocentzero @ricci somewhat relevant https://social.coop/@cwebber/116782178524774816
-
S silverpill@mitra.social shared this topic
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login