Most apps probably want to see (a) records "intended" for that app, and (b) records *about* data "intended" for that app. (think: bluesky boost of an article from a longform writing platform) (a) is straightforward with Jetstream; most apps have one or more "lexicon"s that are basically JSON schemas that they use - selecting a subset of records based on lexicon is straightforward and cheap. Right now, jetstream will let you do this, but the PDS and relays will not. I don't know of any particular reason why they couldn't however; it adds complexity, because they need to maintain different stream buffers for different clients, but I think it's doable. (b) is harder because it requires you to understand more about the records that are flying through, and, frankly, if I understand it right, the current way lexicons are used makes this quite expensive: you can reference other records by cid but that doesn't tell you what lexicon they belong to, so you have to have kept track of all of the IDs of things that you are interested in links to... basically an appview.

yes people are really asking about the "small number of websites" or "copies of the app" phrasing used in the thread.

I believe @cwebber said it before but the power lies in who runs the websites/apps and lessening the concentration of power is why we care about decentralization. the choices they make affect everyone using their services. and even if switching is possible, defaults persist for most users. and their experience changes them, and we share a society with them.

Nice thread. Note that it is better to refer to the official FEP location at https://fediverse.codeberg.page/fep/fep/ef61/

In much the same way that some people are kind of irked at Mastodon for adopting some of its own APIs or doing other ActivityPub things in non-standard ways, jetstream is sort of a hack that solves real problems but does so off-protocol.

Something to be aware of is that in addition to relays, there are jetstreams; these are services that *filter* the events for you. So, if you want to run a small app, one that doesn't care about all the Bluesky posts the relay is sending you, or that only wants events for a few users, or whatever, a jetstream consumes the output of a relay, and outputs just the subset you asked for.

These services have more work to do, because they have to output a different stream for every client. They also do some things that make developers' lives easier, like convert the CBOR binary format to plain JSON, and check a bunch of the crypto for you.

At current scales? Jetstreams are still not too expensive, especially if you have few clients. If you think that the atproto network is approximately done growing, all is good. If you think it's the future of all social applications, good luck running community ones in that future.

You *can* do what's called backfill, where the relay fetches older stuff too, so that the relay's clients can get that older data straight from the relay. However, this is expensive. Real expensive. So expensive that even the official Bluesky relays don't support it anymore. This is one point of common misunderstanding; back when this was the way relays worked, they were up in the ream of VC-money-required. However, this is no longer the case.

People still do backfill for some purposes. However, now, they do it by contacting the PDSes directly.

Also, I have this eternal confusion on whether relays pull from PDSes or PDSes push to relays. If it's the latter, how do new relays advertise themselves to the PDSes?

However, right each PDS only has to *upload* to a fairly small number of relays and a few oddball applications that connect to it directly (actually I run my own PDS, I should check this :). Without a relay, the PDS has to serve every app individually.

I think one thing that has become clear to me in discussions this evening is that many atproto developers are assuming a world with not very many apps, why is why they are not very concerned about this.

The apps have to pull in from the PDS in that case, which makes the app (appview?) itself vastly more compute-intensive, since relays are inherently costly, so to speak.

But this is at odds with the more aggressive boosters who treat this as the way to build all social applications going forward.

If you think atproto will support a collection of applications that are, in aggregate, 10x the current size, sure, community infrastructure will continue to be cheap to run and the large amounts of data *someone* has to ingest for your small app or community is fine. If you think it will get to Twitter size (100x), I think you are probably wrong about this. If you think it will get to Facebook size, no way.

However, I would argue that they are a load-bearing optimization; in a version of the atproto network without them, running a PDS would be vastly more expensive because you'd have to be maintain open websockets to every single application (which is actually what Dan has drawn); this is fine if there are only a couple of applications, but scales poorly.

FWIW, I didn't personally find this article to be particularly snobbish or arrogant. I think it's quite well written, and while it does have a particular viewpoint and is written with a strong voice, I think it's making a good faith attempt to explain something that is a frequent point of confusion/contention when people are discussing the fediverse and atmosphere.

However, I think it uses a metaphor that's flawed enough that it's not really helping; here's my part of the thread on that:

https://bsky.app/profile/ricci.io/post/3moogiuvkjc2e

In short, article positions the data storage (PDS) as the blog in its analogy, but I think the PDS is actually more like the backend database for the blog. When people familiar with the fediverse and Web in general ask "why are there not more atproto instances" they are not asking why there are not more databases with people's raw blog posts in them, they are asking why all the blogs are served through a small number of websites.

This particular exchange, I think, helps to clearly explain the difference in how Dan and I are looking at these networks:

https://bsky.app/profile/danabra.mov/post/3moovf2g2dk2u

https://bsky.app/profile/ricci.io/post/3moovkbywnc2t

In particular, I think the coolest part of the architecture is server-independent (more like server-migratable) identity, which is kind of amazing.

Activitypub/fediverse should adopt https://helge.codeberg.page/fep/fep/ef61/ soon, and I feel like that'd be even better than what atproto has if I understand correctly.

#activitypub #fediverse #atproto #bluesky #mastodon #federated #protocols

In fact, it entirely skips relays, which are so fundamental now, and in particular, that every #atproto app effectively needs to consume one.

(If your atproto app doesn't consume from a relay, it HAS an embedded relay, and in that case, god help your app's compute consumption).

#atproto #bluesky

While having an extremely snobbish and arrogant tone, I think it makes some good points about separating hosting from viewing, which I think the usual #fediverse sort of conflates. And while the doesn't doesn't cover it, #atproto also has the unique feature of your data being relatively portable, largely absent in #fediverse.

#fediverse #atproto #bluesky #mastodon #lemmy #activitypub