@angrybards @innocentzero To be perfectly honest, I think that atproto fragmenting into a one-app-per-firehose model might actually be good for it. The idea that the 'default' way to read data from the network is to read all records produced by everyone on every app on the network - well, it sounds ridiculous just saying it, doesn't it? You ain't gonna build the next Web that way, are you?

Most apps probably want to see (a) records "intended" for that app, and (b) records *about* data "intended" for that app. (think: bluesky boost of an article from a longform writing platform) (a) is straightforward with Jetstream; most apps have one or more "lexicon"s that are basically JSON schemas that they use - selecting a subset of records based on lexicon is straightforward and cheap. Right now, jetstream will let you do this, but the PDS and relays will not. I don't know of any particular reason why they couldn't however; it adds complexity, because they need to maintain different stream buffers for different clients, but I think it's doable. (b) is harder because it requires you to understand more about the records that are flying through, and, frankly, if I understand it right, the current way lexicons are used makes this quite expensive: you can reference other records by cid but that doesn't tell you what lexicon they belong to, so you have to have kept track of all of the IDs of things that you are interested in links to... basically an appview.

@angrybards @innocentzero Frankly, I don't know. I do know that PDSes and relays generally have rate limits built in, so I suspect that has discouraged development of any chatty apps, but again, I don't know. Large payloads in the form of images, etc. are handled via "blobs" that are just URLs to the place they are stored off-protocol. I don't know what the protocol or practical limits on records that are on-protocol are.

I just did a full 5 hour long stream on this blog post and more, I'm currently editing the video down for publication.