There have been some big developments at Bluesky lately, so I figured I’d drop an update on it, since there’s a good chance you’ll start seeing a lot from there soon. (For those who don’t know, Bluesky is a hip new federated social network in the style of Twitter. I wrote an explainer about it in July here, which is rather long but nothing in it is inaccurate. The project began in-house at Twitter intended as a way to open-source the network, then spun off into their own company.)
I’ve been excited about Bluesky for a while now because it is in many ways a piece of technology I’d been waiting for for years. They’ve learned from the failings of prior attempts at big decentralized social networks, such as Mastodon, and are focusing on meeting users where they’re at, which makes sense since its origin is literally a Twitter-built Twitter replacement. Why should you care? Well–we all know the harms that can be caused by centralized social networks run by billionaires. Some of those may be endemic to large networks, but others, such as what Musk has done to Twitter, and the very real problems this may cause in the world, are caused by centralized control. If you’re worried about Zuck or Musk or whoever having a stranglehold on communication, decentralization may be for you! People can’t own something that cannot be purchased. And while Bluesky is a company, the free and open-source software they have developed is not; “the company is a future adversary” is their motto. (See my post linked above for some key details on how Bluesky is not Mastodon.)
Bluesky offers exciting features like a free and open API, custom algorithms and feeds, multiple third-party apps and websites, account self-verification, and the overall online experience we’ve come to take for granted from a service that isn’t owned by a fascist. Still no gifs yet though.
So: recent developments!
Just yesterday was the anniversary of the first post, and they recently hit two million users, more than 200,000 of whom are active on any given day, which is pretty good.
More importantly: they’ll soon be releasing a public website for viewing posts, allegedly around the end of November. It will still be an invite-only service (although those are fairly easy to come by these days), but once that releases, it will be easily accessible to people who aren’t dweebs. I say “people who aren’t dweebs” because almost everything you do there has been public the entire time, via API. For example, here are all posts about cats in real-time. But a public website will still be a big deal. It will greatly expand the visibility of the network, presumably letting you embed posts in articles and the like. The lack of this feature has been a barrier to adoption for many people (as well as a comfort to others), so this will change the world’s relationship with Bluesky in, hopefully, a big and good way.
The second bit of important (to me, haha) news is that they’ve basically finalized the federation architecture, and plan to open the network up to third parties early next year, once they’ve released their moderation software. What will this mean for the users of bsky.social? Nothing, if the team has done their jobs correctly. For everyone else, it offers some exciting possibilities. You’d be able to run your own service, or hop onto a service run by somebody else, which integrates with bsky.social and offers an experience that is, if you want it to be, indistinguishable from being on bsky.social. Or, it could offer a totally different experience. It’s customizable on so many different layers that with any luck there will be something for everyone. They accomplish this with some interesting architectural decisions. Let’s dive into that below the fold.
There are three primary services that make up the ATProto ecosystem.
The first is simple enough–your Personal Data Server, or PDS. This is the database where all your actions live–posts, follows, likes, blocks, etc.* Everybody has one. They are meant to be as lightweight as possible.
The second is the Relay, aka BGS, aka Aggregator. This is a large and complex machine that receives notifications from PDS’s and preprocesses them for distribution. So if I make a post, my client sends it to my PDS, which signs it and stores it. Then, my PDS notifies the Relay about this post once (there is only one Relay right now, and will probably only be one for a while). The Relay updates its internals accordingly. If I’ve included an image, the Relay stores it in its Content Delivery Network for efficient distribution. Etc.
The third service is the App View. This is what provides the user experience. When I load the Bluesky app or website, it tells the App View, okay, get me what I need. The App View reads from the Relay stream and manages its own cache of user timelines; it deals with all the performance concerns and is the most complex machine in the ecosystem. (In some circumstances, a handful of requests may make it all the way back to my PDS.) Then it applies various app-level rules, for example, removing all posts by people who have blocked me. Finally it sends them to my phone or browser, which displays them to me. (Muted words, filtering of NSFW content, etc. happen on my device.)
This architecture addresses many of the problems with things like Mastodon and Nostr, which rely heavily on node-to-node communication. By making efficient aggregator Relays the centerpiece of the network, they’ve struck a fine compromise between distributability and usability, in my opinion. The downside is that only people with money can run a Relay, though this is in some ways its own form of protection against petty drama.
Another method for minimizing petty drama is that it is/will be very frictionless to change your host. Your PDS’s data is certified by two cryptographic keys–the one that the host service has, and the one that overrides it, which you have. If you don’t like your host, you can just move your data by signing a message to that effect with your recovery key. Presumably there will be an easy-to-use service that provides this functionality by the time they launch federation. Contrast with Mastodon, where moving your data requires the consent of your former host and your new host, and only moves your followers, not your posts. When you migrate a PDS, you migrate everything, more or less instantly.
And what will federation look like? The main Relay will begin processing messages from PDS’s that are not owned by bsky.social. By default, I suspect, you will log into the same website, read from the same Relay, and pass through the same AppView, as everybody else. But this is all completely client-configurable. If bsky.social starts serving ads, you can switch to an AppView or client that blocks them. Don’t want to see content from fascism.net? AppView or client, assuming the Relay even accepts their posts. Don’t want to see pictures of hedgehogs? The protocol currently supports freeform labeling of posts and users, though to my knowledge it’s only used for NSFW, spam, etc. In the future, there will be labeling services you can subscribe to, that might say “oh this picture is of a hedgehog” or “this person is a hedgehog enthusiast”. At the AppView or client layers, the labelers you’ve subscribed to, and your rules about those labels, will be applied.
And the cool thing is, as of this week, it’s all been released to production. Everybody has their own PDS database that lives in one of ten bsky.social-managed hosts. The PDS’s were migrated out of a monolithic database via the recovery keys. Each PDS broadcasts user events to the bsky.social Relay. The bsky.social AppView reads from the Relay and sends things to your phone. Turning on federation might be as simple as flipping a switch. And that’s where the rubber will hit the road (or the shit and the fan), and Bluesky will sink or swim.
The bsky.social PDS hosts are all named after mushrooms. I ended up on shiitake, which is the least-cool one (I was hoping for puffball or inkcap), but what’re you going to do? Word on the street is that this naming scheme is a reference to a nonsensical viral tumblr post where a mushroom, when threatened, says, “you cannot kill me in a way that matters.” That is the hope for this network. The technology is out there; it’s free; it’s running right now; they’re opening it to everyone soon. It won’t matter who owns Bluesky PBLLC or whether Jack sits on the board. No one person, we hope, can ruin this thing in a way that matters. It’s billionaire-proof by design, after all.
Stay tuned!
—
*PDS records are stored in a single Merkle tree, a self-certifying data structure that theoretically makes distributing updates easy.
Giving Thanks Recipe Exchange: Leslie Jones Brings Order To The Holiday
Yutsano
You lost me in the vast majority of this post. However, if it’s going to make the things run better I’m all about that!
EDIT: GOT A FRIST AGAIN!
EDIT 2: FYWP.
Major Major Major Major
@Yutsano: The hope is to make things run differently. The sort of jokey term for it is Web0, since the tech stack is inspired much more by IRC than e.g. Facebook–unlike most other twitter alternatives that have come out in the last couple of years.
Baud
Cool. Above my head, but I have my fingers crossed that it’ll be a success.
Gin & Tonic
@Major Major Major Major: I haven’t followed the development, but indeed, from your description I was thinking of IRC or even FidoNet. In tech, everything repeats eventually.
narya
I’ve been using Bluesky–but very much appreciated this post (and bookmarked your previous explainer) because I know nothing. I DO know that, although I thought Mastodon might be the way to go, and I do go there too, I am finding that more folks I follow are heading to Bluesky, and the current “feel” of the place is more similar to what I liked about the bird site.
dmsilev
@Major Major Major Major: Honestly, the distributed/federated architecture sounds very Usenet. So, maybe Web(-1) or thereabouts…
Not being run by Elon Musk is definitely a plus.
pacem appellant
I am less enthusiastic about relays. IMO it suffers the same problem as Twitter, namely, a big corporation owns my data (or the distribution of it). For now, I’m sticking to Mastodon, though I admit the node-to-node model is burdensome and not great for latency. I am less concerned about petty drama than maybe the average Internet denizen. I don’t understand how relays will solve that though (I genuinely don’t! I am happy to be educated on this matter).
Major Major Major Major
@pacem appellant: I think the most likely outcome (assuming it doesn’t fall on its ass) is that the big corpos will have Relays on offer (and competition for users will help minimize enshittification and drama), plus one or two big ones run by Wikimedia or the Internet Archive or something like that. And then smaller purpose-built subnetworks will have more manageably sized ones.
ChrisSherbak
Excellent ‘splainer. I wasn’t aware of the underlying architecture but that all makes sense to me. I’m still a bit befuddled on how things will end up specifically compared to Mastodon where “a thousand flowers (servers) bloom” – and where B’sky seems to be more constrained, by the community if nowhere else. The geekier (and queerer) communities I follow on Mastodon like it there – many of them have been running IRC and Fido nodes for a very long time and this harkens back to that and they tend to be more comfortable with that level of control/distribution. (There has been some falling out with Reddit so maybe the communities there will migrate to – or at least be more exclusive to Mastodon.)
I have 5 invites avail (as of 11/17 1800 cst) if there is interest.
Baud
How do the people running these things make money, or pay for expenses at least?
piratedan
I was a serious doubter in bluesky (and still have some reservations) but Major^4 is correct that I have not seen much in the way of Dorsey and meddling by attempting to be “free-speech” absolutists in regards to “you will accept and see this fascist tripe!”.
Blocking is pretty easy, if someone pisses you off, you block them, if there’s someone who reveals themselves to be a Jill Stein supporter that you followed, you can unfollow them. You can customize your feed to follow certain themes and add them to a list of items to be culled into your feed or you can troll thru the discovery feed and pick and choose what interests you. You can also search for certain persons and follow them.
Tools are coming and you can also use them to find who you follow and who they follow, there’s also ways to compare who is missing from your feed that others you trust follow. It’s still early enough that the personality fiefdoms are relatively small. You can also tweak your feed to ignore or engage with those that like your posts/responses. Like anything else… it takes time. You didn’t build your twitter experience in one day or one week, this will be the same… you’ll find conversations and topics that will engage you or bore you and you can adjust your feed to match that.
the beating back of fascists and asshats is a real thing there and the platform being invite only helps reduce that kind of traffic.
Another Scott
Interesting. Thanks for the update.
As far as “not ruining it in any way that matters…” I think one has to be cautious. I know it’s not the same – really, I do – but the recent adventures with RHEL indicate that companies with deep pockets and lots of lawyers shouldn’t be under-estimated. They can mess with “open source” stuff if they want.
Network effects are real, and nothing makes a MotU’s mouth water more than giant network effects and the potential to have a choke-point on them (maybe like the One True Relay?). Technology is controlled by people – we have to see how the people twist it and run with it.
I wish them well.
Thanks again.
Cheers,
Scott.
Major Major Major Major
@ChrisSherbak: there will be a lot of PDS hosts and probably a lot of app views, but the idea is they’re all different looks at the whole network, so yeah it’s a little different. That’s assuming you want to federate with Bluesky, nobody is saying you have to. You could make Reddit with ATProto.
Major Major Major Major
@Another Scott: yeah I’ll probably be throwing down for a nonprofit relay next year assuming things take off, it’s absolutely a necessity for the vision to work.
@Baud: the relay is the only really expensive bit. Bluesky PBLLC has a couple of services they offer but are mostly sitting on their investor funds right now while they get this all out the door. After that, yeah it’s an extremely important question! Dovetails with my response to Scott—getting some clarity there and some backup options available is key.
CaseyL
I haven’t been on BlueSky, even though I signed onto the wait list way, way back when. I didn’t follow up, just signed onto Mastodon, where I’ve been very happy, but do miss a few folks who have moved to BlueSky rather than Mastodon; or tried Mastodon, decided it wasn’t for them but maybe BlueSky was.,
I have asked WaterGirl for one of the invite codes BJ is collecting. It will be interesting to see how the two sites differ (and how they don’t).
pacem appellant
@Major Major Major Major: Thank you!
I actually like the idea of diversifying social media. I hate that FB or (formerly) Twitter just dominate. I miss the days of special purpose fora. BS and the Fediverse, I hope, will nudge the needle back toward that.
Baud
I know the front pagers here will want to know if BlueSky will have embed codes like Twitter has.
ColoradoGuy
As a Never-Twitter non-user, I’m looking forward to something more civilized. Sounds like a good platform with serious thought behind it.
piratedan
@CaseyL: here… you can use this:
bsky-social-udm65-grnby
twbrandt
I’m on both bluesky and mastodon and find value in both. The masto instance is a small local one which concerns itself mostly with local politics and the personal interests of its members. It’s a nice friendly place.
bluesky is more like early twitter. Many interesting people are there and it is increasingly important for real-time updates on breaking news.
I have some bluesky invites – email twbrandt at gmail if you want one.
Major Major Major Major
@Baud: I’d be surprised if it didn’t. Surely the goal is to increase relevance ahead of the grand opening.
ChrisSherbak
@ChrisSherbak: thanks @twbrandt for the suggestion on how to let people know.
Let my know if you want a BS invite code via csherbak at gmail.
piratedan
sorry for simply sharing a code, but we’re all jackals here and even tho we have delusions of being a top 10,000 blog, I can’t discount the possibility that certain ne’er-do-wells could be trolling our conversations looking for codes…
same is true if someone is in need of a code… you can reach me at piratedan7 via gmail.com if you desire a code.
CaseyL
@piratedan: Woo hoo! I’m in!
Thank you ! Mille grazie!
Excuse me while I set stuff up….
Major Major Major Major
@CaseyL: I’m @thearchduke.bsky.social!
Baud
@Major Major Major Major:
I iz following you.
Major Major Major Major
@Baud: you and some other lurkers in this thread haha
CaseyL
@Major Major Major Major: Thanks, I was just looking for you :)
Yutsano
@Major Major Major Major: Gotcha good sir!
EDIT: AsianGRRLMN is on there as well!
Major Major Major Major
Hello new followers, I strongly recommend you subscribe to my “tankies and shit” moderation list. You can find it in the “lists” tab of my profile. When you subscribe you can set it to auto-mute or auto-block the users. I promise you they are all terrible. Mostly authoritarian communists, or pro-Russia “leftists”.
Baud
@Major Major Major Major:
lol. I want to keep my experience positive.
ETA: oh, I just realized you were suggesting a block list.
Major Major Major Major
@Baud: and this hides them!
Baud
@Major Major Major Major:
Yeah, took me a second.
Chetan Murthy
It looks like this architecture can achieve the goal of separating ads/safety(labeling) from distribution. But (as we say in the transaction-processing biz) a pub/sub system is, in the end, a database. Unless they’ve found a way to partition that pub/sub system, they don’t have a scalable design.
Also, BlueSky may adopt a more-limited notion of conversation and visiblity than Twitter did.
These together form a notion of “conversation visibility”, and they’re a big part of what makes Twitter work. It’s possible for a BGS to compute these visibility relations, so that when a user’s PDS queries a BGS, it will receive all these tweets. But that’s not trivial, and I’ve certainly seen nothing that tells me the Bluesky folks have thought about this.
But even before that, just sticking with capability #1, this still requires a design for scaling the BGS. Maybe they’ve got one. Maybe they haven’t. It’s not trivial. I guess we’ll have to wait and see. I do know that Mastodon ActivityPub protocol isn’t properly scalable, but maybe the Bluesky folks are doing a better job.
Alison Rose
I don’t understand what about 75% of this post is saying, but I just want to know that the Bluesky app I was happy to use (well…I signed up and have followed a handful of folks and, uh…blued? skied? blied? skued? a few times) will stay basically the same and be what I wanted something to be, which is basically Twitter without Muskrat and the rest of the bigot boyz. I don’t want fancy nerd shit that I don’t understand. I just want the app as it is, but hopefully one day with gifs.
Baud
As someone who doesn’t use Twitter, what’s the difference between mute and block?
Betty Cracker
Thanks so much for this!
PS: I have some invite codes if anyone is interested. Email me at bettycrackerfl at gmail dot com, and I’ll hook you up.
Alison Rose
@Baud: Mute means you don’t see their tweets. Block means they can’t see your account.
Bill Arnold
@piratedan:
I expect that spidering automation is regularly trawling sites for “bsky-social-.*” codes. Maybe not for the code then “then prepend bsky-social-” though; that might be safer.
Baud
@Alison Rose:
Thanks!
Major Major Major Major
@Chetan Murthy:
So you say this any time anybody writes about Bluesky and at this point I just don’t think I understand what you mean. The PDS —> Relay(s) communication is not a bottleneck any more than any other database write, and the Relay is not unpartitionable any more than any other black box API service. They’re currently benchmarking as ready for 10m users on the main relay architecture now that they’ve moved on-prem.
Martin
I’m pleased by the federation approach to reduce the chance for a Musk to swoop in an nazi the place up, but I think there are some much more important changes that Mastodon brought that seem trivial but wind up being important. As much as I miss shitposting, retweeting really turns out to undermine a lot of the utility of the platform. It can be fun and harmless, but it can also be a very efficient form of harassment.
I suspect that engagement and harassment are sides of the same coin – you can’t drive the former without incurring the latter, so you need to have very aggressive forms of moderation to keep the place from flying apart. I’ll be curious to see how Bluesky progresses in that way.
Doug R
Not a fan of invite codes. Seems a little “No-Homers-Club” to me.
And not really a way to be inclusive.
Major Major Major Major
@Doug R: it’s because it’s still under development and will be irrelevant soon anyway.
piratedan
@Bill Arnold: understood, methinks we’re okay since CaseyL grabbed the one I sent her, so that code should no longer be valid.
Alison Rose
@Baud: Oh duh, now I realize maybe it works different on Bluesky. But I believe it’s the same???? Someone please correct me if I’m wrong.
MazeDancer
BlueSky is very calm. Which is good.
One tip – You have to follow people in order to see anything. Media, friends, people you like. Gotta follow to see them. There is no firehose feed.
There is a Home+ feed which widens to include people followed by people you follow. M4 clued me into feeds. You can tab them at the top of your feed. I like Gardening.
Not sure when the public can view, what they will see as they won’t be following anyone.
One of the best parts of BlueSky is @darth is there. Except recently went into hibernation. Popehat took refuge there. Cats of Yore and The Herd also post.
I am FifthHouseSun there, same as Twitter.
Alison Rose
@Doug R: Every user ends up with invite codes, and they now seem to increase quickly. I’ve barely used the app but I have 5 codes.
Major Major Major Major
@Alison Rose: that’s correct! The only difference is that blocks are public.
Major Major Major Major
@MazeDancer: Ah, cats of yore, my finest invite.
Central Planning
@dmsilev: I used to run the Usenet/NNTP server for a large manufacturer in town. The whole thing became a clusterfuck when corporate got involved. I quit shortly after that (not because of the NNTP thing)
Alison Rose
@Major Major Major Major: Yeah, I noticed it said that. What exactly does that mean? Like…do they get a notification that you blocked them?
piratedan
and…. if you need to follow me (hah!) I am under the handle of Piratedan7… give me a follow and I will kindly reciprocate.
Baud
Will I be able to do all my banking on BlueSky? That’s a must have with any social media account.
Martin
@Doug R: There aren’t a lot of good ways to throttle access to a service that you are trying to scale. Scale is a physical activity – it takes time. Signing up is almost instantaneous. Too much interest can kill the service badly enough that it never recovers.
I overall agree with the sentiment, but I also don’t have great alternatives.
piratedan
@Baud: there are multiple filters available, pants, pants-less, sans pants and not applicable….
Jackie
Off topic, but this came up on my FB feed:
I wish the Carters peace and comfort while surrounded by their loved ones.
Major Major Major Major
@Alison Rose: it means that’s it’s accessible via API. It’s needed so that all the App Views know who has blocked who, due to the relay architecture. Which means somebody made a website where you can look it up…
CaseyL
Still setting up, trying to find people.
Interesting thing I’m seeing in my Mastodon feed is some people are leaving for BlueSky because Mastodon is getting too shouty. It’s not just the tone police (which are a PITA but not, for me, a reason to leave) but certain topics have certainly brought out the angry. Left wing angry, as opposed to Right Wing angry, but still angry…and apparently as allergic to hearing facts they don’t like as the Righties.
Sigh. We can have nice things… just not for long.
ETA and PS: If anyone’s looking for me, I’m CaseyL there. I’m basically CaseyL everywhere.
Martin
Tangentially related, I’m enjoying the housecleaning over at OpenAI.
Mousebumples
I’m Mousebumples there, like Twitter and Mastodon ([email protected], though I’m not there as much anymore).
I have been blocking anyone who looks like a follow bot – eg following tooooons of people. Idk if they’ll end up doing some AI something or other, but I’m not interested, lol.
Bluesky question – have to Retweet (or Resky, whatever the verb is?) to share content with your followers? Does liking a post do anything other than give a dopamine bump, maybe?
MazeDancer
Was hoping Flying Toaster would come on and mention their Balloon-Juice Blue Sky feed.
It’s lovely the Flying Toaster worked so diligently to create it.
The only draw back is people – like me – have different Balloon-Juice and BlueSky nyms. So you don’t always know to whom you are speaking.
Alison Rose
@Major Major Major Major: I’m sorry to sound dense, but I still don’t fully get it. Someone could still view my account through a third-party site?
CaseyL
@MazeDancer: I’m also finding that Search on BlueSky doesn’t always work, when you put in the username. That may be due to user name not always being the same as the account name, though.
MazeDancer
@CaseyL: The best and easiest thing to do to find people is go to other people’s home pages and click on “Followers”.
Scroll their list, follow people you like.
Do that with several people. You’ll have a good feed in no time.
Major Major Major Major
@Mousebumples: liking doesn’t shove stuff into the feed like it does on twitter but it affects some of the recommendation feeds for your friends since “what do your friends like” is a common input.
@Alison Rose: yes, but that’s always been true and will be quite explicit when they release the read-only public view soon. What I mean is that who you block is technically publicly available, but you have to muck around with the back end API or know somebody who does.
CaseyL
@MazeDancer: Yes, I started doing that – it works great!
Mousebumples
Thanks for clarifying. ❤️
Another follow tip – I’ve used this more for sports – when something Big Happens, search a keyword (eg Giannis, Lillard, etc.) and follow lots of people talking about that topic. And unfollow later if they end up not being too your taste.
Alison Rose
@Major Major Major Major: Well, I don’t really care if someone knows that I blocked them, especially since I’m a nobody so I doubt they will care either.
MazeDancer
@Major Major Major Major: The whole of BlueSky thanks you. Early and often.
twbrandt
@twbrandt: due to a brain fart, I posted an obsolete email address. Email me at tom at tombrandt dot net if you want a bluesky invite.
Major Major Major Major
@CaseyL: yeah the search is atrocious, they keep teasing a fix…
Chetan Murthy
@Major Major Major Major:
Partitionability of a service depends on the data-model of the data inside the black box. That is to say, some things will need to be replicated, some can be partitioned; some might need to be somewhat replicated. So for instance, in the TPC-C benchmark, the “catalog” is replicated, the orders are partitioned. But this depends on clients being able to connect directly to the partition where the order they’re interested in resides. If they have to connect to every partition to find the order of interest, then the system isn’t scalable.
So in a similar way, with a social network, the *tweeting user* sends their new tweet to a single partition; that partition must compute which other partitions the tweet must be sent to, so that *following users* can connect only to the partition they usually connect to — and not to the partition of every user they follow. B/c the latter would result in an unscalable system (users follow >100 other users, etc, etc).
So even as *externally* the BGS looks like a black box, the *implementors* and *designers* of the BGS *must* address this issue, and not doing so means they haven’t actually solved the problem of how to partition/scale Bluesky.
What I mean is, unless they lay out the scaling design for the BGS, they haven’t actually solved scaling. Just kicked the can down the road.
I can illustrate with a more-involved example of how scaling/partitioning a data-model is neither trivial nor obvious, if you’re interested.
laura
I appreciate this information and your efforts to marshall the data and reassure social media luddite/lurkers like myself that better communities are possible and while not conceding the Twitter verse despite the hostile takeover underway. And yet, no smol orange-stripy smol bean who is smol, and no fog-pelted, enigmatic gentleman of the old school. What gives?
Major Major Major Major
@Chetan Murthy: no I’m well aware of how partitioning works, what isn’t clear to me is how you can tell they haven’t solved the “partitioning issue” without knowing anything about how the relay works. You’re just assuming they don’t know how to manage a web service at scale. I seem to remember them saying they weren’t going to open source the bsky.social BGS architecture, which leads one to conclude they think it’s their competitive advantage… confirming.
Chetan Murthy
@Major Major Major Major: A very talented computer scientist once told me “solve the hardest part of the problem *first* — so you know the problem is actually solvable, before attacking the easier bits”. Scalability is the hardest part of building a twitter replacement: it was true of the original twitter (the original engineering team understood when they were in a single room that if they didn’t scale, they’d end up like friendster — dead from failure to scale). If they understood this at Bluesky, they’d have had scalability on their roadmap, and have written down their design.
You’ll note that the BGS isn’t federable — it’s a central node. That’s the same as just declaring surrender to the problem and going home. Because a “federated” system in which the central node is special …. isn’t federated: it’s a centralized system.
rosalind
for the techies: i signed up for Bluesky and have been mostly reading posts, not writing anything much. then last week when i tried to access the log in page it brought up a blank page. i’m on an imac, and have tried both FF and Safari with same result. same thing is happening on my macbook w/safari. i’ve made no changes on my end. no clue what’s going on, but extremely frustrated i can’t even access to delete the account. if anyone has any ideas…
Major Major Major Major
@Chetan Murthy:
Anybody who wants to can open another relay.
Chetan Murthy
@Major Major Major Major:
Also, any partitioning scheme would have to address my points 1-4 (they are a large part of what makes twitter useful today). And doing so is absolutely not trivial.
Major Major Major Major
@Chetan Murthy: again, your argument literally seems to be that it can’t be done because you haven’t seen them do it. You’re just assuming they don’t know how to manage a web service at scale.
Chetan Murthy
@Major Major Major Major: I haven’t seen any writeup whatsoever where it’s even *addressed*. This isn’t trivial stuff, and when you don’t see people even *talking* about a problem when their system has been deployed for many months, it’s a good indicator that they haven’t thought about it.
An example (this isn’t what Bluesky is trying to do, note): During the scaling of Twitter, at least *twice* they fell down because they failed to scale their core network properly. I was there and heard their plans for the runup to their first failure, and ….. they were not thinking about the problem correctly at all. There’s a phrase: “full bisectional bandwidth” that must be uttered by anybody trying to scale an Internet service like Twitter, and nobody was saying that phrase. That was enough to know that they weren’t going to succeed. And they didn’t.
Here’s another example: does the BGS know the entire social graph? If it doesn’t, then you can’t partition it. If it does, then …. when you actually deploy Bluesky to the world, the social graph will be too large to easily fit on a single machine: it *also* will need to be partitioned. But to do so, you have to do it by replicating bits *also*. Again, this isn’t trivial, and somebody that was looking ahead to actual scalability would have thought this thru and actually documented the design considerations, and requirements.
Major Major Major Major
@Chetan Murthy:
How about the fact that this is literally how the application currently works? Also, your PDS never queries anything, the AppView does.
So we’re back to “I’m not privy to their internal documentation, therefore they’ve never thought about this thing that every web engineer with more than a year professional experience knows is a big issue.”
Why don’t you just join the discord and ask them instead of taking incurious potshots? You could even offer to help if you’re so inclined. https://discord.gg/5ApkdzpY
Chetan Murthy
@Major Major Major Major:
Both the ActivityPub and Atproto folks have made obvious mistakes. I’ll give you two:
I’ve seen enough TP systems fail to scale due to mistakes in their design, that when I see people not address these sorts of issues in a system that’s supposed to replace something that’s been around for 15yr and used worldwide, it’s a fair bet that they simply didn’t think about it.
Major Major Major Major
@Chetan Murthy: this literally started as a team at twitter, I genuinely can’t figure out why you think they are this dumb about how things like twitter work. Anyway I think you should join the discord and share some of your deep knowledge! Especially if you’d like to help people have a Musk-free twitter alternative.
Chetan Murthy
@Major Major Major Major:
That cannot scale. What we’re talking about is the construction of the “home timeline” and the “tree of replies”. And it must be done at the BGS, not by the client or AppView. It has to be done in a “push” manner, not a “pull” manner, b/c that’s the only efficient way to do it. This was the key realization at both Twitter and Facebook, back in about 2007 — that you treat the home timeline as a materialized view, and update it as new tweets come in. So that you never have to do the underlying queries to build the home timeline from-scratch.
Major Major Major Major
@Chetan Murthy: join the discord! Share your concerns with somebody who matters. Anyway, if this is something Twitter and Facebook figured out decades ago, it’s really weird that the team from twitter never thought about it.
ETA also I’ve literally had discussions with the devs about this precise question, good lord.
Chetan Murthy
@Major Major Major Major: If you know the email of somebody involved, I’d be happy to start an email convo with them. I tried to do so with the lead for the ActivityPub protocol: sent a nice email saying I had some concerns about the design, and would like to discuss them: got nothing back.
The problem with discord is that it’s useless for design discussions. Too noisy. It’s like slack in that sense.
Major Major Major Major
@Chetan Murthy: I don’t have anybody’s email, but if you change your mind, discord link is in an earlier comment, or you could apply to a backend job. If nothing else that will get you somebody’s attention. The discord has some very knowledgeable people and of course the team members.
https://blueskyweb.xyz/join/backend-developer
Chetan Murthy
@Major Major Major Major:
That’s not a good sign: Twitter did a terrible job of their database: perhaps you are aware of the gynormous debacle that was the Cassandra (and then their Manhattan) database deployment? And their sharded Mysql datastore. I could go on. I’ve known a number of twitter systems folks who quit in disgust b/c the place was so poorly-run.
ETA: In May 2008 IBM lent me to Twitter b/c they’d been down off-and-on for nearly a year (and daily for several months that spring), so I could help them get back onto the road and going straight. So I got to see a lot of what was wrong there, and made enough friends that over the years I was privy to the design and deployment failures of a bunch of their stuff. I’ll note that Twitter never built a federated system: they built a *clustered* system (the two are very, very, very different) so it’s actually not at all the case that somebody who knows how to build a clustered system can design a federated one.
For example, it’s pretty clear that Twitter (and fbook) are evolved to favor a clustered design. Deploying either on a federated system is much more inefficient, involving a significant amount of replicated data that isn’t there in a clustered design.
Major Major Major Major
@Chetan Murthy: this is either a problem twitter started tackling in 2007 and has since solved to your satisfaction, or it isn’t. You’re saying both. ETA makes more sense with your edit. Join the discord!
Chetan Murthy
@Major Major Major Major: I’m saying that Twitter never had to solve the problem of a federated social network design, so you can’t assume that Twitter engineers just intrinsically know how to do it. Hell, they didn’t do a good job of their *clustered* design, either. Their Cassandra, Manhattan, and Gizzard systems were all a mess.
Too an-anj
@ChrisSherbak: Delurking to say “I’d like a BlueSky invite” if you have any left. Thanks!
Chris T.
Mostly idle questions: How do you know which mushroom you’re on? And why would one care, for that matter?
Major Major Major Major
@Chris T.: you can muck around in the backend, or follow jazbot and post “!jazbot whereami”
Why care? Because it’s fun of course.
Brachiator
Snagged an invite, signed up and am looking around.
I think Popehat is over here, now.
Another Scott
@Chetan Murthy: Thanks for the under-the-covers story. They’re always interesting.
Made me look… BlueSkyWeb.xyz (from May 5):
That’s uncomfortably hand-wavy to these eyes, and I know almost nothing about this stuff.
Thanks again.
Cheers,
Scott.
Ron
Bluesky is run by billionaire jack, who is a big supporter of RFK jr. Youre just helping the bottomline of another anti democracy, anti science asshole who isnt quite as big an asshole as musk.
He was also the asshole that refused to boot trump off twitter.
Meet the new boss. Same as the old boss.
Major Major Major Major
@Another Scott: fortunately that’s from may and they’ve since built it and are now using it. It’s also inaccurate, they’re very bad about documentation
Major Major Major Major
@Ron: Bluesky is not run by Jack, he actually hates it so much he deleted his account. But I’m also confused by what people want. Consensus seems to be that twitter 2013 or so was great and everybody wants that back.
Another Scott
@Major Major Major Major: I was getting at (or at least thinking about) the federated vs clustered design aspects. If there’s only One True Relay at the moment, then breaking it up into a federated system of relays seems like a big change. Their post above doesn’t really say how they are going to make it work.
I guess we’ll see soon enough.
Thanks.
Cheers,
Scott.
Major Major Major Major
@Another Scott: ahh, I gotcha. I don’t believe the plan is to break it up into federated relays, per se. Under the current spec I’m not even sure if they’re meant to communicate with each other. There’s been some crosstalk on that front lately… as always I will bring updates here when they’re concrete!
Ruckus
@twbrandt:
I’m in!
FlyingToaster (Tablet)
@MazeDancer: Sorry to be late to the party! Meatspace precluded being online last night.
Feeds are the equivalent of Twitter Lists; back on that hellsite I had a buncha lists including Jackals. You can program your own via Bluesky’s API, or use any of the services to build one — Blueskyfeeds.com and GoodFeeds are the two I’m most familiar with.
I’m @jasiglar.bsky.social, click the flying toaster icon to get to my profilce, and click “Feeds”. All of the feeds that I subscribe to are listed, and you can subscribe to any or all or none.
My Balloon Juice Jackals feed is hand curated; to be added, just @ me in a skeet. Because I’m a parent, logistics manager at Chez Toaster, and it’s a fucking holiday, this may take a few days.
This poast was interrupted thrice by baking. And next is braving the ravening hordes at Hannaford, Shaws, Stop&Shop.
Kristine
@Major Major Major Major: Done
Tim in SF
My problem with BlueSky is my engagement is total shit on that platform. Post after post after post, and I get nothing. Nobody follows me. It’s like I’m talking into a well at midnight. BlueSky is stingy twitter.
Threads, on the other hand, got me up to almost 600 followers in a couple months, and I get interaction on everything I post. All my comments get seen.
It sucks because I like the UI on BlueSky way better than Threads.
Major Major Major Major
@Tim in SF: Bluesky is a lot like early twitter, you have to just reply to people a lot when you’re getting started.
RaflW
@Tim in SF: I’m chosing to just never go to Threads, even though I have a Bookface acct. I’m that squidged out by Zuck.
I admit that having a Xit get 30,000 views and hundreds of likes (on occasion) was a smidge addictive, and Bsky doesn’t scratch that itch for me at the moment.
But it’s friendly, so far, and reminds me of about 2011 or 12 when I had a handful of Twitter followers, and a few dozen follows, and it was interesting and not antagonistic feeling.
We’ll see if it lasts.
(And thanks Major X4 for the assist over there just now!)
nickdag
@Major Major Major Major:
Really appreciate these semi-technical posts about Bluesky.
I’m also glad you commented briefly about Jack. When I heard of his involvement with Bluesky, that gave me a bad taste in my mouth. However, I really no next to nothing about him, his history, or his involvement in Bluesky.
For your next post, if you have more info on Jack’s (non-)involvement or his history with creating (or enshittening) Twitter, please include that. I’d love to get more context on all that.
Thanks!