this post was submitted on 30 Jun 2023
333 points (93.9% liked)

Lemmy.World Announcements

29084 readers
268 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages πŸ”₯

https://status.lemmy.world/

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to [email protected] e-mail.

Report contact

Donations πŸ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 2 years ago
MODERATORS
 

We'll give the upgrade new try tomorrow. I've had some good input from admins of other instances, which are also gonna help troubleshoot during/after the upgrade.

Also there are newer RC versions with fixed issues.

Be aware that might we need to rollback again, posts posted between the upgrade and the rollback will be lost.

We see a huge rise in new user signups (duh.. it's July 1st) which also stresses the server. Let's hope the improvements in 0.18.1 will also help with that.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 0 points 1 year ago* (last edited 1 year ago) (1 children)

There should be substantial performance improvements because it moves Lemmy from using websocket to HTTP API.

Websockets largely have a lower compute cost per request, HTTP requests are slow and expensive vs just firing off data in an already established TCP connection, so this isn't tracking for me?

Was it just the overhead of managing the websockets? Shouldn't an API gateway be doing that anyways...?

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (2 children)

The websocket implementation was streaming federated data to the UI in real-time, and was responsible for a bunch of bugs/UX issues that wouldn't have been evident when there were less people on Lemmy (such as scrolling down New posts on the homepage and seeing them zoom off the screen and back a bunch of pages as new federated posts from busy instances rolled in)

Some instances were also hitting the open socket limit of their reverse proxies IIRC, which was causing some users to get stuck on a spinning loader indefinitely.

I personally used an app instead because of the bugs/ UX issues caused by the websocket implementation, but since the 0.18 update on my instance the site is so much nicer to use.

Edit:

Websockets largely have a lower compute cost per request, HTTP requests are slow and expensive vs just firing off data in an already established TCP connection, so this isn’t tracking for me?

Most browsers send keep-alive headers to request that the webserver keeps the TCP connection open - I believe for HTTP/2 and HTTP/3 the connection is automatically held open regardless. It doesn't address the overhead caused by resending headers etc, but it's faster than establishing a new connection each time, and 'manages itself' by closing the socket automatically after a period of inactivity.

Websockets are still undisputably faster though, but I think it all comes down to implementation

[–] [email protected] 2 points 1 year ago (1 children)

Great explanation, thanks!

You seem to be knowledgeable here, and I'm just onboarding to Lemmy. Is it possible for others to contribute distributed compute/networking/storage resources to Lemmy instances? (Kind of along the lines of Kafka)

I have a cluster that I largely use as nodes for various projects that I enjoy. I'd be more than happy to provision a few VMs to be nodes if such a concept exists here πŸ€”

[–] [email protected] 1 points 1 year ago

No problem πŸ˜€

I don't think it's possible to contribute distributed resources in a normal sense, since the ActivityPub protocol being used is server-to-server (If I make a comment on this post from lemmy.one, the lemmy.one server sends that to the lemmy.world server, which will update it's 'master' copy of the post, and then send that update to other federated servers). There's a spec page that explains this a lot better than me if you're interested: https://www.w3.org/TR/activitypub/#Overview

There might be a way for you to contribute some of your spare compute power in a way that I'm not aware of (maybe one of the underlying technologies, like postgres for the db, or pictrs which is used to handle images?) but I'm not familiar with how that could work

So far, the closest thing would be mods from large reddit communities spinning up their own Lemmy instance for their users to migrate to (like programming.dev from r/programming, and lemmy.dbzer0.com from r/piracy) which helps distribute the users across servers in general without needing to necessarily explain the benefits of doing that or federation in general.

As for supporting instances in general, most are accepting donations (links usually available in the sidebar shown on an instance's homepage) and using that money to upgrade their instance specs, or moving to a dedi server

[–] [email protected] 1 points 1 year ago

Great explanation, thanks!

You seem to be knowledgeable here, and I'm just onboarding to Lemmy. Is it possible for others to contribute distributed compute/networking/storage resources to Lemmy instances? (Kind of along the lines of Kafka)

I have a cluster that I largely use as nodes for various projects that I enjoy. I'd be more than happy to provision a few VMs to be nodes if such a concept exists here πŸ€”