36
Database Corruption Maintenance (00:01 - 06:00 UTC Dec 2) (stacks.programming.dev)
submitted 2 years ago by [M] to c/meta@programming.dev
20 comments fedilink hide all child comments
 

Programming.Dev is going to be going down starting at around 00:01 UTC Monday Dec 2 to attempt to fix the database corruption that happened relating to communities.

This has been affecting attempting to look at the pages for certain communities (it would end up loading forever instead of actually showing the posts for the community) and was affecting federation in some cases.

This is the second attempt to fix the issues with the first one needing to be rollbacked from due to the fix not working.

This fix could take several hours, so the outage window is going to be from 00:01 - 06:00 Dec 2.

you are viewing a single comment's thread
view the rest of the comments
[–] 7 points 2 years ago (9 children)

The fixes are done for tonight, but are not 100% complete. I estimate that 99.4% of cases have been fixed though. You might see problems with individual posts, but every community should be fixed. I've seen at least 1 issue with the !science@lemmy.world community and I'll be investigating that if it doesn't self resolve (stuff could still be catching up).

IF YOU SEE ANY MORE ISSUES PLEASE REPORT THEM HERE.

  • permalink
  • fedilink
  • source
  • hideshow 9 child comments
    •
  • [–] 1 point 2 years ago* (last edited 2 years ago) (2 children)

    Edit: Definitely not relevant, it was a problem with the link I clicked

    original

    Not sure if it's related but I've noticed a couple of smaller communities from other instances which aren't accessible from programming.dev any more, Imaginary Witches and Imaginary Fairies. They give the very helpful error "unknown", this might be a fun one to try to track down. They're accessible on the source sites (1, 2) and other communities from those instances seem to work.

    Screenshot displaying error page with text "The server returned this error: unknown."

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 2 years ago (1 child)
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 2 years ago

    Ah, you're right. I think I see what happened there, I hit those links from a sidebar in a community with href="imaginarywitches@lemmy.dbzer0.com"which breaks if you're on a post page instead of the community page. Easily fixed though, I'll ping the mods of that community.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 2 years ago (3 children)

    https://programming.dev/c/programmer_humor, https://programming.dev/c/teach_programming, and https://programming.dev/c/data_structures still seem to be completely broken. Might need to investigate what's happening for those as well.

    CC: @Ategon@programming.dev

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 2 years ago* (2 children)

    Wait !programmer_humor@programming.dev is broken for you? For me in Jeroba that's one of the communities that's been working just fine this whole time, unless the recent maintenance broke it the last post I have is from yesterday.

    EDIT: Ok, so clicking your link, or even my link gives an error saying community can't be found but I can still pull it up from my subscriptions, so it was either broken by this maintenance or something weird is going on

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago

    For me in Jeroba that’s one of the communities that’s been working just fine this whole time, unless the recent maintenance broke it the last post I have is from yesterday.

    That's strange indeed, the community has been inacessible from the web UI for weeks

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 years ago

    I can access many of the broken communities from external links and I might even be able to create posts in them from the UI but I cannot access their links directly from the UI, and they also don't load from my mobile App.

  • permalink
  • fedilink
  • source
  • parent