Lemmy.world has been down between 02:00 UTC and 05:45 UTC. This was caused by the database spiking to 100% cpu (all 32 cores/64 threads!) due to inefficient queries been fired to the db very often.

I’ve collected the logs and we’ll be checking how to prevent this. (And what caused this)

  • cerevant@lemmy.world
    link
    fedilink
    English
    arrow-up
    9
    arrow-down
    6
    ·
    11 months ago

    I can’t claim to know what the designers intended, but having users spread across a large numbers of servers is terribly inefficient for how Lemmy works: each server maintains a copy of each community that it’s users are subscribed to, and changes to those communities need to be communicated across each of those instances.

    Given this architecture, it is much more efficient and robust to have users concentrate on what are effectively high performance cacheing servers, and communities spread out on smaller, interest focused instances.

    • ewe@lemmy.world
      link
      fedilink
      arrow-up
      4
      arrow-down
      4
      ·
      11 months ago

      Yeah, I think this is the way things should move in the future. Have community vs user focuses on servers instead of having the same server get hit with both high community/comment usage and a server with lots of login/audit/user browsing requests. Servers with big communities could focus on stability and perfomance. Servers with users could focus on cool UIs and features for their users.