Bluesky April 2026 Outage Post-Mortem

(pckt.blog)

59 points | by jcalabro 2 hours ago

6 comments

  • threecheese 1 hour ago
    > What I had missed is that we deployed a new internal service last week that sent less than three GetPostRecord requests per second, but it did sometimes send batches of 15-20 thousand URIs at a time. Typically, we'd probably be doing between 1-50 post lookups per request.

    That’ll do it.

    • 98codes 1 hour ago
      Ahh, the three relevant numbers in development: 0, 1, and infinity.
    • bombcar 1 hour ago
      Zero, one, many, many thousands.
  • goekjclo 29 minutes ago
    > The timing of these log spikes lined up with drops in user-facing traffic, which makes sense. Our data plane heavily uses memcached to keep load off our main Scylla database, and if we're exhausting ports, that's a huge problem.

    I expect this is common.

  • jonstaab 3 minutes ago
    nostr never goes down
    • pfraze 1 minute ago
      All support to other decentralizers but nothing never goes down.
  • rvz 18 minutes ago
    Thank you for the post mortem on this outage.
  • templar_snow 55 minutes ago
    [flagged]
  • jmclnx 25 minutes ago
    Lite Blue on a dark Blue background. That is a new one, I have seen grey text on lite grey, but blue on blue ?

    The article does work in lynx, at least I can read it.