I think we need to have a little more sympathy for GitHub. You could justify the jabs when we could all blame any outage on the migration to Azure, but then they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI.
I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers. Yes, they need to get this figured out, but a little sympathy goes a long way. I personally wish them the best and hope their on-call people can go back to getting normal amounts of sleep soon.
Sympathy? It's a Microsoft company that is being ran with a consistency of a startup in early seed rounds. Their downtime is abhorrent and unacceptable as far as enterprise goes. Their engineers look like absolute amateurs allowing for such low class work it results in their customers experiencing industry leading downtime.
It's a shame that the GitHub org that we use at my job that we pay a lot of money for gets affected the same way my personal nonsense does.
I don't know the architecture or any of that, but I feel like there could be (and it's not like they would've really known this until the last year or two with the massive spike) separate infrastructure for paid users/orgs vs free the same way they make the distinction with enterprise.
I get the massive load changes that they are under over the last two years, but why does a bunch of vibe coded slop take down the same resources that my company pays for every single month and has for years? I imagine properly splitting that out would be an absolute headache and not worthwhile for them vs stabilizing the rest of the service, but damn it sucks when I get blocked at work because GH is down.
Our GitlabCE instance has been sitting in the racks for nearly two years with almost 100% uptime running on 10 year old xeons that have long since paid for themselves.
Of course it is not free of all management but for our use case it is working.
There are hiccups with the CI runners from time time but nothing major and we have another machine in another rack that serves as a backup which can be brought up ~< 20 minutes.
I know companies have long since tossed their expertise for hosting their own stuff in favour of SaaS but at some point its hard to beat the up time of a single machine.
They decided they needed to capture the whole open source ecosystem by turning open source work into social networking... on a proprietary platform (because open source is great, especially when it's others' software). That was before they joined Microsoft.
And then Microsoft pushed AI everywhere, including on GitHub itself with copilot.
I would have liked if they had left the open source projects alone and didn't create that FOMO for not using them.
idk man, we pay stupid amounts of money to microsoft, we are an enterprise customer, expecting better availablity compared to my laptop isnt really a high bar.
Give me a break. Sympathy? For microsoft? That might have flown when github was like seven people, but they have nearly unlimited resources to make it better. They're just choosing not to. Let's talk contracts and money before we pull the sympathy card.
I used to be on-call in a high-traffic environment where single customers pushed more bits than entire nations. I chose the role. I didn't want people's sympathy, if anything, I wanted them to complain to management.
If it gets too bad they can quit. Maybe that would be for the best, just wear the thing down until it outright fails and no one wants to touch it. One less bullshit service sucking all of the oxygen out.
Ahhh, I think the author mixed up two values here. That value seems to actually be the average over the past 3 months.
const incidents = e.detail.incidents;
// ...snip...
const now = new Date();
const threeMonthsAgo = new Date(now);
threeMonthsAgo.setMonth(threeMonthsAgo.getMonth() - 3);
// ...snip...
var currentFreq = recentIncidents.length / 3; // <- We out here, smoking these guns with our homeboy Claude
// ...snip...
var earliest = null;
for (var j = 0; j < incidents.length; j++) {
var dd = new Date(incidents[j].started_at); // <- eventually incidents[j].started_at is "2016-03-01T07:07:37.000Z"
if (!earliest || dd < earliest) earliest = dd;
}
// ...snip...
document.getElementById('n-since').textContent = earliest
? earliest.toLocaleDateString('en-US', { month: 'long', year: 'numeric' })
: '?';
document.getElementById('n-rate').textContent = Math.round(currentFreq * 10) / 10;
#n-since is going to be either march or feburary. It'll change depending on your timezone because JS's Date object always shifts the date around to match the same instant but in the system's timezone.
#n-rate has nothing to do with the #n-since month, it's just the last trailing 3 months. And even then, it's sort of underbaked? It's moving the date back by 3 calendar months not taking into account differing numbers of days, so it'll under-report short months.
I wouldn't trust the stats here.
Edit: whoops, author updated the template while I was writing this! It now says "Over the last 3 months", though that's still calendar months.
Not sure whether it has been updated since your comment, but the sentence now reads:
GitHub has had 1125 incidents since March 2016. Over the last 3 months, they've averaged 24 incidents per month
edit: although they also have 1.2 days of downtime (in a day) for their 'worst days' of downtime table, which suggests some auto number crunching is not working as expected.
Yes I tweaked it! The number and copy were mismatched and are no longer!
That worst day is likely an overlapping incidents accounting issue; I tried to account for overlapping incidents in another view but probably failed to port it over there.
Getting rid of Actions and Copilot and other secondary services almost halves Github's incident rate: https://i.imgur.com/XPcMIFr.png
I'm a big fan of Github Actions and I think people are often a little too harsh on it, but it's clear that it's sad that it's come at such a high cost to the platform's stability
GitHub Actions is really, really badly designed. The security model is fundamentally broken, the YAML hell is as bad as any, the log streaming lags like hell, they charge self-hosted runners for using their coordination plane, jobs queuing is really slow, and their software for actually running jobs is cursed and designed in a way that is practically hostile to self-hosting.
GitHub Actions is god-awful. Have you ever used any other CI tools?
It’s ironic that the SCM that 2026 software engineers landed on, is one with a complicated distributed usage model and is slow/down because of a centralised service
An important thing to consider is how much of their uptime without incidents is not the normal working hours. Their incident-free uptime on 9-5 EST, Mon-Fri, is probably like 60%.
Hey! isgithubcooked.com is my site; I made this site in February '26, I think
The contribution graph as an outage calendar idea is a commonly recurring one :). I definitely saw it somewhere else as a static asset first before I made this site.
I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers. Yes, they need to get this figured out, but a little sympathy goes a long way. I personally wish them the best and hope their on-call people can go back to getting normal amounts of sleep soon.
https://this.weekinsecurity.com/microsoft-wins-lamest-vendor...
I don't think this is commendable at all. I give GitHub a lot of money and I'm tired of it being wasted with downtime.
I don't know the architecture or any of that, but I feel like there could be (and it's not like they would've really known this until the last year or two with the massive spike) separate infrastructure for paid users/orgs vs free the same way they make the distinction with enterprise.
I get the massive load changes that they are under over the last two years, but why does a bunch of vibe coded slop take down the same resources that my company pays for every single month and has for years? I imagine properly splitting that out would be an absolute headache and not worthwhile for them vs stabilizing the rest of the service, but damn it sucks when I get blocked at work because GH is down.
Of course it is not free of all management but for our use case it is working.
There are hiccups with the CI runners from time time but nothing major and we have another machine in another rack that serves as a backup which can be brought up ~< 20 minutes.
I know companies have long since tossed their expertise for hosting their own stuff in favour of SaaS but at some point its hard to beat the up time of a single machine.
They decided they needed to capture the whole open source ecosystem by turning open source work into social networking... on a proprietary platform (because open source is great, especially when it's others' software). That was before they joined Microsoft.
And then Microsoft pushed AI everywhere, including on GitHub itself with copilot.
I would have liked if they had left the open source projects alone and didn't create that FOMO for not using them.
I have no sympathy.
I can have sympathy for the humans caught in the crossfire but only managing one nine of availability on a commercial service is not acceptable.
But we pay enterprise license and GitHub is a big dependency in our software flow.
If this continues to be a problem as an enterprise product they need to do something. Otherwise theyre are going to to start losing business
I used to be on-call in a high-traffic environment where single customers pushed more bits than entire nations. I chose the role. I didn't want people's sympathy, if anything, I wanted them to complain to management.
If it gets too bad they can quit. Maybe that would be for the best, just wear the thing down until it outright fails and no one wants to touch it. One less bullshit service sucking all of the oxygen out.
1125 incidents / 126 months ≈ 8.9 incidents per month, not 24
still terrible, but why such an obvious error in the first sentence...
#n-rate has nothing to do with the #n-since month, it's just the last trailing 3 months. And even then, it's sort of underbaked? It's moving the date back by 3 calendar months not taking into account differing numbers of days, so it'll under-report short months.
I wouldn't trust the stats here.
Edit: whoops, author updated the template while I was writing this! It now says "Over the last 3 months", though that's still calendar months.
That worst day is likely an overlapping incidents accounting issue; I tried to account for overlapping incidents in another view but probably failed to port it over there.
Should be fixed soon!
Looks like they fixed it already
I'm a big fan of Github Actions and I think people are often a little too harsh on it, but it's clear that it's sad that it's come at such a high cost to the platform's stability
GitHub Actions is god-awful. Have you ever used any other CI tools?
This is why I foresaw that centralizing everything to GitHub was just generally a bad idea 6 years ago. [0]
Now that there is no CEO of GitHub, there is no point to GitHub improving.
[0] https://news.ycombinator.com/item?id=22867803
And it may be even worse in EDT, which is currently in effect!
I was ready to click...
https://red-squares.cian.lol/
The contribution graph as an outage calendar idea is a commonly recurring one :). I definitely saw it somewhere else as a static asset first before I made this site.