← FinOpsAid blog

Who tells us when GitHub spend jumps? Anomaly detection beyond a threshold

Most teams find out about a spend spike from the invoice, three weeks after the thing that caused it stopped being fixable. Here's what anomaly detection on GitHub actually requires — a baseline that understands your week, severity that survives alert fatigue, and attribution attached to the alert.

A workflow gets a retry loop on a Tuesday. It burns runner minutes for nine days. Nobody notices, because nothing about a GitHub org looks different when a workflow is failing expensively — the repo is green-ish, the dashboards nobody opened are fine, and the meter just runs. You find out when the invoice arrives, by which point the money is spent and the person who merged the change has moved on to three other things.

So the question a finance partner eventually asks isn't "why was the bill high", it's the sharper version: who was supposed to tell us? That's a detection problem, and the reflex answer — set a budget alert — solves a different problem than the one you have.

Thresholds catch magnitude, not surprise

A static threshold fires when a number crosses a line. That's genuinely useful for one thing: protecting a commitment you've made to finance. It's poor at noticing that something changed.

Two failure modes, both common. A threshold set high enough not to nag you sits above your normal variance, so a 40% jump inside the envelope never fires at all — you only hear about the spike if it also happens to breach the ceiling. And a threshold set tight enough to catch real movement fires every time your month has five Mondays in it, which trains everyone to dismiss it.

The deeper issue is that a threshold has no memory. It doesn't know that your Actions spend always dips over a holiday week, or that you onboarded thirty engineers in March, or that this repo has been trending up steadily for a quarter and today's value is exactly on trend. All of that context is what separates "high" from "surprising", and only the second one is worth waking someone up for.

What "anomalous" actually means here

The workable definition is statistical rather than absolute: a value is anomalous when it deviates from what its own recent history predicted, by enough that chance is an unconvincing explanation.

In practice that means computing a baseline per series — this repo's Actions minutes, this team's Copilot usage, this SKU's storage — and expressing today's value as a deviation from it. A z-score is the plainest version: how many standard deviations from the recent mean. Two is interesting, three is usually real, and the size of the deviation gives you a severity ranking for free.

Two properties make this better than a line in the sand. It's self-calibrating, so a noisy series needs a bigger move to trigger than a stable one, which is exactly right. And it's scale-free, so a small repo doubling its spend can rank above a large repo drifting up 5% — which is often the correct priority, because the small doubling is a defect and the large drift is a budget conversation.

The baseline is the hard part

Everything above depends on having a history worth comparing against, and this is where most homegrown attempts stall.

You need enough of it. A fortnight of data produces a baseline so wide that nothing is ever anomalous, or so narrow that everything is. You need it to understand your week, because a CI series has strong weekly seasonality and comparing Saturday to a flat all-days mean generates a false positive every weekend. You need it to tolerate a trend, or every month of a growing organization reads as an anomaly. And you need it to survive one-off events — a migration week that triples runner usage will poison a naive mean for a month afterwards unless it's handled.

There's an honesty point here too. If your data lands via a nightly sync, your detection is next-day, and it should say so. That's not real-time and shouldn't be dressed up as it. Next-day is still roughly three weeks earlier than the invoice, which is the comparison that matters — the runaway workflow gets nine hours instead of nine days.

What actually trips on GitHub

The catalogue is fairly stable across organizations:

A workflow that starts retrying on failure and doesn't stop. A matrix that expands when someone adds an OS or a language version, quietly multiplying every run. A larger runner label applied "temporarily". A self-hosted runner fleet scaled up for a load test and never scaled down. Codespaces that never hit their idle timeout because something in the container keeps them warm. A bulk seat assignment during onboarding. An AI-credit series that jumps because one team pointed an agent at a repository and let it run.

Notice that almost every item is a change someone made deliberately, with no intent to spend money, and no feedback loop telling them they had. That's the gap detection fills. It isn't catching bad actors, it's closing a loop that GitHub's own billing surface leaves open.

An alert nobody can act on is noise

Detection is half the job. The other half is that the person reading the alert must be able to do something before they've finished reading it.

That means the alert carries attribution — which repo, which workflow, which team, which SKU — not just "Actions spend is anomalous". It means it carries the comparison that made it anomalous, so the reader can judge it in one glance: this repo ran 4,200 minutes against a 900-minute baseline. And it means severity is real, with low-severity deviations aggregated into a digest rather than paged, because attribution is what turns a number into a decision and volume is what turns an alert channel into a muted one.

A working setup

• Baseline per series, not per org. Org totals are too smooth; the interesting failures hide inside them.
• Respect the week. Day-of-week seasonality is the single biggest source of false positives on CI data.
• Rank by deviation, act on the top few. A ranked list you read is worth more than a complete list you don't.
• Attach the cause to the alert. Repo, workflow, team, and the baseline it broke.
• Keep thresholds as well. They're the right tool for a commitment you must not breach; anomalies are the right tool for everything you didn't anticipate. Run both.
• Review the misses monthly. The spike you found in the invoice is the best possible input to your detection rules — Actions spend in particular rewards this.

FinOpsAid computes deviation-scored anomalies with severity across your cost series nightly and surfaces them on Alerts and the Overview dashboard, each one attributed to the repo, workflow, or team that moved — with the baseline it broke shown next to it, and an as-of timestamp so you know exactly how current the finding is.

Frequently asked questions

What is the difference between a cost threshold and an anomaly?

A threshold catches magnitude — a number crossed a line you drew. An anomaly catches surprise — a value that is unusual against your own recent history, whatever its absolute size. Thresholds miss a doubling that stays under the limit; anomaly detection catches it.

Why does a cost baseline need to understand the week?

Because engineering activity is weekly. CI minutes collapse at weekends and spike on release days, so a flat baseline flags every Monday and misses a genuine mid-week jump. A baseline that does not model the weekly shape produces alerts nobody trusts within about two weeks.

Why didn't anomaly detection catch our gradual cost increase?

Because a trailing baseline follows you. A cost that creeps up a few percent a week is absorbed into the baseline and never flagged as unusual. Anomaly detection catches step changes; gradual drift is a trend question and needs a period-over-period comparison instead.

Want to hear about the runaway workflow tomorrow instead of at month end? Connect your GitHub org — read-only and free during beta — or explore the demo dashboard first.