← All posts

FinOpsAid blog

Copilot's metrics now flag their own blind spots. Read those before you read the rest.

"Is the AI code review worth what we pay for it?" "Why does a third of our Copilot seats show zero lines of code?" Until this month, GitHub's own metrics couldn't answer either question cleanly.

November's releases don't settle them, but they get much closer. The most useful change isn't a new number. It's a pair of flags that tell you which users' numbers are incomplete. Here's what shipped between November 3 and 6, and the one thing to keep in mind throughout: metrics and billing are separate ledgers.

What changed in November

Date Change Where Data source
Nov 3 Local BYOK usage and enterprise custom models reported separately API, user and aggregate reports Local BYOK: client telemetry. Custom models: server-side
Nov 5 Code review addressed rate and merged-without-change-request count API, repos-1-day reports Pull request outcomes
Nov 5 client_telemetry_present and version-requirement flags API, user reports and daily counts Server vs. client comparison
Nov 6 Chat and VS Code Agents window in feature engagement Dashboard and 28-day reports Client telemetry

All four need the Copilot usage metrics policy turned on, and an enterprise owner, billing manager, organization owner or a custom role with View Copilot Metrics. None of them backfills. Report days before each release simply don't have the new fields.

Start with the users your metrics can't see

On November 5, GitHub added client_telemetry_present per user per day (GitHub Changelog, November 2026). false means GitHub saw the user's Copilot activity, but no usage events arrived from their editor. A second field, usage_metrics_requirements, marks each IDE and extension as met, unmet or unknown against the minimum versions.

This matters for cost per seat. Those users still count as active, but their rows lack lines of code and feature and model breakdowns. Divide "lines accepted" by "active users" and every telemetry-dark developer drags the ratio down. Your ROI looks worse than it is, and the seats look idle when they aren't. Outdated editors, disabled IDE telemetry and corporate proxies are the usual causes.

GitHub's caveats are specific. Don't add missing_client_telemetry_user_count and below_usage_metrics_minimum_user_count together, because one user can be in both. Don't sum either across days to count unique users. A null means "unavailable", not false. And meeting every version minimum doesn't guarantee telemetry, since a proxy can still block it.

BYOK is usage that isn't on your GitHub bill

On November 3, the usage API started separating two kinds of usage that don't use GitHub-hosted models (GitHub Changelog, November 2026).

Local BYOK covers requests from Copilot CLI or the Copilot app that go straight to a developer's own provider key or a local model like Ollama or LM Studio. New fields include used_local_byok, totals_by_local_byok (split by byok_kind and surface) and daily, weekly and monthly active BYOK user counts. It's client telemetry, so it undercounts when telemetry is off. Some local providers don't report tokens, and it isn't broken out by model.

Enterprise custom models are admin-configured and routed through GitHub. Each totals_by_client[].totals_by_model[] entry now carries is_custom_enterprise_model. GitHub sees these requests on the server, so they're counted from every client, including IDEs.

The finance point is about the bill, not the metrics. A request sent to a developer's own provider key is paid on that provider's invoice, which may be a corporate API account or a personal card. Either way it isn't in GitHub's AI credits. So a team can look like light Copilot spenders while running heavy agent sessions on someone else's meter. used_local_byok is the first way to find that from GitHub's side.

Where each kind of Copilot request is counted and where it is billed Three rows. Copilot-hosted models are counted server-side in totals_by_client and billed as GitHub AI credits. Enterprise custom models are counted server-side and flagged with is_custom_enterprise_model; GitHub's changelog does not say how they are billed. Local BYOK requests from Copilot CLI and the Copilot app are counted from client telemetry in totals_by_local_byok, can undercount, and are billed by the developer's own provider or cost nothing for a local model, so they never appear in GitHub AI credits. COUNTED WHERE · BILLED WHERE REQUEST TYPE METRICS FIELD ON THE GITHUB BILL? Copilot-hosted model server-side count totals_by_client Yes — AI credits Enterprise custom model server-side count, all clients is_custom_enterprise_model Not stated by GitHub Local BYOK (CLI, Copilot app) client telemetry · can undercount totals_by_local_byok No — provider's invoice Local BYOK is a subset of the CLI and Copilot app totals. Existing totals didn't change.
BYOK usage shows up in GitHub's metrics but is paid somewhere else. Field names and data sources from GitHub's November 3, 2026 changelog. The billing column is our reading. GitHub doesn't describe custom-model billing in that entry. Retrieved 2026-11-14.

Code review finally has an outcome metric

Copilot code review consumes AI credits on every review. Until now, the only signal of value was total_copilot_applied_suggestions, which counts one-click suggestions someone applied. On November 5, the repos-1-day reports gained a copilot_code_review object (GitHub Changelog, November 2026).

Divide total_comments_addressed by total_comments_evaluated and you get an addressed rate per repository. "Addressed" is broad on purpose: applying the suggestion, asking Copilot to fix it, editing the code by hand, or deferring the fix to a later pull request all count. There are breakdowns by severity, review tier, comment type and customization, plus total_merged_no_human_change_request_after_review.

Two timing details matter when you set this against spend. Comments are counted on the day their pull request merges, not the day Copilot posted them, while credits are billed when the review runs. A month's addressed rate and a month's review credits won't cover the same reviews. Also, bot-opened pull requests such as Copilot cloud agent's usually aren't evaluated, though they do count toward merged-without-change-request. If the review tier breakdown shows deeper reviews getting addressed no more often than standard ones, that's a cost conversation worth having. Our post on code review effort levels covers the billing side.

The impact dashboard counts chat now

On November 6, the impact dashboard's Agent adoption chart added Copilot Chat and the VS Code Agents window, making nine experiences. The 28-day reports gained chat and vscode_agent in copilot_feature_engagement.totals_by_feature (GitHub Changelog, November 2026). "Passive users" are now called "Chat-first users".

It's a renaming, not a re-segmentation. Adoption phases are unchanged, chat-first users still appear as No Cohort in the API, and cohort sizes don't move. Engagement means using a feature on at least two days in 28, and chat includes agent mode, so one user counts under several features. Don't add the features together to get a user total.

Our finding: FinOpsAid reads GitHub's Copilot metrics reports, but we don't parse any of these November fields yet. That covers telemetry flags, BYOK, custom-model markers and the code review object. Our AI credit figures come from billing, which has no BYOK line at all, because BYOK spend isn't GitHub's to bill. The telemetry flag is the one we'd prioritize, because it's the fix for a known distortion: users who look idle by lines of code but are active by billing.

What to do with this

  1. Pull client_telemetry_present before your next seat review. Don't reclaim a seat from someone whose editor just isn't reporting.
  2. Check used_local_byok against your provider invoices. BYOK traffic is real spend, just not on GitHub's bill.
  3. Track the addressed rate by review tier. Set it against code review credits by quarter, not month, because of the merge-day lag.
  4. Don't sum feature engagement. Chat includes agent mode, so the same users appear in both.
  5. Reconcile cost from billing. When metrics and invoice disagree, the invoice wins.

Where FinOpsAid fits

FinOpsAid reads your org read-only and shows Copilot credits from billing next to adoption from metrics, side by side rather than blended, on the Copilot page. The honest limits: per-user credits have no model, client or BYOK breakdown, we don't parse the November fields yet, and GitHub's metrics lag two to three days. Related: October's usage metrics changes, measuring Copilot ROI, and finding idle Copilot seats.

Frequently asked questions

Does Copilot BYOK usage appear on my GitHub bill?

Local BYOK requests go straight from Copilot CLI or the Copilot app to a developer's own provider key or local model, so they're paid on that provider's invoice, or cost nothing for a local model. Since November 3, 2026, the usage metrics API reports them in totals_by_local_byok and used_local_byok, from client telemetry that can undercount.

What is the Copilot code review addressed rate?

It's total_comments_addressed divided by total_comments_evaluated in the repos-1-day usage metrics report. A comment counts as addressed if the developer applied it, asked Copilot to fix it, changed the code by hand or deferred the fix. Comments are counted on the day the pull request merges.

Why do some Copilot users show activity but no lines of code?

Usually their editor isn't sending client telemetry, because of an outdated IDE or extension, disabled telemetry, or a proxy. Since November 5, 2026, client_telemetry_present is false for those users, and usage_metrics_requirements shows which IDE or extension is below the minimum version.

Sources

Want billing credits and adoption metrics side by side without blending the two? Connect your GitHub org — read-only and free during beta — or explore the demo dashboard first.

copilotmetricsai-creditsroigovernancefinops

See it in your own org

FinOpsAid meters the GitHub spend no invoice itemizes — read-only, and free during beta.