What to Actually Measure in a .NET Service

IMeterFactory is the right way to create meters in a modern .NET service — it participates in dependency injection, it's testable, and it manages meter lifetime for you. Code Maze has a good walkthrough of the mechanics.

The mechanics are the easy part. The hard part, and the part I've watched teams get wrong repeatedly, is deciding what to measure. Most services I've inherited had metrics. Very few had metrics anyone used during an incident.

The Test for a Good Metric

One question: when this moves, do I know what to do?

A metric that changes and prompts no action is decoration. It costs storage, it costs dashboard space, and worse, it costs attention during exactly the moments attention is scarce.

The metrics people actually use at 3am are boring and few. Request rate, error rate, latency distribution, saturation of whatever resource is scarce. Dashboards full of counters that nobody has ever acted on are a sign a team instrumented what was easy rather than what was diagnostic.

Start With Four Numbers

For any request-serving component:

Rate — how much traffic. The denominator for everything else, and the first thing you check when something looks wrong, because a doubled error count against tripled traffic is a different story.

Errors — how many failed, split by kind. A validation rejection and a database timeout are both "errors" and mean entirely different things. If your error metric can't separate "the caller sent something invalid" from "we are broken", it can't drive a decision.

Duration — as a histogram, never as an average. An average latency hides everything worth knowing. Your p99 is the experience of your most frustrated users, and in a claims pipeline the p99 is often a specific payer with unusual payload sizes. The average tells you none of that.

Saturation — how close the constrained resource is to its limit. Connection pool usage, queue depth, thread pool starvation. This is the leading indicator; the other three are lagging. Saturation tells you about the incident you're going to have.

Get those four right for each meaningful component and you've covered most of what you'll want during an outage.

Cardinality Will Get You

The most expensive mistake in metrics is a tag with unbounded values.

Every distinct combination of tag values creates a separate time series. Tag by endpoint — a few dozen series, fine. Tag by customer — hundreds, probably fine. Tag by claim ID, user ID, or anything per-request and you've created millions of series, and you will discover this via a bill or an outage in your metrics backend.

The rule: tag values must come from a small, bounded, known set. If you can't write down the complete list of possible values, it isn't a tag. Per-request identifiers belong in traces and logs, which are built for high cardinality. Metrics are for aggregates.

This is worth stating explicitly in a code review checklist, because the mistake looks completely reasonable when you make it — tagging by claim ID feels like exactly the detail you'd want.

Business Metrics Are Worth More Than System Metrics

The instrumentation I'd fight hardest to keep isn't CPU or memory. It's the numbers that describe whether the system is doing its job.

Claims submitted per hour. Adjudication outcomes by category. Queue age for items awaiting review. Time from submission to resolution.

These catch a class of failure that system metrics never will: everything healthy, nothing working. CPU normal, latency normal, error rate zero — because a queue stopped being consumed, or an upstream stopped sending, and every remaining request is being served perfectly. System metrics are blind to that. A throughput metric with an alert on "unusually low" catches it in minutes.

The most valuable alert I know of in a pipeline system is on work not arriving.

Practical Notes

• Use IMeterFactory through DI rather than static meters. It makes meters testable, which means you can assert in a test that the metric you rely on is actually emitted — and metrics that silently stopped being emitted are a real failure mode.
• Name things consistently, following the OpenTelemetry conventions where they exist. Future you will be grateful when correlating across services.
• Instrument the boundaries first — inbound requests, outbound calls, queue operations. That's where the information density is highest.
• Delete metrics nobody looks at. A dashboard is a tool, not an archive.

The Summary

The library question — Meter, IMeterFactory, which instrument type — is well documented and quickly answered. The question worth spending time on is which four or five numbers per component would actually tell you what's wrong, and whether you could find them under pressure.

Instrument for the incident you'll have, not for the dashboard screenshot.

Source: Measure Application Performance in .NET Using IMeterFactory, Code Maze.