Measurement Resolution

Measurement resolution is the smallest change that a measurement system can distinguish in a stated configuration. It is often confused with displayed digits or ADC bit count, but those are only parts of the problem. A pressure channel with a 16 bit converter may still have poor practical resolution if the sensor is noisy, the input range is poorly matched, the reference is unstable, the filter is too wide, the installation vibrates or the display rounds the result to a coarse increment.

For engineering decisions, resolution must be stated in the unit that matters: newtons, pascals, degrees Celsius, millimeters, microvolts, radians per second, counts per second, optical watts or another release quantity. A bare statement such as “12 bit resolution” is incomplete unless the range, gain, noise, bandwidth and calibration basis are also known.

Quantization Step

For an ideal ADC with usable input span V_{span} and N bits, the voltage step is:

If the calibrated measurement sensitivity after the analog front end is K volts per engineering unit, the ideal engineering-unit step is:

For a direct full-scale statement, if the measured range is FS, an ideal count step can be approximated by:

This is a quantization limit, not a complete resolution claim.

Display Least Count

The display or reported data format can be coarser than the electronics. If software reports pressure to 0.01 bar, then the displayed least count is:

Even if the ADC step is much smaller, the user cannot distinguish changes below the reported increment unless raw data or a finer output is available. Conversely, adding more displayed digits does not improve true measurement resolution if noise or drift dominates.

Noise-Limited Resolution

Noise sets another practical limit. If repeated readings at a constant input have standard deviation sigma_n, a simple screening rule can define a distinguishable change as:

where k is chosen for the decision risk. A common engineering screen uses k=3, but the value should be stated. For comparing the difference between two independent readings, the noise of the difference can be approximated as:

The final rule must match the application. A control display, a calibration release, a biomedical alarm and a fatigue-test data channel may require different confidence and bandwidth choices.

Effective Resolution Screen

A practical first screen is to take the largest active limit:

where R_{hyst} and R_{drift} represent hysteresis and short-term drift over the stated use condition. This expression is not a universal metrology standard; it is a conservative engineering screen. It prevents teams from quoting a tiny ADC step while ignoring the effects that actually hide small changes.

Worked Example

A pressure channel uses a 0 to 5 V ADC range, a 16 bit converter and a calibrated sensitivity of:

The ideal voltage step is:

The ideal pressure step is:

The display reports values to 0.01 bar, and a constant-pressure test at the released bandwidth gives:

Using k=3:

The effective resolution screen is therefore:

If the requirement is to resolve 0.02 bar, the channel passes this simplified resolution screen. It still does not prove accuracy, bias, linearity, traceability or uncertainty compliance.

Resolution Is Not Accuracy

Resolution describes distinguishability. Accuracy describes closeness to a reference. A system can resolve small changes and still be biased. Another system can have coarse resolution but a well-controlled mean value. That distinction is why resolution should sit beside offset error, gain error, linearity error, repeatability and uncertainty in a release package.

Evidence For Release

A defensible resolution statement includes measurement range, output format, ADC or counter range, analog gain, sensor sensitivity, bandwidth, filter settings, sample rate, averaging, noise test, environmental condition, calibration state, zero state, warm-up time and the acceptance rule used to define a distinguishable change. For safety or compliance decisions, include raw data or histograms from a constant-input test rather than only a calculated bit count.

Common Mistakes

Common mistakes include equating bit depth with resolution, reporting display digits as if they were physical capability, reducing bandwidth after calibration without updating the measurement definition, averaging data until noise looks small while hiding dynamic events, ignoring hysteresis and drift, improving ADC resolution when sensor noise dominates, and using resolution as a substitute for an uncertainty budget.

The practical rule is to state resolution in engineering units, at a stated bandwidth and configuration, with enough evidence to show what actually limits the smallest useful change.