Gage bias and linearity studies: how to run them and read the results

What bias and linearity mean

Bias is a systematic offset. In the VIM (JCGM 200:2012, 4.20), instrumental bias is what you get when you subtract a reference value from the mean of repeated readings. A gage with a bias of +0.002 mm reads 0.002 mm high on average at that point. Averaging more readings does not remove it.

Linearity describes whether that bias stays the same across the range the gage is used over. A gage can be close to true at the low end and read progressively high toward the top. The NIST/SEMATECH e-Handbook of Statistical Methods gives linearity a narrow meaning along the same lines: equal steps in the measured quantity should give equal steps in the reading, so that any bias stays constant.

Both belong to the accuracy side of measurement systems analysis (MSA). Gage R&R looks at how much readings scatter between repeats and between appraisers. Bias and linearity look at where the readings sit against a known value. Stability, the third accuracy study, follows the bias on one reference part over time.

• Gage R&R and MSA explained
• Intermediate checks and control charts

When to run a bias or linearity study

The AIAG Measurement Systems Analysis reference manual, fourth edition, describes both studies. IATF 16949 clause 7.1.5.1.1 asks automotive suppliers for statistical studies of each type of measurement system named in the control plan, with methods and acceptance criteria from the MSA reference manuals unless the customer approves others.

Outside automotive work the same studies answer practical questions.

• A new gage, or a gage moved to a new job, before its readings are used to accept product.
• A gage that has been repaired or adjusted.
• A gage used across a wide range, where calibration results show the error growing toward one end.
• Doubts about readings in the working environment. The NIST/SEMATECH e-Handbook points out that calibration relates a gage to a reference under ideal conditions, while temperature, operators and wear can add bias where the gage is actually used.

• Calibration for IATF 16949
• Gage R&R calculator

How to run a bias study

A bias study uses one reference part and one appraiser who normally uses the gage.

• Pick a reference part at a value the gage is used at, often near the middle of its working range.
• Establish its reference value with equipment more accurate than the gage under study, traceable to the SI, or use a calibrated standard such as a gauge block with a certificate.
• Have the appraiser measure the part repeatedly, in the normal way.
• Work out the bias of each reading, the average bias and the standard deviation of the readings.
• Check the spread before the bias. If the repeatability is large compared with the tolerance or the process variation, find the cause of the scatter first.

• Type 1 gage study template
• Metrological traceability explained

Is the bias significant? The t test

Data case: n individual readings of one reference part, taken by one appraiser and assumed independent and roughly normal. The bias is the average reading minus the reference value. Its standard error is s / √n, where s is the standard deviation of the readings.

The test statistic is t = bias / (s / √n), with n − 1 degrees of freedom. Compare its absolute value with the two-sided critical value of Student's t, usually at a significance level of 0.05. If |t| is larger, the bias is statistically different from zero. The confidence interval gives the same answer: the bias is significant when zero falls outside bias ± t × s / √n. This is the one-sample t test described in section 7.2.2 of the NIST/SEMATECH e-Handbook.

Example. A digital micrometer reads a 12.5 mm gauge block ten times. The readings minus the block's certified size are +2, +3, +1, +2, +3, +2, +2, +4, +1 and +2 µm. The average bias is +2.2 µm and s is 0.92 µm, so the standard error is 0.29 µm and t is 7.57. With 9 degrees of freedom the two-sided critical value at 0.05 is 2.262. The bias is significant: this micrometer reads high at 12.5 mm.

Statistically significant is not the same as too big. With enough readings, even a tiny bias tests as significant. Judge its size against the tolerance the gage supports. For a part tolerance of ±0.02 mm, a bias of 2.2 µm is 5.5 percent of the 0.04 mm tolerance width.

• Bias and linearity calculator

How to run a linearity study

A linearity study repeats the bias check at several reference values spread over the range the gage is used across. The NIST/SEMATECH e-Handbook asks for at least five reference standards covering the range in roughly equal steps, each measured more than once. It suggests measuring each standard once and then repeating the whole round, rather than taking all the repeats on one standard together.

The AIAG manual describes the same design with production parts whose reference values come from more accurate measurement, all measured by one appraiser who normally uses the gage. Gauge blocks, setting rings or other calibrated standards do the same job when production parts are hard to measure precisely.

Fitting the line: regression of bias on reference value

Work out the bias of each individual reading: the reading minus the reference value of its part. Then fit a straight line by least squares, with the bias on the vertical axis and the reference value on the horizontal axis. Data case: all N individual readings, not the averages per part. The residual standard deviation s of the fit has N − 2 degrees of freedom.

The slope says how much the bias changes per unit of reference value. The intercept is the fitted bias at a reference value of zero. The NIST e-Handbook fits the readings themselves against the reference values and tests for a slope of 1. That is the same test, because the bias is the reading minus the reference.

Test each coefficient with a t statistic, the estimate divided by its standard error, against Student's t with N − 2 degrees of freedom. For the slope, the standard error is s divided by the square root of Sxx, the sum of squared deviations of the reference values from their mean. For the intercept, it is s × √(1/N + m²/Sxx), where m is the mean reference value. The e-Handbook uses the same tests, at a significance level such as 0.05, to decide whether a calibration curve needs correcting.

The AIAG manual adds a graphical check. Plot the fitted line with its confidence bands, together with the line of zero bias. AIAG treats linearity as acceptable when the zero-bias line stays inside the bands across the whole range and neither the slope nor the intercept differs significantly from zero.

Worked example: a micrometer that reads high toward the top of its range

A 0 to 25 mm digital micrometer is checked with gauge blocks of 5, 10, 15, 20 and 25 mm, four readings on each, taken in rounds. The table lists each reading minus the block size in µm.

The fit over all 20 readings gives a slope of 0.13 µm per mm, or 0.00013 when readings and references are both in mm. The intercept is +0.15 µm, and the residual standard deviation is 0.62 µm with 18 degrees of freedom. The slope's t statistic is 6.64 against a two-sided critical value of 2.101, so the slope is significant. The intercept's t statistic is 0.46, so it is not.

The micrometer has a linearity error. On the fitted line its bias grows from about +0.8 µm at 5 mm to +3.4 µm at 25 mm, or 1.3 µm for each 10 mm. An error that grows along the range points to something like wear or a setting that was off at the ends, rather than a simple zero offset. Whether it matters depends on the tolerances the micrometer is used for.

• Check the example in the bias and linearity calculator

What to do when a gage shows bias or a linearity error

Find the cause first. For a linearity error, the causes the MSA manual lists, and the NIST/SEMATECH e-Handbook repeats, include a gage not calibrated properly at the low and high ends of its range, an error in the reference value at the ends, a worn gage and internal design problems. Check the reference parts too, because a wrong reference value looks exactly like gage bias. In dimensional work, temperature is a common source of bias as well.

Then choose what to do.

• Recalibrate or adjust the gage, then repeat the study.
• Repair or replace a worn gage.
• Correct the readings for a known, stable bias, and include the uncertainty of the correction. The e-Handbook describes this approach for differences between gauges.
• Where correcting is not worth the effort, treat the bias as a component of the measurement uncertainty, as the e-Handbook also allows.
• Restrict the gage to the part of its range where the bias is acceptable, and label it so.

• How to build a measurement uncertainty budget
• What to do when a gage fails calibration

How Axiospec fits

Axiospec is calibration management software, and it runs MSA studies on the instrument record. On an instrument's page, a manager or admin can start a bias, linearity, Gage R&R or attribute agreement study, on any plan. The study is saved with the instrument, next to its calibrations, certificates and due dates.

The bias study takes a reference value and at least five readings. It reports the bias, the t statistic against the two-sided critical value at 0.05 and, when you enter a tolerance, the bias as a percentage of it. The linearity study takes up to 10 reference values with 2 to 5 readings each. It fits the individual biases against the reference values and reports the slope, the intercept, r² and a t test of the slope. The intercept test and the AIAG confidence bands are not in the app. Each verdict is advisory, and the decision on the gage stays with you.

The free plan covers up to 50 active instruments. A new workspace starts with a 45-day trial of Professional, with room for 1,500 active instruments and no credit card. We load your existing records for you at no charge on every plan.

• Free bias and linearity calculator
• Free Type 1 gage study template
• See the live demo, no signup
• Start a 45-day trial
• Compare plans