Bias is the systematic difference between what your gage reads and the true value. Linearity is how that bias changes across the gage's operating range. A bias and linearity study measures reference standards spread across the range, compares the readings to their reference values, and checks whether the error is small, significant and constant. If it is not, the gage needs calibration, repair or a different use before its data goes into an inspection report.
This guide covers the definitions, the study design, the calculations, how to interpret the output and what to do when the result is bad. It is written for quality engineers and inspectors in aerospace, automotive and medical device production, where measurement data has to survive an audit.
TL;DR
- Bias is the average measured value minus the reference value. It is a constant, systematic offset.
- Linearity is the change in bias across the gage's operating range. A gage can be accurate at one size and off at another.
- Use at least 5 reference parts or standards that span the range, measured repeatedly, then fit a regression of bias against reference value.
- Judge the result by statistical significance and by how large the bias is compared with your process variation or tolerance, not by the p-value alone.
- Failures usually trace back to calibration, worn gages, wrong reference values, temperature or operator technique.
Table of Contents
- What bias means in a measurement system
- What linearity means, and how it differs from bias
- When to run a bias and linearity study
- Before you start: prerequisites
- How to design the study
- Step-by-step procedure
- How bias and linearity are calculated
- How to interpret the results
- What to do when the study fails
- Where bias and linearity fit in the larger MSA picture
- Our perspective
- Put your measurement data to work
- FAQ
- Sources
What bias means in a measurement system
In plain terms, bias is a consistent lean in one direction. iSixSigma describes it as the systematic, unidirectional tendency of a measurement to under- or overestimate the true value. Its example is a scale that reads 49.25 pounds for 50-pound bags, a bias of 0.75 pounds.
NIST's engineering handbook defines bias as the difference between the average of measurements made on the same object and its true value. It adds that bias can be found through calibration of standards, check standards, measurement assurance programs and interlaboratory comparisons. ASTM frames it the same way: the difference between the mean of test results on a reference material or item and an accepted reference value. It treats bias and precision as two separate attributes of a measurement method.
That separation matters. Repeatability and reproducibility tell you how much readings scatter. Bias tells you where the center of that scatter sits relative to the truth. A gage can be very repeatable and still be consistently wrong. If you are also assessing scatter, our practical guide to Gage R&R studies covers that side.
What linearity means, and how it differs from bias
Bias is a single number at a single point. Linearity asks whether that number holds still as you move across the range of the gage.
The AIAG MSA Reference Manual, 4th Edition, defines linearity as the change in bias over the normal operating range. Lockheed Martin's supplier training material describes it similarly as the difference in accuracy over the expected measurement range of the gage.
A practical way to see the difference:
- A micrometer that reads 0.0005 in. high at every size has bias but no linearity problem. The error is constant, so an offset correction fixes it.
- A micrometer that reads correctly near 0.5 in. but 0.002 in. high near 3 in. has a linearity problem. One correction value cannot fix both ends.
The NIST handbook gives the statistical version. It fits a line of the form Y = a + bX plus error. If the slope differs from one, the gauge is non-linear and needs calibration or repair. If the intercept differs from zero, the gauge has a bias.
Bias and linearity sit next to stability in the MSA location family. Stability is the change in bias over time, also called drift. The AIAG manual describes a stable process as one in statistical control with respect to location. So bias is the offset, linearity is the offset across size, and stability is the offset across time.
When to run a bias and linearity study
Run one when the answer to "is this gage right across everything we measure with it?" is not already documented. Typical triggers:
- A new gage or CMM program goes into service for a characteristic with a wide size range.
- A gage comes back from repair or recalibration.
- A customer or an AS9100 or IATF 16949 audit asks for evidence that the measurement system is suitable.
- A Gage R&R passes, but parts of different sizes show unexplained disagreement between gages.
- A trend on a check-standard chart suggests wear or drift.
Dennis Craggs notes that after a first bias study you may assume the bias is constant, but a linearity study is how you verify that assumption. That is the logic of the whole exercise: do not assume the offset is the same everywhere, test it.
Before you start: prerequisites
A study is only as good as its setup. Lockheed Martin's MSA training document lists the conditions:
- The equipment is calibrated and traceable to a national or international standard.
- The people involved are trained and competent.
- The sample parts represent the whole production operating range, ideally the whole allowed tolerance range.
- The gage has adequate resolution, with the document citing one tenth of the specification.
Two more habits help. Let the gage, the standards and the parts reach the same temperature before measuring. NIST points out that temperature probably has the most potential for introducing bias, and that a constant heat source can cause serious errors in dimensional measurements of metal objects. Also, decide in advance who measures and in what order, and write it into the study plan so the record is clean.
If you are unsure how calibration differs from the checks your records need to show, read calibration vs verification first.
How to design the study
How many standards and readings
The published guidance is consistent in spirit and slightly different in numbers:
| Source | Reference standards | Readings |
|---|---|---|
| AIAG MSA 4th Edition, as summarized by Craggs | At least 5 parts spanning the operating range | Each part's reference value determined, then measured at least 10 times by the usual operator, in randomized order |
| NIST/SEMATECH handbook | More than 4, covering the range in fairly equal increments | More than 1 measurement on each, with the process repeated |
| Bosch Booklet No. 10 | At least 5 standards with different reference values | If only two are available, type-1 studies at the limits of the tolerance range |
Bosch also states that the AIAG linearity procedure does not give reliable results under all conditions. That is a good reason to treat the procedure as a framework and follow whatever your customer's quality manual specifies.
Choosing reference values
Spread the standards evenly from the low end to the high end of what you actually measure, ideally covering the tolerance range. Reference values should come from a higher-accuracy source: gage blocks, certified masters or a calibration laboratory. Craggs notes that production parts can serve as reference standards if they are measured in a laboratory or on a flat-plate inspection station first. If you do this, remember that an error in the reference value shows up in your results as bias or non-linearity.
Run order
Randomize. Measuring all ten readings of one part before moving on invites a warm-up trend, operator memory and drift to hide inside your data. Mix the parts, record the order and use the same method production uses.
Step-by-step procedure
- Define the characteristic, the gage, the unit and the operating range from your control plan and drawing.
- Select at least 5 standards or parts across that range.
- Establish each reference value with a higher-accuracy method and record its traceability.
- Confirm the gage's calibration status and resolution, and let everything stabilize at room temperature.
- Have the usual operator measure each part at least 10 times in randomized order, using the standard work instruction.
- Compute the average at each reference value and the bias at each point.
- Fit a regression of bias against reference value and check the statistics.
- Document the result, the decision and any corrective action in the measurement system record.
How bias and linearity are calculated
Bias at one point
Lockheed Martin's document gives the formula as bias equals the average of readings minus the reference value. It then expresses bias as a percentage of process variation, where process variation is 6 standard deviations.
Worked example with made-up numbers. A reference ring measures 1.0000 in. Ten readings average 1.0004 in. Bias is +0.0004 in. If the process variation for that characteristic is 0.0100 in., bias is 4% of process variation.
Linearity across the range
Now repeat at each reference value. Suppose five standards give these average biases:
| Reference value (in.) | Average bias (in.) |
|---|---|
| 0.5000 | +0.0001 |
| 1.0000 | +0.0004 |
| 2.0000 | +0.0009 |
| 3.0000 | +0.0014 |
| 4.0000 | +0.0019 |
The bias grows steadily with size. A fitted line through these points has a clear positive slope, and that slope is the linearity signal. EngineRoom's explanation notes the regression line is sloped when linearity is significant. In this example, no single offset would correct the gage everywhere, so a calibration check at the upper end is the next move.
Most teams run this in statistical software or a validated spreadsheet. Whatever you use, keep the raw readings with the output so an auditor can trace every figure back to a measurement.
How to interpret the results
Two questions, always asked together: is the effect statistically real, and is it big enough to matter?
Statistical significance
Study output typically gives a p-value for the bias at each reference value and for the regression slope. By the usual convention, a p-value below 0.05 means the departure from zero is unlikely to be chance. A significant slope means linearity is present. A significant bias at a point means the gage is reading off the reference there.
Practical size
Significance depends on sample size. With enough readings, a tiny bias becomes statistically significant. So compare the size of the bias, and the change in bias across the range, with your process variation or tolerance. Acceptance limits come from your customer's requirements and your own quality system, so apply those rather than a generic threshold. Some published templates use a single percentage target, but your customer's supplier manual governs.
Reading the combination
| Result | What it suggests | Typical next step |
|---|---|---|
| Small bias, flat slope | Gage reads true across the range | Document and release |
| Significant bias, flat slope | Constant offset | Correct by calibration or adjustment, then repeat |
| Slope significant | Error changes with size | Investigate calibration at both ends, wear, standards |
| Bias varies with no clear line | Possible reference errors or poor technique | Recheck reference values and procedure |
What to do when the study fails
Start with the likely causes. The NIST handbook, quoting the MSA manual, lists four causes of non-linearity:
- The gauge is not properly calibrated at the lower and upper ends of its range.
- There is an error in the value of X at the maximum or minimum of the range.
- The gauge is worn.
- There are internal design problems, for example in the electronics.
For high bias, Lockheed Martin lists appraisers not following the measurement procedure, an error in the reference value, and instability, where a trend on an SPC chart can show a wearing device or drifting calibration.
NIST says bias can be eliminated or reduced through calibration of standards or instruments. Reference standards and check standards are the backbone of that work, and the same discipline shows up in good quality assurance in manufacturing practice of any kind.
A practical response sequence:
- Confirm the reference values and their traceability.
- Re-read the procedure with the operator and observe a measurement.
- Check temperature conditions and fixturing.
- Inspect the gage for wear and send it for calibration.
- Repeat the study after the fix and file both studies.
For CMMs, programming and datum choices can mimic bias. If errors vary with part orientation, look at datum shift in GD&T inspection and at your CMM file checks before blaming the machine.
Where bias and linearity fit in the larger MSA picture
A complete measurement system analysis looks at location (bias, linearity, stability) and spread (repeatability and reproducibility). A common order is to establish bias and linearity first, since a gage that is systematically wrong across its range will contaminate everything downstream. Then run the Gage R&R, and keep a check standard on a control chart for stability.
Measurement uncertainty ties it together. Bias that you cannot remove becomes part of the uncertainty budget. Our five-step measurement uncertainty workflow shows how to carry those contributions into a decision about conformance to tolerance.
Our perspective
We see the same pattern in shops that move from spreadsheets to a structured quality system: the measurement study is rarely the problem, the paper trail around it is. A bias and linearity study done well can still fail an audit if the reference values, the gage ID, the operator and the raw readings live in four different files.
That is why we build QA-Report around linking the drawing, the ballooned characteristic and the measured result in one place. When a dimension on a ballooned drawing is tied to its measurements, out-of-tolerance results are flagged automatically, and CMM data can be imported and mapped to the right characteristics instead of retyped. Retyping is where transposed digits become phantom bias.
Our view is that traceability works like a chain: one missing link, and the whole record is questioned. Reference value, instrument, measurement and report should connect. We designed the platform, in cloud or on-premise deployment for sensitive industries, so that your team can keep that chain intact without extra paperwork.
Michael Chen
Put your measurement data to work
Once your gages are proven, the next job is turning measurements into reports that stand up to AS9102, PPAP and AS9100 review. QA-Report links ballooned drawing dimensions to measured results, flags deviations and produces PDF reports. You can see how CMM data import works and start on the free plan at $0 per month. Basic is $49.99 per month and PRO is $149.99 per month, with details on the pricing page.
On-Premise and Enterprise plans are available for larger or more sensitive operations, quoted individually.
FAQ
What is the difference between bias and linearity?
Bias is the systematic difference between the average reading and the reference value at one point. Linearity is how that bias changes across the gage's operating range. A constant offset is bias alone, while an offset that grows or shrinks with part size also indicates a linearity problem.
How many parts do I need for a bias and linearity study?
At least 5 reference parts or standards spanning the operating range is the common guidance, from the AIAG manual as summarized by Craggs and from Bosch. The AIAG approach calls for at least 10 readings on each part in randomized order. Follow your customer's requirements if they specify more.
Does a significant p-value mean the gage fails?
Not automatically. The p-value says the bias or slope is unlikely to be chance, not that it is large. Compare the size of the effect with your process variation or tolerance and apply the acceptance limits in your quality system or your customer's requirements.
How is a bias and linearity study different from Gage R&R?
Bias and linearity address accuracy against a reference across the range. Gage R&R addresses variation from the gage and from different operators. A thorough measurement system analysis uses both, and our Gage R&R guide explains the second.
Sources
- Bias, iSixSigma dictionary
- NIST/SEMATECH e-Handbook: Bias and Accuracy
- NIST/SEMATECH e-Handbook: Linearity of the gauge
- ASTM: Fundamental Measurement Concepts
- AIAG MSA Reference Manual, 4th Edition
- Lockheed Martin: Measurement System Analysis
- Bosch Booklet No. 10: Capability of Measurement and Test Processes
- MSA 4: Gage Linearity, Accendo Reliability
- Linearity and Bias Study, EngineRoom
- Quality assurance, Gather Packaging