8 Sources of Uncertainty in Measurement for Every Uncertainty Budget

uncertainty in measurement

 

Introduction

Have you ever wondered what sources of uncertainty in measurement to include in your uncertainty budget? I have, and I am sure that you have too.

Today, I am going to teach you 8 sources of uncertainty in measurement that should be include in every uncertainty budget. The reason that you should include these uncertainty sources each time is because they typically influence every measurement that you will ever make.

To give you another reason to use these sources of uncertainty in measurement, consider that these are common uncertainty contributors that are being required by many accreditation bodies. Just check A2LA’s R205 Requirements Document and scroll down to section 6.7.1.

A2LA R205 Requirements

To help you create better uncertainty budgets and more appropriately estimate measurement uncertainty, I have created a list of 8 sources of uncertainty in measurement that should be in every uncertainty budget.

Furthermore, I am going to go beyond just telling you what these uncertainty sources are, I am going to give you the proper definitions to these uncertainty sources and teach you how to estimate their magnitude.

 

Sources of Uncertainty

Uncertainty in measurement can be influenced by many different factors. Below is a list of the 6 most common sources of uncertainty in measurement. When you begin to identify sources of measurement uncertainty, you should start by think about influences that are in these categories.

6 common sources of uncertainty in measurement:

  • Equipment
  • Unit Under Test
  • Operator
  • Method
  • Calibration
  • Environment

 

Sources for Every Uncertainty Budget

Now, I mentioned earlier that I am going to teach you the 8 Sources of Uncertainty in Measurement that should be included in every uncertainty budget. The influences that I will cover today are provide in the list below. So, take a look and let me know if I skip anything.

8 Sources of Uncertainty in Measurement that should be included in every uncertainty budget:

  • Repeatability
  • Reproducibility
  • Stability
  • Bias
  • Drift
  • Resolution
  • Reference Standard
  • Reference Standard Stability

 
Click here to download the 8 sources of uncertainty calculator for free!

 

Repeatability

Repeatability is a source of uncertainty in measurement that should be included in the every uncertainty budget. It is an influence that you can test yourself to see how much variability is in your measurement results under repeatable conditions.

Most accreditation bodies now require that repeatability is included in your uncertainty analysis. With this type of demand, you will notice more assessors asking to see your Type A data and checking to verify that it is included in your uncertainty budget.

Definition of Repeatability

 1: Measurement precision under a set of repeatability conditions of measurement

To simplify, repeatability is the measurement precision under a set of repeatable conditions. So, to perform a repeatability test, you must continually repeat the measurement process until you record your desired number of samples.

After you have collected your desired number of samples, you can begin to analyze the data to find the random error or variance of your measurement process. This can be accomplished by simply calculating the standard deviation of the set of samples that you have collected.

How to Calculate Repeatability

Follow this instructions to calculate repeatability:
1. Repeat a measurement ‘n’ number of times
2. Record the results of each measurement.
3. Calculate the standard deviation.

Example

Imagine that you need to perform a repeatability test where you collect 20 samples. To find the repeatability of your measurement process, just collect the 20 samples and calculate the standard deviation of your results. The result will be your repeatability.

In the image below, I simulated a set of 20 samples, normally distributed, where the nominal value was 10 and the standard deviation was 5 parts-per-million or ppm. Once the 20 samples were simulated, I calculated the standard deviation of my sample set to determine that repeatability is 4.5 ppm with 19 degrees of freedom. See the highlight red rectangle.

Just accomplish this using Microsoft Excel, I used the formula:

 =stdev(cell1:celln)

repeatability test uncertainty

 

Reproducibility

Reproducibility is a source of uncertainty in measurement that should be included in the every uncertainty budget. It is an influence that you can test yourself to see how much variability is in your measurements under reproducible conditions.

Most accreditation bodies now require that reproducibility is included in your uncertainty analysis. With this type of demand, you will notice more assessors asking to see your Type A data and checking to verify that it is included in your uncertainty budget.

What makes reproducibility different from repeatability is you need to change something (a variable) in your measurement process. Here is a list of the 5 most common comparisons for reproducibility testing.

5 most common comparisons for reproducibility testing:

  • Operator vs Operator Reproducibility
  • Equipment vs Equipment Reproducibility
  • Method vs Method Reproducibility
  • Day vs Day Reproducibility
  • Environment vs Environment Reproducibility

Definition of Reproducibility
1: Measurement precision under reproducibility conditions of measurement

How to Calculate Reproducibility
Follow this instructions to calculate reproducibility:
1. Perform a Repeatability Test
2. Calculate the mean of average
3. Change a variable and repeat the Repeatability Test
4. Calculate the mean or average.
5. Calculate the standard deviation of the test averages.

Example
Imagine that you need to perform a reproducibility test where you want to learn how reproducible your measurement results are when performed with different methods. Let’s find the reproducibility of your measurement process.

In the image below, I simulated 2 sets of 20 samples, normally distributed, where the nominal value was 10 and the standard deviation was 5 parts-per-million or ppm. Once the 20 samples were simulated, I calculated the mean (i.e. average) of each sample set.

Next, I calculated the standard deviation of the two calculated means to determine that the reproducibility is 14 ppm with 1 degrees of freedom. See the highlight red rectangle.

Just accomplish this using Microsoft Excel, I used the formula:

=stdev(cell1,cell2)

reproducibility test uncertainty

 

Stability

Stability is a source of uncertainty in measurement that should be included in the every uncertainty budget. It is an influence that you can test yourself or calculate from your calibration data to see how much variability is in your measurements over time.

Stability is a random uncertainty. It is commonly confused with Drift, which is a systematic uncertainty (we will cover this later). Essentially, stability determines how stable your measurement process is over time.

Stability can be determined in two ways. However, to keep it simple, I will only teach you the easy way to estimate stability.

Most accreditation bodies do not require you to include stability in your uncertainty budget. However, many assessors consider stability a significant contributor to uncertainty in measurement. So, I recommend that you include it in your measurement uncertainty analysis.

Definition of Stability
1: Property of a measuring instrument, whereby its metrological properties remain constant in time

How to Calculate Stability
Follow this instructions to calculate stability:
1. Review your last 3 calibration reports.
2. Record the results from each calibration report.
3. Calculate the standard deviation of the calibration results.

Example
Imagine that you need to determine the stability of your measurement process. So, you grab your last three calibration reports and record the values reported from calibration. Find the stability of your measurement process.

In the image below, I grabbed 3 calibration reports for one of my Keysight 34401A Multimeters and placed the data side by side. The parameter that I focused on was the 10 Volt measurement for the DC Voltage function.

drift uncertainty 1

Now, you can see that there was some variation in measurement capability from 2013 to 2015. This is what we want to evaluate.

So, look at the image below. I calculated the standard deviation of the 3 measurement results in the image above to determine stability. As a result, we have determined that the stability of this instrument is 4.6 ppm. See the highlight red rectangle.

To accomplish this using Microsoft Excel, I used the formula:

=stdev(cell1:celln)

stability uncertainty

 

Bias

Bias is a source of uncertainty in measurement that can be optionally added to your uncertainty budget. Whether or not you decide to make it part of your estimation of measurement uncertainty depends on how you use your equipment to perform measurements.

To determine whether or not you should include bias in your uncertainty budget, read the follow scenarios and see which best applies to your measurement process.

Scenario 1: I calibrate equipment using a known reference standard and report the result only.

If this describes you, then add bias to your uncertainty budget.

In Scenario 1, you would add bias to your uncertainty budget because you do not account for it when reporting your measurement results. Therefore, the bias of the reference standard could further contribute to the uncertainty in measurement results.

Scenario 2: I calibrate equipment using a known reference standard and report both the Standard value and the Unit Under Test value.

If this describes you, then DO NOT add bias to your uncertainty budget.

In Scenario 2, you would not add bias to your uncertainty budget because you have already accounted for it in your reported measurement results. Therefore, the bias of the reference standard can be eliminated as a contributor to the uncertainty in measurement results.

Bias is really a systematic error rather than an uncertainty. It informs you of how accurate your measurements are compared to the target value. However, depending on how you perform comparison measurements, bias may be a contributor to measurement uncertainty.

Definition of Bias
1: Estimate of systematic measurement error
2: Average of replicate indication minus a reference quantity value

How to Calculate Bias
Follow these instructions to calculate bias:
1. Review your latest calibration report.
2. Find the As Left value or measurement result.
3. Find the Nominal value or standard value.
4. Calculate the difference.

Example
In the image below, I grabbed 2 calibration reports and compared the results side by side. The first report (left image) is from my Fluke 5720A Calibrator and the second report (right image) is from my Keysight 34401A Multimeter.

bias uncertainty 1

Using the data from the image above, I calculated bias using Microsoft Excel in the image below. To calculate bias, all you need to do is subtract the standard value from the measured result of the unit under test. In this case, we determined that the bias of this instrument was 7.3 ppm. See the highlight red rectangle.

bias=measured value-standard value
b=mv-sv

To accomplish this using Microsoft Excel, I used the formula:

=cell2 – cell1

bias uncertainty 2

 

Drift

Drift is a source of uncertainty in measurement that should be included in the every uncertainty budget. It is an influence that you can calculate from your calibration data to see how much the error in your measurements changes over time.

Drift is a systematic uncertainty. It is commonly confused with Stability, which is a random uncertainty. Essentially, drift determines how the error in your measurement process changes over time, and how much it can contribute to your estimate of uncertainty in measurement.

Definition of Drift
1: Continuous or incremental change over time in indication, due to changes in metrological properties of a measuring instrument

How to Calculate Drift
Follow these instructions to calculate drift:
1. Review your last 3 calibration reports.
2. Record the results from each calibration report.
3. Record the date each calibration was performed.
4. Calculate the average daily drift rate.
5. Multiply the average daily drift rate by your calibration interval (in days).

Example
In the image below, I grabbed 3 calibration reports for one of my Keysight 34401A Multimeters and placed the data side by side. The parameter that I focused on was the 10 Volt measurement for the DC Voltage function.

drift uncertainty 1

Now, you can see that there was some change in measurement performance from 2013 to 2015. Using this data, you can calculate the average drift rate.

First, calculate the daily drift rate from 2014 to 2015.

drift uncertainty 2

Then, calculate the daily drift rate from 2013 to 2014.

drift uncertainty 3

Next, calculate the average daily drift rate.

drift uncertainty 4

Finally, multiply the daily drift rate by 365.25 days to calculate the average drift rate per year.

drift uncertainty 5

Look at the image below. I calculated the average, annual drift rate from my calibration report data. As a result, we have determined that the average drift of this instrument is 7.6 ppm. See the highlight red rectangle.

To accomplish this using Microsoft Excel, use the equations that I used in the images above.

drift uncertainty 6

 

Resolution

Resolution is a source of uncertainty in measurement that must be include in every uncertainty budget. To perform uncertainty analysis, you must include the resolution of the standard and the unit under test. However, whether or not you decide to include UUT resolution uncertainty as part of your estimation of measurement uncertainty depends on your process for estimating measurement uncertainty.

To determine whether or not you should include UUT resolution in your uncertainty budget, read the follow scenarios and see which best applies to your measurement process.

Scenario 1: I estimate measurement uncertainty for one single measurement process, or a process where the type of UUT never changes.

If this describes you, then add UUT Resolution to your uncertainty budget.

In Scenario 1, you would add UUT resolution to your uncertainty budget because you are evaluating the uncertainty of a single process (e.g. on-time measurement) or your measurement process always tests the same type of UUT.
This prevents you from calculating measurement uncertainty after every test or calibration.

Scenario 2: I estimate measurement uncertainty for a single measurement function or parameter where the UUT type can vary.

If this describes you, then DO NOT add UUT Resolution to your uncertainty budget.

In Scenario 2, you would not add UUT resolution to your uncertainty budget because you will account for it later when calculating measurement uncertainty after each test or calibration. To learn more about this, make sure you read the ILAC P14 policy for calculating calibration uncertainty.

If you are ISO/IEC 17025 accredited, you must meet the requirements of the ILAC P14 policy.

Definition of Resolution
1: Smallest change in a quantity being measured that causes a perceptible change in the corresponding indication

How to Find Resolution
Follow these instructions to calculate resolution:
1. Look at your measurement system or equipment
2. Find the least significant digit
3. Observe the smallest incremental change

Example
Determining resolution is not always as simple as you may think. In some scenarios, you can quickly find the resolution by looking at your measurement equipment and the unit under test. However, other times it can be a little more complicated; especially for artifacts and analog devices.

Finding the resolution of digital devices is pretty easy. Look at the digital display of the device and observe either the least significant digit or the smallest change of the least significant digit.

In the image below is a digital multimeter. Looking at the digital display, we can observe the least significant digit and the smallest change. From the observations, you can see that the resolution is 10 micro-volts.

multimeter resolution uncertainty

For devices with an analog scales, you will need to observe the marker spacing of the scale, the width of the markers, and the width of the needle or pointer. From these factors, you can determine the resolution uncertainty of your measuring equipment or the unit under test.

pressure gauge resolution uncertainty

When using artifacts, you need to look at your calibration reports to determine the least significant digit of the reported calibration value. Since artifacts do not have scales or displays, you can only determine your resolution from the known value of the artifact.

Gage Block resolution uncertainty

 

Reference Standard Uncertainty

Reference standard uncertainty is a source of uncertainty in measurement that should be included in the every uncertainty budget. It is an influence that you can find by looking at your calibration reports.

Reference standard uncertainty is a systematic uncertainty. It is introduced from the calibration of your equipment or certified reference material. Additionally, its value is traceable to a national or international standard which is why it is so important.

I see more laboratories get deficiencies for leaving reference standard uncertainty out of their uncertainty budget.”

Do not make the same mistake. Get your equipment calibrated by an ISO/IEC 17025 accredited laboratory and include the reference standard uncertainty in your uncertainty budget.

Definition of Reference Standard Uncertainty
1: Uncertainty of a measurement standard designated for the calibration of other measurement standards for quantities of a given kind in a given organization or at a given location

How to Calculate Reference Standard Uncertainty
Follow these instructions to find reference standard uncertainty:
1. Review your latest calibration report.
2. Find the reported estimate of measurement uncertainty

Example
When your equipment is calibrated by an ISO/IEC 17025:2005 accredited laboratory or a national metrology institute, you receive a report with measurement results and estimates of uncertainty in measurement. Each reported estimate of measurement uncertainty is your reference standard uncertainty.

In the image below, you will see a section of a calibration report for a Keysight 34401A Digital Multimeter. In the report is a series of measurement results, each with an estimate of uncertainty in measurement.

reference standard uncertainty 1

Since we have been evaluating the uncertainty at 10 VDC, I will select the uncertainty for the 10 VDC measurement result. This will be your reference standard uncertainty. See the red rectangle.

reference standard uncertainty 2

Expert Tip:
Now, some people will average the last three values of their reference standard uncertainty and put the calculated average in their uncertainty budget.

Other people will use the most recently reported reference standard uncertainty value in their uncertainty budget.

Either approach is acceptable. So, use the method that works best for you. If you are seeking for lower your estimates of measurement uncertainty, use the method that gets you the smallest result.

 

Reference Standard Stability

Reference standard stability is a source of uncertainty in measurement that should be included in the every uncertainty budget. It is an influence that you can calculate using data from your calibration reports to see how much variability is in your reference standard.

Reference standard stability is a random uncertainty. It is similar to calculating stability, but you calculate the standard deviation of your reference standard uncertainty instead of your measurement results. Essentially, your goal is to determine how stable your reference standard is over time.

If you are wondering why this is important, let me explain. There are two scenarios that make this contributor to measurement uncertainty relevant.

Scenario 1: Your equipment is calibrated by the same laboratory, but their reported estimate of uncertainty in measurement changes each time.

Sometimes the reported measurement uncertainty in your calibration report changes, even if only slightly with each calibration. In this scenario, the goal is determine the stability of your calibration laboratory’s reference standard.

Scenario 2: Your equipment is calibrated by different laboratories, each with a different reported estimate of uncertainty in measurement.

Sometimes your equipment is calibrated by different laboratories (for whatever reason). Each laboratory will report their own value of estimated uncertainty. In this scenario, the goal is to determine the stability of your traceable uncertainty.

Definition of Reference Standard Stability
1: stability of a measurement standard designated for the calibration of other measurement standards for quantities of a given kind in a given organization or at a given location

How to Calculate Reference Standard Stability
Follow these instructions to calculate reference standard stability:
1. Review your last 3 calibration reports.
2. Record the uncertainty estimate from each calibration report.
3. Calculate the standard deviation.

Example
In the image below, you will see that I have collected the reported estimates of measurement uncertainty from my last 3 calibration reports. The data is from one of my Keysight 34401A Digital Multimeters.

reference standard stability 1

In the next image, I have calculated reference standard stability by calculating the standard deviation of the three values highlighted in the image above. As a result, the reference standard stability is 0.29 ppm.

To accomplish this using Microsoft Excel, I used the formula:

=stdev(cell1:celln)

reference standard stability uncertainty

 

Other Uncertainty Sources

There are many other contributors to uncertainty in measurement results. There is no way that I could list them all here. So, I recommend that you always start your uncertainty analysis with the sources I have given you.

Afterward, evaluate your measurement process to identify additional sources of measurement uncertainty. If you get stumped, do some research.

Some great places to find sources of measurement uncertainty;

  • Online Search
  • Uncertainty Guides
  • Textbooks
  • Journal Articles
  • Conference Papers
  • Other Laboratories

If you are still stuck after searching all of these information sources, then contact me! I will be glad to help you or even create an uncertainty budget for you.

 
Click here to download the 8 sources of uncertainty calculator for free!

 

Conclusion

In this article, I have provided you with 8 sources of uncertainty in measurement that should be included in every uncertainty budget. Additionally, I have given you detailed information and shown you to quantify each source.

Now, I want you to download my guide and try these calculations yourself. Then, I want you to include these contributors to measurement uncertainty in your next uncertainty budget.

 

References

A2LA. (2015). R205 – Specific Requirements: Calibration Laboratory Accreditation Program. Frederick: A2LA.
JCGM. (2012). International Vocabulary of Metrology: Basic and General Concepts and Associated Terms. Sèvres: BIPM.

Posted in:
About the Author

Richard Hogan

Richard Hogan is the CEO of ISO Budgets, L.L.C., a U.S.-based consulting and data analysis firm. Services include measurement consulting, data analysis, uncertainty budgets, and control charts. Richard is a systems engineer who has laboratory management and quality control experience in the Metrology industry. He specializes in uncertainty analysis, industrial statistics, and process optimization. Richard holds a Masters degree in Engineering from Old Dominion University in Norfolk, VA. Connect with Richard on Google+ and LinkedIn.

18 Comments

  1. Good article, one question I found the resolution of Excel calculations far exceeds the resolution of our data or our ability to measure that small should we be truncating or rounding to the resolution of our data in Excel.

    • Hi Ron,

      I would use the resolution of your measurement equipment or unit under test. If needed, you can use the round or trunk functions in excel. Sometimes I find these quite helpful. I have also seen people use the ‘left’ and ‘right’ functions too.

      Personally, I like to count the number of places after the decimal point and use the following equation: =10^(-1*cell#). I can quickly type the number of places after the decimal point into a cell and it is automatically converted to a value equivalent to my resolution. For example, If I have 3 figures after the decimal point, I type ‘3’ into a particular cell and my function converts it to ‘0.001.’ It’s very helpful for calibration checklists and templates!

      Rick

  2. I would like to have your comments about the process that we can use to accept the results off calibration report ( ex : error plus uncertainty should be less than tolerance ) |error + uncertainty | =or < tolerance

  3. Rick, thank you so much for this latest articles! They are extremely valuable to me because they are very clear, well written and allow me to practice the calculations in discrete pieces. They also prompt my curiosity about how the elements of an estimate fit together. Your explanations of when to include an element and when to avoid adding that element twice are invaluable because they let me see the coverage provided by each element in the estimate.

    Wishes for future articles? 1) How about exploring why people confuse drift and stability and what tests will detect which is which? 2) I would also like to see you analyze the Excel random number generator in terms of its “reproducibilty” to use your terms. I noticed in your article that when you asked Excel for a number set with a deviation of 5 ppm what you got back was a set with slightly less. I makes sense to me but my trust was increased after I tried 20 iterations and looked at the variability. Monte Carlo is a relatively recent addition to the tools available to uncertainty estimators. It would be crazy to find out later that Excels version was biased.

    These article are a great example of the upside of the advent of social media. In the old days, people with your level of writing skill and technical ability had no outlet that could assist other practitioners. Please keep it up!

    • Hi Stephen,

      Thank you for your comments and feedback. I have noted your requests for future articles, and I will add your ideas to my list.

      Regarding the reproducibility of the random number generator in excel, I briefly covered it in this article, click here to read. I found that when a used a coverage factor of 2 (95.45% confidence), my results were 95.46% compliant of having 1 outlier or less out of 22 trails. Hint: The failure rate for 95.45% confidence is 1 in 22.

      Also, I must say that advances in web design and social media have allowed me build a platform for me to spread my work to thousands of people around the world. The amount of growth in subscribers and customers that I have experienced is amazing!

      • I can think of two reasons why this growth is not really amazing.

        First, after all the social media and disruption dust has settled, someone still has to produce some content. Hopefully that content will be high quality like yours, and users will continue to recognize it when they see it.

        Second, a whole lot can be done if getting the credit is not the primary goal.

  4. I think that the title of the article can cause confusion between peopele with not enough experience in the uncertainty calculations, I think that mus be clear since the begin of the publication that all mentioned sources are only the sources and not include the way to estimate the uncertainty of each one.

    • Hi Miguel,

      Please tell me what more can I add to the article to help you out. I did explain and show how to calculate each uncertainty source listed in the article. Are there more sources that you think I should add? Please leave a comment and let me know what I can do to make the article better.

  5. Hi, it is an excellent article. I have a problem about resolution (deferences between scenario 1 & 2). Would you please give an example about this scenarios? I think with some examples about each of scenario it will be more clear for me.

    With best regards,
    Somayeh Gholipour.

  6. Another question. Imagine we use a certified equipment as a reference standard. This equipment in the measurement processe plays a role as a reference standard, but in its calibration process it is UUT.
    In its calibration process we use its resolution as a source of uncertainty. But when it is used as a reference standard in measurement process, is it true to include its resolution again or not? I think because its resolution uncertainty calculated in its calibration process and include in its calibration certificate, it isn’t necessary to add it again.
    Would you please help me about this question?

    With best regards.

    • Hi Athar,

      You can try repeatedly viewing the pressure gauge at slightly different angles (that you believe are center-aligned) and record your results to find your parallax error.

      If you want to get really technical with it, you can use physics equations to see how your distance from the gauge, the refractive index of the glass, etc affect your measurement results. I think this method is a little bit of overkill. However, if you have the time, it could be fun to see the results.

      Rick

  7. Sir,

    Can we use assigned error in current calibration certificate and uncertainty thereof instead of using manufacturers specification

    • Hi Shubhangi,

      Yes. You can use the assigned error as bias and the calibration uncertainty as reference standard uncertainty. You may want to also consider stability and drift in your analysis if you are going to eliminate the use of the manufacturer’s accuracy specification.

      I hope this helps.

      Rick

Leave a Reply

Your email address will not be published. Required fields are marked *