The evaluation of uncertainty in measurement results is becoming an important requirement for laboratories seeking to attain or retain ISO/IEC 17025 accreditation. When assessors evaluate a laboratory’s uncertainty budgets, they commonly look for the inclusion of type A, or actual test, data. Type A data may include, but is not limited to, evaluations of repeatability, reproducibility, and stability testing. However, the most common expression of Type A data found in an uncertainty budget is the results of repeatability testing. Repeatability testing is the most common because it is usually quick and easy to complete; but, it is not always performed correctly. In a series blog posts, I am going to show you how to perform repeatability experiments and evaluate the data.
01 | Create A Plan
When deciding to conduct repeatability experiments, it is important to have a plan. Planning allows you to design and develop your experiment to minimize the risk of mistakes and errors. By creating a plan, you will be able to increase the effectiveness and efficiency of your experimental efforts. There is nothing worse than expending time and effort to perform a task where the results are invalidated and the experiment must be repeated.
To develop a plan, you should first identify your goals; what do you expect to accomplish as a result of your experiment? Next, you should focus on the design of your experiment. To keep this process simple and effective, answer the questions; who, what, when, where, why, and how. Accomplish this task and you should have a pretty good idea of how to plan your experiment.
02 | Select A Function or Parameter
After establishing your goals, now you need to identify what you are going to test. Using your laboratory’s scope of accreditation, select a specific function or parameter of interest. Next, identify what range or ranges you prefer to evaluate. Although this task is simple, it will drastically narrow the breadth of your experiment. Therefore, it is important to target what you will be testing.
03 | Select A Method
Now that you have chosen what you are going to test, let’s determine how you are going to test it. Similar to any test or calibration that your laboratory performs, you will need to select a method. Choosing a method allows you to further narrow the scope of your experiment, control how the experiment will be conducted, and maintain consistency when you repeat these experiments in the future. Similar to having a plan, selecting a method or procedure is an intelligent idea.
04 | Select The Equipment
Selecting equipment to perform an experiment is important to achieving your goals. Preferably, you should select the equipment that is listed in your scope of accreditation and ensure that the equipment that you have selected is identified in your test method. Furthermore, it is recommended to choose equipment that will best showcase your laboratory’s measurement capability. Selecting the right equipment can have a significant impact of the results of your experiment. So, be sure to choose your equipment wisely.
05 | Select The Personnel
Who you select to perform experiments is an important variable that can have a significant impact on the results of your experiment. You should only allow authorized personnel to perform experiments. Additionally, you should select personnel whose expertise is in the function or parameter you are planning to test. It is important that the personnel performing experiments should be aware of the importance of their task. By selecting the appropriate personnel, you should be able to increase confidence and reduce uncertainty in the measurement results.
06 | Perform The Experiment
Now, that you have identified your goals and selected what you will test, how you will test it, and with whom, it is time to perform a repeatability experiment. However, before you begin, it is recommended that you identify how many samples will be collected (i.e. how many times the test will be repeated), how the samples will be recorded, and where will they be stored. Once this is accomplished, you should be ready to begin the experiment.
When conducting the experiment, it is important to monitor the environmental conditions. Similar to test and calibrations, I recommend that you monitor and record this information. After your experiment has been repeated over time, it will allow you evaluate how environmental conditions affect the test results. Furthermore, it will allow you identify and invalidate test results prior to analysis and evaluation. Invalidated data can have a negative or false impact on your estimation of measurement uncertainty.
07 | Perform The Analysis
After performing your experiment, it is time to analyze the results. Although there are many ways to quantitatively analyze data, I recommend that you focus on calculating the mean (i.e. average), standard deviation, and degrees of freedom from the collected data. The mean is statistical average of your pool of collected data. Recording this information will help you with other experiments such as the evaluation of reproducibility and stability. The standard deviation is the average dispersion of data about the mean. It is best predictor of the uncertainty in measurement results observed during experimentation under repeatable conditions. The degrees of freedom are the number of values which the result is free to vary. This is an important value to consider should you decide to perform uncertainty analysis using Bayesian Statistics.
08 | Conduct An Evaluation
With the completion of quantitative analysis, you may wish to perform qualitative analysis and evaluate the results of your experiment. Comparing the expressed CMC, or Calibration Measurement Capability, published in your scope of accreditation, how did the results of your repeatability experiment compare? If the results are less than the CMC expressed in your scope, then your current CMC may be adequately represented. If the results are greater than the CMC expressed in your scope, then you may need to investigate why the results of your experiment do not conform. Regardless of the results, it is important to evaluate the outcome of your experiments and use what you have learned to improve your process.
09 | Establish A Schedule
One test does not validate a postulate (i.e. hypothesis). It is important to repeat experiments and perform verification studies before validating your results. Scheduling future experiments will help you organize and control your experimentation process. No one likes to (and should not) cram hundreds of experiments right before an assessment. Instead, develop a schedule and allow yourself adequate time to properly evaluate results. It will reduce the occurrence of invalidated results and reduce the impact on your laboratory’s productivity.
Repeatability tests are an important factor when performing uncertainty analysis. Laboratories should consider incorporating these experiments into their quality processes so they become routine and not burdensome. Otherwise, an assessor may determine to report a laboratory as deficient and mark parameters on a laboratory’s scope as ‘TBD.’ Furthermore, you will not have any evidence to support your calculations of uncertainty in measurement results.
In this series of articles related to repeatability testing, I will go more into depth about the steps described in this article. I will demonstrate and explain how to determine the number of samples that you should collect, how you should record the data, how to analyze the data, and much more. Be sure to ‘Like Us’ on Facebook and ‘Follow Us’ on Twitter as I post new articles related to repeatability testing.
