What is CMC Uncertainty

Short answer: According to the proceedings from the 96th meeting of the CIPM (2007), CMC uncertainty is a calibration and measurement capability available to customers under normal conditions: As published in the BIPM key comparison database (KCDB) of the CIPM MRA or as described in the laboratory’s scope of accreditation granted by an ILAC-MRA signatory.

Short answer: According to the proceedings from the 96th meeting of the CIPM (2007), CMC uncertainty is a calibration and measurement capability available to customers under normal conditions:

  1. As published in the BIPM key comparison database (KCDB) of the CIPM MRA; or
  2. As described in the laboratory’s scope of accreditation granted by an ILAC-MRA signatory.

 

CMC Definition per 96th meeting of the CIPM (2007)

In the context of the CIPM MRA and ILAC Arrangement, and in relation to the CIPM-ILAC Common Statement, the following shared definition is agreed upon:

“a CMC is a calibration and measurement capability available to customers under normal conditions:

  1. as published in the BIPM key comparison database (KCDB) of the CIPM MRA; or
  2. as described in the laboratory’s scope of accreditation granted by a signatory to the ILAC Arrangement.”

 

CIPM Notes Accompanying the CMC Definition

The Notes to accompany the definition are of crucial importance, and aim to clarify issues of immediate relevance to the definition. They do not claim to cover every implication, or to address related issues. They may, however, be developed further, either in the current draft ILAC policy document on the estimation of uncertainty in calibration, or in any guidance subsequently developed by the JCRB, for approval by the CIPM.

 

Note 1

The meanings of the terms Calibration and Measurement Capability, CMC, (as used in the CIPM MRA), and Best Measurement Capability, BMC, (as used historically in connection with the uncertainties stated in the scope of an accredited laboratory) are identical. The terms BMC and CMC should be interpreted similarly and consistently in the current areas of application.

 

Note 2

Under a CMC, the measurement or calibration should be:

  • performed according to a documented procedure and have an established uncertainty budget under the management system of the NMI or the accredited laboratory;
  • performed on a regular basis (including on demand or scheduled for convenience at specific times in the year); and
  • available to all clients.

 

Note 3

The ability of some NMIs to offer “special” calibrations, with exceptionally low uncertainties which are not “under normal conditions,” and which are usually offered only to a small sub-set of the NMI’s clients for research or for reasons of national policy, is acknowledged. These calibrations are, however, not within the CIPM MRA, cannot bear the equivalence statement drawn up by the JCRB, and cannot bear the logo of the CIPM MRA. They should not be offered to clients who then use them to provide a commercial, routinely available service. Those NMIs which can offer services with a smaller uncertainty than stated in the database of Calibration and Measurement Capabilities in the KCDB of the CIPM MRA, are, however, encouraged to submit them for CMC review in order to make them available on a routine basis where practical.

 

Note 4

Normally there are four ways in which a complete statement of uncertainty may be expressed (range, equation, fixed value and a matrix). Uncertainties should always comply with the Guide to the Expression of Uncertainty in Measurement (GUM) and should include the components listed in the relevant key comparison protocols of the CIPM Consultative Committees. These can be found in the reports of comparisons published in the CIPM MRA KCDB as a key or supplementary comparison.

 

Note 5

Contributions to the uncertainty stated on the calibration certificate and which are caused by the client’s device before or after its calibration or measurement at a laboratory or NMI, and which would include transport uncertainties, should normally be excluded from the uncertainty statement. Contributions to the uncertainty stated on the calibration certificate include the measured performance of the device under test during its calibration at the NMI or accredited laboratory. CMC uncertainty statements anticipate this situation by incorporating agreed-upon values for the best existing devices. This includes the case in which one NMI provides traceability to the SI for another NMI, often using a device which is not commercially available.

 

Note 5a

Where NMIs disseminate their CMCs to customers through services such as calibrations or reference value provision, the uncertainty statement provided by the NMI should generally include factors related to the measurement procedure as it will be carried out on a sample, i.e., typical matrix effects, interferences etc. must be considered. Such uncertainty statements will not generally include contributions arising from the stability or inhomogeneity of the material. However, the NMI may be requested to evaluate these effects, in which case an appropriate uncertainty should be stated on the measurement certificate. As the uncertainty associated with the stated CMC cannot anticipate these effects, the CMC uncertainty should be based on an analysis of the inherent performance of the method for typical stable and homogeneous samples.

 

Note 5b

Where NMIs disseminate their CMCs to customers through the provision of certified reference materials (CRMs) the uncertainty statement accompanying the CRM, and as claimed in the CMC, must indicate the influence of the material (notably the effect of instability, inhomogeneity and sample size) on the measurement uncertainty for each certified property value. The CRM certificate should also give guidance on the intended application and limitations of use of the material.

 

Note 6

The NMI CMCs which are published in the KCDB provide a unique, peer reviewed traceability route to the SI or, where this is not possible, to agreed-upon stated references or appropriate higher order standards. Assessors of accredited laboratories are encouraged always to consult the KCDB (http://kcdb.bipm.org) when reviewing the uncertainty statement and budget of a laboratory in order to ensure that the claimed uncertainties are consistent with those of the NMI through which the laboratory claims traceability.

 

Note 7

National measurement standards supporting CMCs from an NMI or DI are either themselves primary realizations of the SI or are traceable to primary realizations of the SI (or, where not possible, to agreed-upon stated references or appropriate higher order standards) at other NMIs through the framework of the CIPM MRA. Other laboratories that are covered by the ILAC Arrangement (i.e. accredited by an ILAC Full Member Accreditation Body) also provide a recognized route to traceability to the SI through its realizations at NMIs which are signatories to the CIPM MRA, reflecting the complementary roles of both the CIPM MRA and the ILAC Arrangement.

 

Note 8

Whereas the various parties agree that the use of the definitions and terms specified in this document should be encouraged, there can be no compulsion to do so. We believe that the terms used here are a significant improvement on those used before and provide additional guidance and help so as to ensure consistency in their use, understanding, and application worldwide. We therefore hope that, in due course, they will become commonly accepted and used.

 

CMC Uncertainty Examples

Below are common examples for expressing CMC Uncertainty in a scope of accreditation per the ILAC P14 policy.

 

CMC Uncertainty as a Function

In the image below, is an example of expressing CMC uncertainty as an explicit function. The function is used to estimate uncertainty within the range stated in the scope of accreditation. The functions in the image are highlighted by a red rectangle. Explicit functions can be expressed in many ways and formats. Review other accredited lab scopes for additional examples.
CMC Uncertainty Example - Function per ILAC P14

 

CMC Uncertainty as a Single Value

In the image below, is an example of expressing CMC uncertainty as a single value. The single value is constant and represents the measurement uncertainty within the range stated in the scope of accreditation. The single value in the image is highlighted by a red rectangle.
CMC Uncertainty Example - Single Value per ILAC P14

 

CMC Uncertainty as a Matrix

In the image below, is an example of expressing CMC uncertainty as a matrix. The matrix is a table that expresses the uncertainty as a function, single value or relative uncertainty against two factors, such as range and additional factors. The matrix in the image is highlighted by a red rectangle and expresses single value uncertainties by range and frequency.
CMC Uncertainty Example - Matrix per ILAC P14

 

CMC Uncertainty as a Range

In the image below, is an example of expressing CMC uncertainty as a range. This form of expression is not common, but it is typically used for S-parameter Transmission (S12/S21) and Reflection (S11/S22) measurement results. The range in the image is highlighted by a red rectangle.
CMC Uncertainty Example - Range per ILAC P14

 

FAQ

What CMC means?

The abbreviation “CMC” stands for Calibration and Measurement Uncertainty according to the International Committee for Weights and Measures (CIPM) and organizations under the CIPM and ILAC arrangements.

 
 

How to Calculate CMC Uncertainty Uncertainty?

CMC Uncertainty is calculated following ISOBudgets’ 7-step process based on GUM (JCGM 100) principles:

  1. Specify the Measurand.
  2. Identify Sources of Uncertainty.
  3. Quantify Sources of Uncertainty.
  4. Characterize Sources of Uncertainty.
  5. Convert Uncertainties to Standard Uncertainty.
  6. Calculate Combined Standard Uncertainty.
  7. Calculate Expanded Uncertainty.

The resulting expanded uncertainty will be your CMC uncertainty. Per ILAC P14, additional evaluations can be performed to express CMC Uncertainty as a mathematical function, range, or matrix. Otherwise, CMC uncertainty can be expressed a single value.

 
 

6 are acceptable; Must include contributions from the best existing device; and For laboratories that offer reference value provision services: Must include factors related to the measurement procedure as it is carried out on the sample (i.e. typical matrix effects, interferences, etc.), Generally will not include contributions due to the materials instability or inhomogeneity, Must be based on the inherent performance of the method for typical stable and homogeneous samples.”>

What are ILAC P14’s Requirements for Reporting CMC Uncertainty in Scope of Accreditation?

Per ILAC P14 Section 4, calibration uncertainty reported in a scope of accreditation must meet the following requirements:

  • Must be expressed in terms of:
    1. measurand or reference material;
    2. calibration or measurement method or procedure and type of instrument or material to be calibrated or measured;
    3. measurement range and additional parameters where applicable (e.g. frequency of applied voltage);
    4. measurement uncertainty.
  • Must be expressed using one or more of the following methods:
    1. A single value, which is valid throughout the measurement range.
    2. A measurement range. In this case a calibration laboratory shall ensure that linear interpolation is appropriate in order to find the uncertainty at intermediate values.
    3. An explicit function of the measurand and/or a parameter.
    4. A matrix where the values of the uncertainty depend on the values of the measurand and additional parameters.
    5. A graphical form, providing there is sufficient resolution on each axis to obtain at least two significant digits for the uncertainty.
  • Shall be no ambiguity in the smallest measurement uncertainty that the laboratory can achieve.
  • Must not be expressed as an open interval.
  • Must be expressed as an expanded uncertainty with a coverage probability of approximately 95 %.
  • Must be reported as an absolute uncertainty or relative uncertainty.
  • Must not use terms such as “PPM” or “PPB.” Instead, terms such as percent, μV/V, or part per 106 are acceptable.
  • Must include contributions from the best existing device.
  • For laboratories that offer reference value provision services:
    1. Must include factors related to the measurement procedure as it is carried out on the sample (i.e. typical matrix effects, interferences, etc.)
    2. Generally will not include contributions due to the materials instability or inhomogeneity.
    3. Must be based on the inherent performance of the method for typical stable and homogeneous samples.

 
 

What is uncertainty in a calibration certificate?

The uncertainty reported in calibration certificates is the uncertainty associated with the reported result or property value. It establishes an uncertainty interval (y – U < y < y + U) which the actual, true result should be within. Typically, the uncertainty reported is expressed at a 95.45 % confidence interval where the coverage factor (k) is 2.

Furthermore, it establishes a link in the metrological traceability chain to the International System of Units (SI).

This uncertainty is commonly referred to as “calibration uncertainty,” “reference standard uncertainty,” or “traceable uncertainty.”

 
 

What is the difference between accuracy and uncertainty in calibration?

The difference between the accuracy and uncertainty is:

  • Accuracy is a specification or measure of closeness of agreement. As a specification, it expresses an interval which an item’s performance must conform to ensure suitability for continued use (i.e. item must perform within specifications). However, it is defined as a “closeness of agreement between a measured value and a true value of a measurand” per the JCGM 200 (VIM), definition 2.13.
  • Uncertainty is an expression of doubt or quality. It establishes an interval which the actual or true value is expected to lie within. Per the JCGM 200 defintion 2.26, it is defined as a “non-negative parameter characterizing the dispersion of a quantity value being attributed to a measurand.”

An item can perform within accuracy specifications, but the uncertainty will allow you to determine quality or risk associated with the result and decisions made on conformity (e.g. Pass or Fail).

While accuracy and uncertainty are commonly confused, the example below will hopefully provide clarification. The image below comes from JCGM 106.

In the image below, you will see a result (y), a tolerance interval (TL, TU), and an uncertainty interval (y – u, y + u). Accuracy is the specification that established the tolerance interval. It can also be (mistakenly) expressed as bias or error from the nominal or target value (yi – yref). Uncertainty is the interval around the result (y). The actual value of y lies within this interval (y – u, y + u).

Conformance probability of measurement result and expanded uncertainty in a tolerance limit interval

If you notice that the uncertainty interval overlaps the upper tolerance limit, there is a chance (probability) that the item did not conform to accuracy specifications. While the result (y) was found to within the tolerance limits, the actual value of y is actually somewhere within the uncertainty interval (y – u, y + u). Since the uncertainty interval overlaps the tolerance limit, there is a small chance that result (y) is actually outside the tolerance limit. This would cause a Type I Error (i.e. False Acceptance).

Hopefully, this example provide clarity to understand the difference between the two terms.

 
 

Can I get consulting services to manage CMC uncertainty?

Yes. ISOBudgets manages CMC uncertainty for accredited laboratories. We estimate uncertainty, create uncertainty budgets, and manage the CMC uncertainties in your scope of accreditation.

Additionally, we will maintain a copy of your records (used to support the budgets and calculation) in you misplace it during an assessment.

Our uncertainty evaluations are widely accepted globally by many ILAC MRA signatories and commonly praised for their accuracy and completeness.

If you ever need to bring the capability inhouse to manage CMC uncertainty, we offer training and design our uncertainty workbooks in a way that make it easy for our clients to update and manage them their selves.

If you would like to learn more, book a free consultation by clicking the link below.

Book a Free Consultation for Uncertainty Budgets

 
 

What software tools are available for modeling CMC uncertainty in formulation development?

Most statistical software tools can evaluate and model CMC uncertainty. Some popular options include:

NIST has been promoting R software for uncertainty analysis and modeling for almost 15 years. JMP is popular among engineers and scientists, especially if they Design of Experiments (DOE) capabilities. SPSS and Minitab are quite popular for statistical analysis with some labs using them for uncertainty analysis. Microsoft Excel is the most popular software (by the total number of users and applications) for estimating measurement uncertainty.

Most of ISOBudgets uncertainty calculators are made with Microsoft Excel. The following calculators include the capability to evaluate and model CMC uncertainty functions and values:

If you need a custom template to evaluate and model CMC uncertainty, contact us to request a consultation.

Glossary

CMC Uncertainty
calibration and measurement capability available to customers under normal conditions: As published in the BIPM key comparison database (KCDB) of the CIPM MRA or as described in the laboratory’s scope of accreditation granted by an ILAC-MRA signatory. (Source: CIPM 96th Meeting)
Measurement Uncertainty
non-negative parameter characterizing the dispersion of the quantity values being attributed to a measurand, based on the information used. (JCGM 200:2012, 2.26)
Standard Measurement Uncertainty
measurement uncertainty expressed as a standard deviation. (Source: JCGM 200:2012, 2.30)
Expanded Measurement Uncertainty
the product of a combined standard measurement uncertainty and a factor larger than the number one. (Source: JCGM 200:2012, 2.35)
Level of Confidence
the likelihood that a set of measurement values are contained within a specified coverage interval. (Source: JCGM 200:2012, 2.37)
Relative Measurement Uncertainty
measurement uncertainty expressed in a term relative to the measurand.
Absolute Measurement Uncertainty
measurement uncertainty expressed in the same unit of measurement as the measurand.