• Table of Content
• Foreword
• Scope
• Uncertainty
• Analytical Measurement
• The Process
• Step 1: Specification
• Step 2: Identification
• Step 3: Quantification
• Step 4: Calculation
• Reporting Uncertainty
• Appendix A: Examples
  • Example 1: Calibration Standard
  • Example 2: Standardising NaOH
  • Example 3: An Acid/Base Titration
  • Example 4: In-house Validation Studies
  • Example 5: Determ. Cadmium Release
  • Example 6: Crude Fibre in Animal Feed
  • Example 7: Lead in Water - double IDMS
• Appendix B: Definitions
• Appendix C: Analytical Proc.
• Appendix D: Analyse U Sources
• Appendix E: Statistics
• Appendix F: MU at the Limit
• Appendix G: Common Sources
• Appendix H: Bibliography

Appendix - Example A4: Uncertainty Estimation from In-house Validation Studies - Determination of Organophosphorus Pesticides in Bread

Summary

Goal
The amount of an organophosphorus pesticide residue in bread is determined employing an extraction and a GC procedure.

Measurement Procedure
The stages needed to determine the amount of organophosphorus pesticide residue are shown in figure a4.1

Measurand:

Identification of the Uncertainty Sources:
The relevant uncertainty sources are shown in the cause and effect diagram in Figure A4.2.

Quantification of the Uncertainty Components:
Based on in-house validation data, the three major contributions are listed in table a4.1 and shown diagrammatically in figure A4.3 (values are from table A4.5).

Table A4.1: Uncertainties in pesticide analysis

Description	Value x	Standard uncertainty u(x)	Relative standard uncertainty u(x)/x	Comments
Repeatability(1)	1.0	0.27	0.27	Based on duplicate tests of different types of samples
Bias (Rec) (2)	0.9	0.043	0.048	Spiked samples
Other sources (3) (Homogeneity)	1.0	0.2	0.2	Estimation based on model assumptions
u(Pop)/Pop	1.0	0.34	0.34	Relative standard uncertainty

Figure A4.3: Uncertainties in pesticide analysis

The values of u(y,x_i)=(y/x_i).u(x_i) are taken from table a4.5

Example A4: Determination of Organophosphorus Pesticides in Bread: Detailed Discussion

A4.1 Introduction

This example illustrates the way in which in-house validation data can be used to quantify the measurement uncertainty. The aim of the measurement is to determine the amount of an organophosphorus pesticides residue in bread. The validation scheme and experiments establish traceability by measurements on spiked samples. It is assumed the uncertainty due to any difference in response of the measurement to the spike and the analyte in the sample is small compared with the total uncertainty on the result.

A4.2 Step 1: Specification

The specification of the measurand for more extensive analytical methods is best done by a comprehensive description of the different stages of the analytical method and by providing the equation of the measurand.

Procedure
The measurement procedure is illustrated schematically in figure A4.4. The separate stages are:

1. Homogenisation: The complete sample is divided into small (approx. 2 cm) fragments, a random selection is made of about 15 of these, and the sub-sample homogenised. Where extreme inhomogeneity is suspected proportional sampling is used before blending.
2. Weighing of sub-sampling for analysis gives mass m_sample
3. Extraction: Quantitative extraction of the analyte with organic solvent, decanting and drying through a sodium sulphate columns, and concentration of the extract using a Kuderna-Danish apparatus.
4. Liquid-liquid extraction:
5. Acetonitrile/hexane liquid partition, washing the acetonitrile extract with hexane, drying the hexane layer through sodium sulphate column.
6. Concentration of the washed extract by gas blown-down of extract to near dryness.
7. Dilution to standard volume V_op (approx. 2 ml) in a 10 ml graduated tube.
8. Measurement: Injection and GC measurement of 5l of sample extract to give the peak intensity I_op.
9. Preparation of an approximately 5 g ml^-1 standard (actual mass concentration c_ref).
10. GC calibration using the prepared standard and injection and GC measurement of 5 l of the standard to give a reference peak intensity I_ref.

Calculation
The mass concentration c_op in the final sample is given by

and the estimate P_op of the level of pesticide in the bulk sample (in mg kg^-1) is given by

or, substituting for c_op,

Scope
The analytical method is applicable to a small range of chemically similar pesticides at levels between 0.01 and 2 mg kg^-1 with different kinds of bread as matrix.

A4.3 Step 2: Identifying and Analysing Uncertainty Sources

The identification of all relevant uncertainty sources for such a complex analytical procedure is best done by drafting a cause and effect diagram. The parameters in the equation of the measurand are represented by the main branches of the diagram. Further factors are added to the diagram, considering each step in the analytical procedure (a4.2), until the contributory factors become sufficiently remote.

The sample inhomogeneity is not a parameter in the original equation of the measurand, but it appears to be a significant effect in the analytical procedure. A new branch, F(hom), representing the sample inhomogeneity is accordingly added to the cause and effect diagram (figure a4.5).

Finally, the uncertainty branch due to the inhomogeneity of the sample has to be included in the calculation of the measurand. To show the effect of uncertainties arising from that source clearly, it is useful to write

where F_hom is a correction factor assumed to be unity in the original calculation. This makes it clear that the uncertainties in the correction factor must be included in the estimation of the overall uncertainty. The final expression also shows how the uncertainty will apply.

NOTE: Correction factors: This approach is quite general, and may be very valuable in highlighting hidden assumptions. In principle, every measurement has associated with it such correction factors, which are normally assumed unity. For example, the uncertainty in c_op can be expressed as a standard uncertainty for c_op, or as the standard uncertainty which represents the uncertainty in a correction factor. In the latter case, the value is identically the uncertainty for c_op expressed as a relative standard deviation.

A4.4 Step 3: Quantifying Uncertainty Components

In accordance with section 7.7., the quantification of the different uncertainty components utilises data from the in-house development and validation studies:

• The best available estimate of the overall run to run variation of the analytical process.
• The best possible estimation of the overall bias (Rec) and its uncertainty.
• Quantification of any uncertainties associated with effects incompletely accounted for the overall performance studies.

Some rearrangement the cause and effect diagram is useful to make the relationship and coverage of these input data clearer (figure A4.6).

NOTE: In normal use, samples are run in small batches, each batch including a calibration set, a recovery check sample to control bias and random duplicate to check precision.

Corrective action is taken if these checks show significant departures from the performance found during validation. This basic QC fulfils the main requirements for use of the validation data in uncertainty estimation for routine testing.

Having inserted the extra effect "Repeatability" into the cause and effect diagram, the implied model for calculating P_op becomes

eq.a4.1

That is, the repeatability is treated as a multiplicative factor F_Rep like the homogeneity. This form is chosen for convenience in calculation, as will be seen below.

The evaluation of the different effects is now considered.

1. Precision Study

The overall run to run variation (precision) of the analytical procedure was performed with a number of duplicate tests (same homogenised sample, complete extraction/determination procedure) for typical organophosphorus pesticides found in different bread samples. The results are collected in table a4.2.

Table a4.2: Results of Duplicate Pesticide Analysis

Residue	D1 [mg kg-1]	D2 [mg kg-1]	Mean [mg kg-1]	Difference D1-D2	Difference/ mean
Malathion	1.30	1.30	1.30	0.00	0.000
Malathion	1.30	0.90	1.10	0.40	0.364
Malathion	0.57	0.53	0.55	0.04	0.073
Malathion	0.16	0.26	0.21	-0.10	-0.476
Malathion	0.65	0.58	0.62	0.07	0.114
Pirimiphos Methyl	0.04	0.04	0.04	0.00	0.000
Chlorpyrifos Methyl	0.08	0.09	0.085	-0.01	-0.118
Pirimiphos Methyl	0.02	0.02	0.02	0.00	0.000
Chlorpyrifos Methyl	0.01	0.02	0.015	-0.01	-0.667
Pirimiphos Methyl	0.02	0.01	0.015	0.01	0.667
Chlorpyrifos Methyl	0.03	0.02	0.025	0.01	0.400
Chlorpyrifos Methyl	0.04	0.06	0.05	-0.02	-0.400
Pirimiphos Methyl	0.07	0.08	0.75	-0.10	-0.133
Chlorpyrifos Methyl	0.01	0.01	0.10	0.00	0.000
Pirimiphos Methyl	0.06	0.03	0.045	0.03	0.667

The normalised difference data (the difference divided by the mean) provides a measure of the overall run to run variability. To obtain the estimated relative standard uncertainty for single determinations, the standard deviation of the normalised differences is taken and divided by to correct from a standard deviation for pairwise differences to the standard uncertainty for the single values. This gives a value for the standard uncertainty due to run to run variation of the overall analytical process, including run to run recovery variation but excluding homogeneity effects, of

NOTE: At first sight, it may seem that duplicate tests provide insufficient degrees of freedom. But it is not the goal to obtain very accurate numbers for the precision of the analytical process for one specific pesticide in one special kind of bread. It is more important in this study to test a wide variety of different materials and sample levels, giving a representative selection of typical organophosphorus pesticides. This is done in the most efficient way by duplicate tests on many materials, providing (for the repeatability estimate) approximately one degree of freedom for each material studied in duplicate.

2. Bias Study

The bias of the analytical procedure was investigated during the in-house validation study using spiked samples (homogenised samples were split and one portion spiked). Table a4.3 collects the results of a long term study of spiked samples of various types.

Table A4.3: Results of Pesticide Recovery Studies

Substrate	Residue Type	Conc. [mg kg-1]	N1)	Mean2) [%]	s2) [%]
Waste Oil	PCB	10.0	8	84	9
Butter	OC	0.65	33	109	12
Compound Animal Feed I	OC	0.325	100	90	9
Animal & Vegetable Fats I	OC	0.33	34	102	24
Brassicas 1987	OC	0.32	32	104	18
Bread	OP	0.13	42	90	28
Rusks	OP	0.13	30	84	27
Meat & Bone Feeds	OC	0.325	8	95	12
Maize Gluten Feeds	OC	0.325	9	92	9
Rape Feed I	OC	0.325	11	89	13
Wheat Feed I	OC	0.325	25	88	9
Soya Feed I	OC	0.325	13	85	19
Barley Feed I	OC	0.325	9	84	22

1. The number of experiments carried out
2. The mean and sample standard deviation s are given as percentage recoveries.

The relevant line (marked with yellow colour) is the "bread" entry line, which shows a mean recovery for forty-two samples of 90%, with a standard deviation (s) of 28%. The standard uncertainty was calculated as the standard deviation of the mean

.

A significance test is used to determine whether the mean recovery is significantly different from 1.0. The test statistic t is calculated using the following equation

This value is compared with the 2-tailed critical value t_crit, for n-1 degrees of freedom at 95% confidence (where n is the number of results used to estimate ). If t is greater or equal than the critical value t_crit than is significantly different from 1.

In this example a correction factor (1/) is being applied and therefore is explicitly included in the calculation of the result.

3. Other Sources of Uncertainty

The cause and effect diagram in figure a4.7 shows which other sources of uncertainty are (1) adequately covered by the precision data, (2) covered by the recovery data or (3) have to be further examined and eventually considered in the calculation of the measurement uncertainty.

1. Repeatabilty (F_Rep in equation a4.1) considered during the variability investigation of the analytical procedure.
2. Considered during the bias study of the analytical procedure.
3. To be considered during the evaluation of the other sources of uncertainty.

All balances and the important volumetric measuring devices are under regular control. Precision and recovery studies take into account the influence of the calibration of the different volumetric measuring devices because during the investigation various volumetric flasks and pipettes have been used. The extensive variability studies, which lasted for more than half a year, also cover influences of the environmental temperature on the result. This leaves only the reference material purity, possible nonlinearity in GC response (represented by the 'calibration' terms for I_ref and I_op in the diagram), and the sample homogeneity as additional components requiring study.

The purity of the reference standard is given by the manufacturer as 99.53% ±0.06%. The purity is potential an additional uncertainty source with a standard uncertainty of

(Rectangular distribution). But the contribution is so small (compared, for example, to the precision estimate) that it is clearly safe to neglect this contribution.

Linearity of response to the relevant organophosphorus pesticides within the given concentration range is established during validation studies. In addition, with multi-level studies of the kind indicated in table a4.2 and table a4.3, nonlinearity would contribute to the observed precision. No additional allowance is required. The in-house validation study has proven that this is not the case.

The homogeneity of the bread sub-sample is the last remaining other uncertainty source. No literature data were available on the distribution of trace organic components in bread products, despite an extensive literature search (at first sight this is surprising, but most food analysts attempt homogenisation rather than evaluate inhomogeneity separately). Nor was it practical to measure homogeneity directly. The contribution has therefore been estimated on the basis of the sampling method used.

To aid the estimation, a number of feasible pesticide residue distribution scenarios were considered, and a simple binomial statistical distribution used to calculate the standard uncertainty for the total included in the analysed sample (see section a4.6). The scenarios, and the calculated relative standard uncertainties in the amount of pesticide in the final sample, were:

• Residue distributed on the top surface only: 0.58.
• Residue distributed evenly over the surface only: 0.20.
• Residue distributed evenly through the sample, but reduced in concentration by evaporative loss or decomposition close to the surface: 0.05-0.10 (depending on the "surface layer" thickness).

Scenario (a) is specifically catered for by proportional sampling or complete homogenisation: It would arise in the case of decorative additions (whole grains) added to one surface. Scenario (b) is therefore considered the likely worst case. Scenario (c) is considered the most probable, but cannot be readily distinguished from (b). On this basis, the value of 0.20 was chosen.

NOTE: For more details on modelling inhomogeneity see the last section of this example.

A4.5 Step 4: Calculating the Combined Standard Uncertainty

During the in-house validation study of the analytical procedure the repeatability, the bias and all other feasible uncertainty sources had been thoroughly investigated. Their values and uncertainties are collected in table a4.4.

Table A4.4: Uncertainties in Pesticide Analysis

Description	Value x	Standard uncertainty u(x)	Relative standard uncertainty u(x)	Comments
Repeatability(1)	1.0	0.27	0.27	Duplicate tests of different types of samples
Bias (Rec) (2)	0.9	0.043	0.048	Spiked samples
Other sources (3) (Homogeneity)	1.0	0.2	0.2	Estimation founded on model assumptions
u(Pop)/Pop	1.0	0.34	0.34	Relative standard uncertainty

The relative values are combined because the model (equation a4.1) is entirely multiplicative:

The spreadsheet for this case (table a4.5) takes the form shown in table a4.5. Note that the spreadsheet calculates an absolute value uncertainty (0.373) for a nominal corrected result of 1.1111, giving a value of 0.373/1.11=0.34.

Table A4.5: Uncertainties in Pesticides Analysis

"	A	B	C	D	E
1			Repeatability	Bias	Homogeneity
2		value	1.0	0.9	1.0
3		uncertainty	0.27	0.043	0.2
4
5	Repeatability	1.0	1.27	1.0	1.0
6	Bias	0.9	0.9	0.943	0.9
7	Homogeneity	1.0	1.0	1.0	1.2
8
9	Pop	1.1111	1.4111	1.0604	1.333
10	u(y,xi)		0.30	-0.0507	0.222
11	u(y)2,u(y,xi)2	0.1420	0.09	0.00257	0.4938
12
13	u(Pop)	0.377	(0.377/1.111 = 0.34 as a relative standard uncertainty)

The values of the parameters are entered in the second row from C2 to E2. Their standard uncertainties are in the row below (C3:E3). The spreadsheet copies the values from C2-E2 into the second column from B5 to B7. The result using these values is given in B9 (=B5xB7/B6, based on equation A4.1). C5 shows the value of the repeatability from C2 plus its uncertainty given in C3. The result of the calculation using the values C5:C7 is given in C9. The columns D and E follow a similar procedure. The values shown in the row 10 (C10:E10) are the differences of the row (C9:E9) minus the value given in B9. In row 11 (C11:E11) the values of row 10 (C10:E10) are squared and summed to give the value shown in B11. B13 gives the combined standard uncertainty, which is the square root of B11.

The relative sizes of the three different contributions can be compared by employing a histogram. Figure a4.8 shows the values |u(y,xi)| taken from table A4.5.

Figure A4.8: Uncertainties in pesticide analysis

The values of u(y,x_i)=(y/x_i).u(x_i) are taken from table a4.5

The repeatability is the largest contribution to the measurement uncertainty. Since this component is derived from the overall variability in the method, further experiments would be needed to show where improvements could be made. For example, the uncertainty could be reduced significantly by homogenising the whole loaf before taking a sample.

The expanded uncertainty U(P_op) is calculated by multiplying the combined standard uncertainty with a coverage factor of 2 to give:

A4.6 Special Aspect: Modelling Inhomogeneity for Organophosphorus Pesticide Uncertainty

Assuming that all of the material of interest in a sample can be extracted for analysis irrespective of its state, the worst case for inhomogeneity is the situation where some part or parts of a sample contain all of the substance of interest. A more general, but closely related, case is that in which two levels, say L₁ and L₂ of the material are present in different parts of the whole sample. The effect of such inhomogeneity in the case of random sub-sampling can be estimated using binomial statistics. The values required are the mean and the standard deviation of the amount of material in n equal portions selected randomly after separation.

These values are given by

[1]

[2]

where l₁ and l₂ are the amount of substance in portions from regions in the sample containing total fraction L₁ and L₂ respectively, of the total amount X, and p₁ and p₂ are the probabilities of selecting portions from those regions (n must be small compared to the total number of portions from which the selection is made).

The figures shown above were calculated as follows, assuming that a typical sample loaf is approximately cm, using a portion size of cm (total of 432 portions) and assuming 15 such portions are selected at random and homogenised.

Scenario (a)
The material is confined to a single large face (the top) of the sample. L₂ is therefore zero as is l₂; and L₁=1. Each portion including part of the top surface will contain an amount l₁ of the material. For the dimensions given, clearly one in six (2/12) of the portions meets this criterion, p₁ is therefore 1/6, or 0.167, and l₁ is X/72 (i.e. there are 72 "top" portions).

This gives

NOTE: To calculate the level X in the entire sample, is multiplied back up by 432/15, giving a mean estimate of X of

This result is typical of random sampling; the expectation value of the mean is exactly the mean value of the population. For random sampling, there is thus no contribution to overall uncertainty other that the run to run variability, expressed as or RSD here.

Scenario (b)
The material is distributed evenly over the whole surface. Following similar arguments and assuming that all surface portions contain the same amount l₁ of material, l₂ is again zero, and p₁ is, using the dimensions above, given by

i.e. p1 is that fraction of sample in the "outer" 2 cm. Using the same assumptions then

.

NOTE: The change in value from scenario (a)

This gives:

Scenario (c)
The amount of material near the surface is reduced to zero by evaporative or other loss. This case can be examined most simply by considering it as the inverse of scenario (b), with p₁=0.37 and l₁ equal to X/160. This gives

However, if the loss extends to a depth less than the size of the portion removed, as would be expected, each portion contains some material l₁ and l₂ would therefore both be non-zero. Taking the case where all outer portions contain 50% "centre" and 50% "outer" parts of the sample

giving an RSD of

In the current model, this corresponds to a depth of 1 cm through which material is lost. Examination of typical bread samples shows crust thickness typically of 1 cm or less, and taking this to be the depth to which the material of interest is lost (crust formation itself inhibits lost below this depth), it follows that realistic variants on scenario (c) will give values of not above 0.09.

NOTE: In this case, the reduction in uncertainty arises because the inhomogeneity is on a smaller scale than the portion taken for homogenisation. In general, this will lead to a reduced contribution to uncertainty. It follows that no additional modelling need be done for cases where larger numbers of small inclusions (such as grains incorporated in the bulk of a loaf) contain disproportionate amounts of the material of interest. Provided that the probability of such an inclusion being incorporated into the portions taken for homogenisation is large enough, the contribution to uncertainty will not exceed any already calculated in the scenarios above.