拸۵

02/03/55

21

Slide 41

CCCCOOOOMMMM

PPPPAAAA

RRRRABILITYABILITYABILITYABILITY

EEEECCCC

AAAARRRR

TTTT

IN MEASUREMENTS

OF AMOUNT OF SUBSTANCE

©1992: P De Bievre

Slide 42

1. Results required to be related to stated references, such as Reference Materials (RMs)

�ހ

02/03/55

22

Slide 43

“Tested once, accepted everywhere”

Reliable and accurate measurements

Competent staff

Validated procedure

Comprehensive quality system

Traceability to stated reference

Comparability of results can be achieved through the link of all

individual measurement results to some common, stable reference or

measurement standard. This linking of results to a reference is termed

“traceability”.

02/03/55 Slide 44

Measurement in Measurement in Measurement in Measurement in

chemistrychemistrychemistrychemistry

44

�ހ ¾

02/03/55

23

� Measurement is set of operations having the object of determining a value of a quantity

� Chemical analytical measurements are about measuring “something” in “something”

◦ Identifying and counting

◦ How much there is of a specified substance (chemical element, molecular species) in a simple, complex or very complex material (water, sediment, soil, blood)

� Number and its unit including an uncertainty

45

quantity intended to be measured

� quantity we measure:

e.g, length, mass, temperature, volume, pressure, amount

concentration, mass concentration, mass fraction, electric current,

…

� the specification of the measurand results from the intention of the

analyst about what (s)he is going to measure

� it is a decisionmade before the measurement

� the measurand can be operationally defined, e.g.,

leachable Cd concentration in a ceramic plate, measured

according to a specified procedure

since we cannot specify perfectly any measurand, there is a ‘definitional

uncertainty’ associated with the measurand which must appear in our

uncertainty budget after the measurement

see ACQUAL 12(2007)561-562, 12(2007)613-61446

�ހ

02/03/55

24

ISO/IEC 17025 Clause 5.4.2

Method published in international, regional or national standards shall preferably be used

ISO/IEC 17025 Clause 5.4.4

The method developed shall have been validatedappropriately before use

47

ISO defines Validation as:

‘is the confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use are fulfilled’

ISO/IEC 17025 clause 5.4.5.1

From Eurachem:

“…the process of defining an analytical requirement, and

confirming that the method under consideration has

performance capabilities consistent with what the application

requires. Implicit in this is that it will be necessary to evaluate

the method’s performance capabilities. …The judgement of

method suitability is important; in the past method validation

has tended to concentrate on the process of evaluating the

performance parameters.”

-The fitness for Purpose of Analytical Methods, 3.1

48

�ހ

02/03/55

25

The latter definition identifies three discrete stages of methodvalidation:

1. Define the Analytical Requirement(s)

2. Determine (by collection and analysis of relevant data)the Appropriate Method Performance Metrices

3. Confirm the Adequacy (based on the determinedperformance metrics) of the Method to Meet the EstablishedAnalytical Requirement(s)

Method validation is a separate process from development andoptimization. It is an objective, systematic, evaluation that can onlybe undertaken AFTER development and optimization are complete.Unsatisfactory validation data may dictate the need for additionaldevelopment and optimization work, after which a new validationmust be executed.

49

Validation may be either a single laboratoryvalidation (SLV) or a collaborative study (CS) involvingmultiple laboratories.

SLV- Provides estimates of all performance metrics

(except reproducibility) within a single laboratory

CS- Usually designed to estimate only repeatability

and combined intermediate precision/ reproducibility. Can

be designed to give explicit estimates of intermediate

precision and reproducibility.

Collaborative studies are virtually always preceded by a

complete SLV, which is required to evaluate readiness forundertaking a CS.

50

拸۵

02/03/55

26

‘is the confirmation by examination and provision of evidence that specified requirements have been met’

ISO/IEC 17025

51

� There are minimum procedures which are fundamental to

any laboratory, and include the following:

◦ Documentation procedures

◦ Procedures for labeling, preparation and storage of

chemicals, solutions and reagents

◦ Environmental monitoring (measuring temperature-

critical items such as lab refrigerators, freezers,

incubators, etc.)

◦ General lab housekeeping (cleaning, waste disposal)

◦ Calibration of instrument and equipment with

measurable components (including balances,

thermometers, pipettes, etc.)

◦ Laboratory instrument and equipment maintenance

procedures

◦ Analyst training

Basic laboratory procedureBasic laboratory procedureBasic laboratory procedureBasic laboratory procedure

52

�ހ

02/03/55

27

� Work to an agreed customer requirement

� Use validated methods and calibrated equipment

� Staff are qualified and competent

� Participate in independent assessments of technical performance

� Use sound quality assurance and quality control practices

� metrological traceability (Ensure comparability with measurements made in other laboratories)

53

VAM, LGC, UK

� Documented evidence

� High degree of assurance

� Specific process

� Consistently

� Predetermined specifications

54

�ހ

02/03/55

28

� IUPAC Technical Report (2002) (International Union of

Pure and Applied Chemistry)

� NATA Technical Note 17 (National Association of

Testing Authorities)

� ICH Guidelines (2005) (International Conference on

Harmonization)

� Joint FAO / WHO Food Standards Programme (Food

and Agriculture Organization, World Health

Organization)

� USFDA Bioanalytical Method Validation Guidance for

Industry (2001) (Food and Drug Administration)

55

� Non-standard methods

� Laboratory-designed/develop methods

� Standard methods used outside of their intended

scope

� Amplifications and modifications of standard

methods

ISO/IEC 17025 clause 5.4.5.2

56

�ހ

02/03/55

29

◦ New method developed for particular problem

◦ Established method revised to incorporate improvements or extended to a new problem

◦ When quality control indicates and established method is changing with time

◦ Established method used in a different laboratory, or with different analyst or different instrumentation

◦ To demonstrate the equivalence between two methods

◦ Major instruments replaced

◦ New batches of standards

◦ Used of validated method after out-of use for a long time

57

Fit for Purpose?

�Demonstration that a method is “fit for purpose” is the

ultimate goal of the validation process.

� One widely utilized measure of acceptable method

performance (precision) in a SLV is the Horwitz Ratio.

Although most often, and appropriately used for evaluating

data from collaborative studies, it is sometimes applied to

SLVs. For repeatability:

Horratr = RSDr

PRSDr

Where RSDr = repeatability relative standard deviation from validation data and

PRSDr = repeatability relative standard deviation predicted by the Horwitz

equation:

PRSDr = C-0.15

Only precision

is addressed/

RSDs tend to

be overestimated

58

�ހ

02/03/55

30

59

Acceptance criteria

� HorRat = Horwitz Ratio =

Reference HORRAT

AOAC

Codex, EU

ectedexp

obs

RSD

RSD

2≤

2<

59

60

� This equation was defined by Horwitz et al1 from a practical consideration of a number of collaborative studies done by AOAC over many years.

� Reproducibility, CVR

� Repeatability, CVr

C = concentration of analyte in term of decimal fraction

1505.0R

)Clog5.01(R

C2CV

2CV

−

−

=

=

)Clog5.01(r 266.0CV −

×=

60

�ހ

02/03/55

31

61

Horwitz noted that values for RSDr (the repeatability CV) were usually between half and two-thirds that of RSDR.

For this reason, repeatability acceptabilities are proposed as the Horwitz values for RSDR

x 0.67.

61

� Confirmation of identity

� Specificity/Selectivity

� Limit of detection (LOD)

� Limit of quantitation (LOQ)

� Linearity and working range

� Sensitivity

� Accuracy

� Precision

� Ruggedness and Robustness

� Measurement Uncertainty

� Metrological Traceability

62

�ހ

02/03/55

32

Key performance characteristic

Major compounds-quantitative

Major compounds and traces-

quantitative

Traces-qualitative

Traces-quantitative

LOD

LOQ

Linearity

Range

Precision

Accuracy

Specificity

Ruggedness

N

N

Y

Y

Y

Y

Y

Y

N

Y

Y

Y

Y

Y

Y

Y

Y

N

N

N

N

N

Y

N

N

Y

Y

N

Y

Y

Y

Y

63

From: Validation and Qualification in Analytical Laboratories by Ludwig Huber

64

Confirmation is an analytical process where a measurement is performed by more than onetechnique, to conform the presence of an analyte

That the entity you are observing and measuring some quantity is what you think it is!

-Speciation-Interferences and impurities

CITAC/Eurachem Guide to Quality in Analytical

Chemistry: An Aid to Accreditation (2002)

拸۵

02/03/55

33

� Specific: a method that produces a response for a target analyte only

� Selective: a method that provides responses for a number of chemical entities that may or may not be distinguished from each other

- The ability to measure accurately an analyte in the presence of interferences (USP)

- Analyze at least five independent sources of control matrix to assess interference effects

How to improve selectivity

- Good sample preparation

- Selective instrument e.g. mass spectrometer vs. UV spectrometer

65

Procedure

◦ Analyze samples and reference materials by

candidate and other independent methods

◦ Analyze samples containing various suspected

interferences in the presence of the analyte of

interest

◦ If detection or quantification is inhibited by the

interferences, further method development will be

required

Eurachem Guide

66

�ހ

02/03/55

34

What is the smallest concentration that can be detected?

� The smallest amount or concentration of an analyte that can be reliably distinguished from zero with a specified level of confidence

� The lowest value measured by a method that is greater than the uncertainty associated with it (NATA technical Note 17, 2006)

67

Sample blank produce signal

◦ Single measurement of each 10 independent sample blank, then calculate mean and standard deviation, s

No signal from sample blank

◦ Single measurement of each 10 independent sample blank fortified at lowest acceptable concentration,

then calculate standard deviation (s)

LOD = mean of sample blank +3s LOD = 0+3s

LOD = 0+4.65s

(Eurachem Guide page 17-18)

68

�ހ

02/03/55

35

� LOD determined from calibration curve

◦ is the standard error of the regression

◦ is the slope of the linear calibration

xys /

b

b

sc

xy

dl

/3=

∑ −++=

−

i

i

xyn

dlcc

c

nmb

stc

)(

1122

2

/2,'05.0 From IUPAC

69

� Sample blanks spiked with the analyte at a

range of concentration levels

� Measure 10 independent replicates at

each concentration level

� Calculate % positive or % negative at each

concentration level

� LOD is the lowest concentration at which

the test becomes unreliable

70

�ހ

02/03/55

36

Spiked conc, mg/kg No. of replicates Positive/negative results

200 10 10/0

100 10 10/0

80 10 8/2

60 10 5/5

71

If the criteria is 100% positive: cut-off (threshold) is 100 mg/kg =LOD

Limit of Detection (LOD) for Limit of Detection (LOD) for Limit of Detection (LOD) for Limit of Detection (LOD) for

Qualitative MeasurementQualitative MeasurementQualitative MeasurementQualitative Measurement

What is the smallest concentration that can be

measured with a given measurement uncertainty?

� The lowest concentration of analyte that can be

determined with an acceptable level of

repeatability precision and trueness

(Eurachem Guide)

� The concentration of analytes that can be

determined with an acceptable uncertainty (NATA technical Note 17)

72

�ހ

02/03/55

37

� Sample blank produce signal◦ Single measurement of each 10 independent sample blank

� No signal from sample blank◦ Single measurement of each 10 independent sample blank

fortified at different concentration close to LOD

Calculate standard deviation (s) at each concentration level that can be quantify with the acceptable uncertainty

LOQ = mean sample blank value +5s (or 6s or 10s)LOQ = the lowest analyte conc. which can be

determined with acceptable uncertainty(Eurachem Guide page 19)

73

What is the range over which the calibration model holds?

� Working or measuring range is the set if values of measurands for which the error of a measuring instrument is intended to lie within specified limits

(Eurachem Glossary A18)

� Linear range is by inference the range of analytical concentrations over which the method gives test results proportional to the concentration of the analyte

(Eurachem Glossary A12)

74

�ހ

02/03/55

38

0.2

0.4

0.6

0.8

1.0

0.2 0.4

0.6

0.8 1.0 1.20

LOD

LOQ

Working range

Linear range

Concentration

Response

Linearity and Working RangeLinearity and Working RangeLinearity and Working RangeLinearity and Working Range

75

1. Blank plus RMs or fortified sample blanks at

various concentrations (at least 6 conc. Plus

blank), single measurement at each

concentration

◦ Plot the responds against concentration of standard

◦ Different concentrations should be prepared

independently, and not from aliquots of the

same master solution

76

�ހ

02/03/55

39

2. at least 6 different concentrations within the linear range, 3 replicate measurements at each conc.

� Plot the responds against concentration of standard

� Calculate regression coefficient

� Calculate and plot the residues at each concentration

Random distribution of residual about the straight line confirms linearity. Systematic trends indicate non-linearity

If non-linear curve is necessary, functions higher than quadratic are not advised

77

The change in the response of a measuring instrument divided by the corresponding change in the stimulus

No specification for sensitivity, consider the uncertainty of a

result

78

�ހ

02/03/55

40

Do we get the right answer?

Accuracy is the quality of the result in terms of “Trueness” and “ Precision” in relation to the requirements of its use

Trueness is the closeness of agreement between the average value obtained from a large set of test results and accepted reference value

Bias is the different between the expectation of the test results and an accepted reference value

79

1. Analyze reagent blank and CRM using

candidate method

� 10 replicates

� Calculate the average concentration of CRM

(with blank subtraction) determined by candidate method

� Difference between values obtained by

candidate method and certify value can be used to estimate method bias

“Method’s Bias”

80

�ހ

02/03/55

41

2. Analyze reagent blank and RM using candidate and independent method preferably primary method

- 10 replicates

- Calculate the average concentration of RM (with blank subtraction)

determined by candidate method

- Calculate the average concentration of RM (with blank subtraction) determined by independent/primary method

- Difference between results obtained from these 2 methods can be used to estimate method bias

“Method’s Bias” relative to independent/primary method

81

The fraction of analyte added to a test sample (fortified or

spiked sample) prior to analysis, the unfortified and fortified samples, percentage recover (%R) is calculated as follows:

% Recovery = (C1-C2)x100/C3

C1= concentration determined in fortified sample

C2 = concentration determined in unfortified sample

C3 = concentration of fortification

Analyse matrix blank, CRM or sample fortified with interested

analyte at different concentration level (low mid high) 6

replicates at each concentration and calculate % recovery

82

�ހ

02/03/55

42

Analyte % Analyte ratio Unit Mean recovery

(%)

100 1 100% 98-102

10 0.1 10% 98-102

1 0.01 1% 97-103

0.1 0.001 0.1% 95-105

0.01 0.0001 100 ppm 90-107

0.001 0.00001 10 ppm 80-110

0.0001 0.000001 1 ppm 80-110

0.00001 0.0000001 100 ppb 80-110

0.000001 0.00000001 10 ppb 60-115

0.0000001 0.000000001 1ppb 40-120

83

AOAC manual for the Peer-Verified Methods Program (November 1993)

Can we reproduce the result?

� Repeatability: ◦ Same lab, same analyst, same instrument, relative short time span

◦ At least 6 determinations of 3 different matrices at 2 or 3 different concentrations the calculate the RSD (ICH)

◦ At least 9 replicates covering the complete specified range e.g. 3 concentrations with 3 injections at each concentration (ICH)

◦ At least 10 injections with RSD <1% (US EPA)

84

�ހ

02/03/55

43

� 95% confidence interval on the difference between

duplicate results measured under repeatability

conditions

rr ssr 8.222 ==

85

Analyte % Analyte Ratio Unit RSD (%)

100 1 100% 1.3

10 10-1 10% 1.8

1 10-2 1% 2.7

0.1 10-3 0.1% 3.7

0.01 10-4 100 ppm 5.3

0.001 10-5 10 ppm 7.3

0.0001 10-6 1 ppm 11

0.00001 10-7 100 ppb 15

0.000001 10-8 10 ppb 21

0.0000001 10-9 1 ppb 30

86AOAC manual for the Peer-Verified Methods Program (November 1993)

Expected %RSDr calculated from Horwitz’s equation = 0.66x2 (1-0.5logC)

�ހ

02/03/55

44

� Intermediate precision (ICH)◦ The long term variability of the measurement

process

◦ Comparing the results of a method run within a

single lab over a number of weeks

87

� Reproducibility (ICH)

- Precision obtained from different labs, different analysts, different operational and environmental conditions

- Analyzing aliquots from homogeneous lots

“Inter-laboratory tests”

Reproducibility Limit

88

RR ssR 8.222 ==

�ހ

02/03/55

45

Precision

Intermediate Precision

Reproducibility

Instrument same different different

Batches of accessories e.g. columns

same different different

Analyst same different different

Sample matrices same different different

Concentration same different different

Batches of material e.g. reagents

Same different different

Environmental conditions Same different different

Laboratory same same different

89

When there are small changes in conditions do we get the same answers?Robustness of an analytical procedure is a measure

of its capability to remain unaffected by small, but

deliberate variations in method parameters and

provided an indication of its reliability during normal

usage (Eurachem Glossary A25.2)

90

�ހ

02/03/55

46

� Identify parameters that will affect performance of test method

� Study the effect by RMs or CRMs

� Evaluate the effect of each parameters

� Improve the test method or control the effect

91

� Human factors◦ Control via education and on the job training

� Technical factors◦ Stability of analytical solutions◦ Sample preparation◦ Influence of variation of pH, temperature, flow rate, etc

� Environmental factors◦ Temperature, pressure, humidity, pollution, contamination

92

�ހ

02/03/55

47

Techniques used for ruggedness testing◦ Significance testing, F-test� Are the sets of data significantly different?

◦ Factorial design� Allow investigation of the interaction of variables� More economical when variables are less than 5

◦ Plackett-Burman method design� Assumes interaction effects are negligible � Calculate the effect of each factor on a measurable

quantity

◦ Youden-Steiner testing� Assumes interaction effects are negligible � Calculate the effect of each factor on a measurable

quantity

93

� JCGM 100:2008 – Guide to the Expression of Uncertainty in Measurement – The GUM (2008)◦ Available for free from the BIPM (www.bipm.org)

� Eurachem Guide – Quantifying Uncertainty in Analytical Measurement (2000) (http://www.eurachem.ul.pt/)

� NATA Technical Note 33 - Guidelines for Estimating and Reporting Measurement Uncertainty in Chemical Test Results

� Nordtest Report – Handbook for Calculation of Measurement Uncertainty in Environmental Laboratories (2004)

� Eurolab Technical Report No. 1/2007 – Measurement Uncertainty Revisited: Alternative approaches to uncertainty evaluation (2007)

� http://www.measurementuncertainty.org/mu/examples/

94

�ހ

02/03/55

48

Degree of external validation

1. The method is externally validated in a method-performance study

2. The method is externally validated but is used on a new matrix or using new instrument(s)

3. Well established but not tested method

Recommended internal validation

1. Verification of trueness and precision

2. Verification of trueness and precision, possibly also detection limit

3. Verification, possibly a more extensive validation

95

Recommended internal validation

4. Verification, possibly a more extensive validation

5. The method needs to be fully validated

6. The method needs to be fully validated

Degree of external validation

4. The method is published in the scientific literature and states importance characteristics

5. The method is published in scientific literature without presentation of performance characteristics

6. The method was internally developed

96

�ހ

02/03/55

49

1. Design a validation or protocol: - The needs of the client- What is analytical achievable- Conditions of lab (environment, equipment)

2. Determine specificity and standard curve3. Determine precision, expressed as repeatability and

reproducibility4. Determine trueness5. Determine working range/measuring range6. Determine detection limit7. Determine limit of quantification/limit of determination8. Determine robustness9. Determine sensitivity10. Evaluate the result from step2-8 against the protocol11. Documented the work in a report

97

� Blanks: reagent blanks and sample blanks

� Fortified/Spiked samples

� Incurred materials

� Reference materials (RMs)

� Certified reference materials (CRMs)

98

�ހ

02/03/55

50

Reagent blanks (solvent blanks)◦ Reagents used during the analytical process (including

solvents used for extraction or dissolution) are analyzed in isolation in order to see whether they contribute to the measurement signal. The measurement signal arising from the analyte can then be corrected accordingly

Sample blanks (matrix blanks)◦ The matrices with no analyte. They are difficult to obtain but

such materials are necessary to give a realistic estimate of interferences that would be encountered in the analysis of test samples

99

Validation ToolsValidation ToolsValidation ToolsValidation Tools

Fortified samples

◦ These are materials or solutions which have been fortified with the analyte(s) of interest.

◦ The fortification is usually made by spiking. These materials or solutions may already contain the analyte of interest

◦ Fortification with a known amount of analyte enables the increase in response to the analyte to be measured and calculated in terms of the amount added

Spiked samples

- Similar to the fortified sample but spiking does notnecessary have to be restricted to the analyte of interest

100

Validation ToolsValidation ToolsValidation ToolsValidation Tools

�ހ

02/03/55

51

Incurred samples◦ The materials in which the analyte of interest may be

essentially alien, but has been introduced to the bulk at

some point prior to the material being sampled

◦ The analyte is thus more closely bound in the matrix than it

would be by spiking

1. Herbicides in flour from cereal sprayed with herbicides

during its growth.

2. Growth promoters in meat derived from beast fed with

feeds containing the promoters

3. Active ingredients in pharmaceutical formulations added at

the formulation stage

101

Reference materials (RMs)◦ Any materials used as a basis for reference, and could

include laboratory reagents of known purity, industrial

chemicals, or other artifacts

◦ The property or analyte of interest needs to be stable and

homogenous

◦ The material does not need to have the high degree of

characterization, traceability and certification more

properly associated with certified reference materials

102

�ހ

02/03/55

52

103

3.1 reference material RM Material, sufficiently homogeneous and stable with respect to one or more specified properties, which has been established to be fit for its intended use in a measurementprocess.

NOTE 1 RM is a generic term

NOTE 2 Properties can be quantitative or qualitative, e.g., identity of substances or species.

NOTE 3 Uses may include the calibration of a measurement system, assessment of a measurement procedure, assigning values to other materials, and quality control.

NOTE 4 An RM can only be used for a single purpose in a given measurement ISO REMCO

VIM, 3rd Edition103

Certified reference materials (CRMs)

◦ The characterization of the parameter of interest in a certified reference material is generally more strictly controlled than for a reference material

◦ The characterized value is certified with a stated uncertainty by a recognized institution

◦ Characterization is normally done using several different methods, so that as far as possible,

◦ Any bias in the characterization is reduced or even eliminated

104

拸۵

02/03/55

53

105

3.2 certified reference material CRM reference material, characterized by a metrologically valid procedure for one or more specified properties, accompanied by a certificate that provides the value of the specified property, its associated uncertainty, and a statement of metrological traceability

NOTE 1 The concept of value includes qualitative attributes such as identity or sequence. Uncertainties for such

attributes may be expressed as probabilities.

NOTE 2 Metrologically valid procedures for the production and certification of reference materials are given in,

among others, ISO Guides 34 and 35.

NOTE 3 ISO Guide 31 gives guidance on the contents of certificates.

VIM, 3rd Edition

ISO REMCO

105

� RM � Homogeneous� Stable� Specified properties value

106

• CRM � Homogeneous

� Stable

� Specified property value

� Certificate (acc. ISO Guide 31)

+ Uncertainty + Traceability

���� Metrologically valid procedure

RM

106

拸۵

02/03/55

54

� Method development and validation– evaluation of trueness

– uncertainty estimation

� Calibration of instrument

� Proof of method performance– statistical quality control

– establishing traceable results

– equipment qualification

� Proficiency testing– training and verification of competence

107

Use of reference material in chemical analysis

107

� Eurachem Guide “The Fitness for Purpose of Analytical Methods” LGC, UK, December 1998

� NATA Technical Note 17 “Guidelines for the validation and verification of chemical test methods”, Australia, April 2009

� Train-the –trainer course “Analytical Method Validation”, National Measurement Institute, Australia, September, 2010

� The Association of Official Analytical Chemists (AOAC international), “Peer-Verified Methods Program”, November, 1993

108

�ހ

02/03/55

55

109

Thank YouThank YouThank YouThank You