Control de calidad en R Estándares de control posicionalcoello.ujaen.es/investigacion/.../2016_MECPosicional_IPGH_Quito_A… · Control de Calidad en R: Estándares de control posicional

Francisco Javier Ariza Ló[email protected]

Universidad de Jaén (España)

Control de calidad en REstándares de control posicional

Quito, Noviembre de 2016

Francisco Javier Ariza‐López [email protected] / Universidad de Jaén (España) Proyecto: Evaluación de la calidad de la IG en América Latina

Control de Calidad en R:Estándares de control posicional

Índice & IntroPrerrequisitosVisión generalAlgunos MECP

Índice

• Introduction• Prerequisites• General View• Some PAAMs

iii mty yye iii mtx xxe

iii mtz zze



Prereq Check

Good analysis

Cartographic prerequisites Positional interoperability of data

Control work prerequisites Quality of the source of reference

Statistical prerequisites Confidence on the statistical procedures


Prerrequisitos



Positional interoperability between the

assesses product and the reference dataset.

Mathematical Cartography:

Ellipsoid, datum and projection

This is called CRS (Coordinate Reference

System)

CRS = Datum + Coordinate System

CRS = a CS related to the Earth by a Datum

CS = set of mathematical rules for specifying

how coordinates are to be assigned to points

Cartographic prerequisites


Prerrequisitos



Independence of the assessment work no correlation, no same bias.

Accuracy The accuracy of the assessment work (at least × 2‐ × 3, better ×5) the

assessment work does not affect calculations.

There is no agreement (× 2, × 3, ×5, × 10). More usual case × 3. (at least)

There are methods that consider "safety factors“.

Control work prerequisites

Product QC Method Estimation Introduced Error1 1,000 1,414 41,42%1 0,800 1,281 28,06%1 0,600 1,166 16,62%1 0,500 1,118 11,80%1 0,400 1,077 7,70%1 0,333 1,054 5,41%1 0,250 1,031 3,08%1 0,200 1,020 1,98%1 0,167 1,014 1,38%1 0,143 1,010 1,02%1 0,125 1,008 0,78%1 0,111 1,006 0,62%1 0,100 1,005 0,50%1 0,001 1,000 0,00%

0,800

0,900

1,000

1,100

1,200

1,300

1,400

0,000 0,200 0,400 0,600 0,800 1,000


Prerrequisitos



Prerequisites (measuring method accuracy 3x, etc.).

Bias = 1/3 σ Bias = 1/3 σPopulation

Sampling Sampling

% of the sample within [‐1σ, +1 σ] = 97,72%

% of the sample within [‐1σ, +1 σ] = 65.46%


PrerrequisitosControl work prerequisites




Visión general de los MECP



Perspectives

How the product is?

How the production processes are?

How I can improve it?

How I can be more efficient?

How I can reduce liability of my products?

Is this product suitable for my application?

Is this product good enough for me?

Is this product integrable with my other datasets?

User’s point of view. Producer’s point of view.

Objectives• Help to the users and producers to take better decisions.• Standardize the positional accuracy assessment method.• Transparency in market relations and supplies.





Perspectives

User’s point of view. Producer’s point of view.When they want assess the quality, they can be interested in:

• To know, as good as possible, the exact level of

uncertainty to assure the quality or, if the quality

is not adequate, in order to improve the

production process and product characteristics.

• Estimation

• To decide if the product satisfies the

requirements. In this case we do not mind

the true quality parameter, but only if it is

adequate or not for our purposes.

• Hypothesis test (control)

μ, σ, ξ, αValor medio, desviación, precisión, significación, tamaño muestra

Tol, α, βTolerancia, significación, potencia (riesgos tipo I y II),





• Planimetry and altimetry independently, or altimetry adjustableby planimetry.

• Point-based methods (points, well-defined, well-distributed)• Reduced sample (sample size: “at least 20”)• Reference source of higher accuracy (accuracy “at least x3”)• Majority of PAAM are based on the Normality of errors• Weak definition of statistical prerequisites and absence of

specifications on the control of statistical prerequisites.• Report Scarce information about the entirely process.

Main common facts





Points:

• Well-defined points: A well‐defined point represents a feature for which thehorizontal position is known to a high degree of accuracy and position with respectto the geodetic datum. For the purpose of accuracy testing, well‐defined pointsmust be easily visible or recoverable on the ground, on the independent source ofhigher accuracy, and on the product itself. The selected points will differ dependingon the type of dataset and output scale of the dataset. For graphic maps andvector data, suitable well‐defined points represent right‐angle intersections ofroads, railroads, or other linear mapped features, such as canals, ditches, trails,fence lines, and pipelines. For orthoimagery, suitable well‐defined points mayrepresent features such as small isolated shrubs or bushes, in addition to right‐angle intersections of linear features. For map products at scales of 1:5,000 orlarger, such as engineering plats or property maps, suitable well‐defined pointsmay represent additional features such as utility access covers and intersections ofsidewalks, curbs, or gutters.


Visión general de los MECPControl elements



Points:

• Well-defined points:

• Well-distributed points


Visión general de los MECPControl elements



Sample size:• “at least 20” is a repeated sentence.

• In general the sample size is not related with the size of thepopulation of controlled elements or area.

• The value 20 is related with the central limit theorem in statistics.

• The value 20 is related to the approximation to normal distribution.

• This value may be adequate for hypothesis testing but not forparameters estimation.

• No guidance for substitution.

• The more control points the more $ cost


Visión general de los MECPSample



Reference source:

• of higher accuracy (“at least x3”)

• Usually, the accuracy of the reference source is not taken intoaccount in the computations, it is only a condition.


Visión general de los MECPSource



Weak definition of statistical prerequisites and absence ofspecifications on the control of statistical prerequisites.

Some common prerequisites (hypothesis) are:

randomness

absence of outliers

independency between error components

variational behavior is similar for X and Y (σx≈ σy)

Normality of error data

PAAMs do not provide guidance for they control.In general, nobody test these hypotheses!!!


Visión general de los MECPStatistical prerequisites



Limited reporting about the entirely processReport centered on results, no information about the process

PAAMs propose a “sentence” to inform about the result but the don't inform

about many important issues (prerequisites, quality of the reference,

management of bias, management of outliers, etc.). No graphic results are

indicated by the PAAMs.

ASPRS


Visión general de los MECPReport



• Weak definition of statistical prerequisites and absence ofspecifications on the control of statistical prerequisites.

• Weak definition of conditions (e.g. RMSEx=RMSEy in theNSSDA).

• Significance level: Simultaneously application of several statisticaltest without considering Bonferroni, or other similar techniques.

• Scarce information about the entirely process.• Based on the normality of errors.• They consider different probabilities for limiting errors (90%,

95%...)• They consider different graphical thresholds for limiting errors

(0.2mm, 0.25mm, 0.30mm...)• Majority of them are not adequate for continuous supplies.

Summary of problems





Common problems of people applying the standards

• Combination of radial (horizontal) accuracies with linear accuracies.• Confusion between planimetic (radial, 2D) values) and X/Y values (1D).• Apply RMSE instead of SIGMA.• Apply normal‐based expansion factors to the RMSE (when μ≠0).• Apply normal‐based expansion factors to non‐normal/Chi2 data.• Apply 1D expansion factors to the 2D case.• Apply 2D expansion factors to the 1D case.• Elimination of outliers from the report.• Confusion when considering the tail of a distribution function (right or left?).• Poor report.


Visión general de los MECPSummary of problems



There are a lot of PAAMs:

• EMAS (Engineering Map Accuracy Standard) (1983), By The American Society of Civil Engineers• ASPRS‐ASLSM (ASPRS Accuracy Standards for large‐scale Maps) (1989), By the American Society of

Photogrammetry and Remote Sensing.• AS4P (Accuracy Standards for positioning V 1.0) (1996), By Geomatics Canada.• NSSDA (National Standard for Spatial Data Accuracy) (1998), By The Federal Geographic Data

Committee. • STANAG 2215 (Evaluation of maps, aeronatical charts and digital topographic data) (2002), By NATO.• CPATT (France 2003 Arrêté du 16 septembre 2003 portant sur les classes de précision applicables aux

catégories de travaux topographiques réalisés par l'Etat, les collectivités locales et leursétablissements publics ou exécutés pour leur compte) (2003).

• MVMASG (Model Virginia Map Accuracy Standards Guideline)(2009), By Virginia Information Technologies Agency.

• AMSDHAS (Australian Map and Spatial Data Horizontal Accuracy Standard) (2009), By ICSM.• Kontroll av Geodata (2013), By Norska standarden.• ASPRS‐PAS4DGD (ASPRS Positional Accuracy Standards for Digital Geospatial Data) (2015), By the

American Society of Photogrammetry and Remote Sensing.


Algunos MECP



• NMAS (National Map Accuracy Standard) (1947), by the United States Bureau of the Budget.• NIST‐ASM4BP (An Acceptance Sampling Method for 2D/3D Building Plans) (2009), by National

Institute of Standards and Technology.• ISO 2859‐1. Sampling procedures for inspection by attributes — Part 1: Sampling schemes

indexed by acceptance quality limit (AQL) for lot‐by‐lot inspection (1999), by ISO.• IPGH Especificaciones topograficas (1978), By the Instituto Panamericano de Geografía e

Historia.• Brasil (1984). Normas Técnicas da Cartografia Nacional, Decreto nº 89.817, de 20 de junho de

1984. • ISO 3951‐1 Sampling procedures for inspection by variables ‐‐ Part 1: Specification for single

sampling plans indexed by acceptance quality limit (AQL) for lot‐by‐lot inspection for a single quality characteristic and a single AQL, (2005), By ISO.

• UNE 148002 Metodología de evaluación de la exactitud posicional de la información geográfica (2016), by AENOR.

• Positional control by two tolerances (2016), by Ariza‐López & Rodríguez‐Avi, 2016.…and many others…¡¡¡

Few are Pass/fail methodsMajority of them are not adequate for continuous supplies

Many of them are measured‐based methods


Algunos MECPThere are a lot of PAAMs:



NMAS: H & V. Very simple calculations. Ease of understanding (pass / fail). Values on the "paper“ and fixed tolerances. Statistics unexplained. {n≥20, ×?}

EMAS: X, Y, Z. Complex calculations. 4 hypothesis test for planimetry + 2 hypothesis test for altimetry. Values on the "ground". The error limits can be user‐defined. It allows to know the process and its problems (bias, precision). Significance levels with problems. It assumes that bias has been eliminated and that errors are normally distributed but no control process is proposed. {n≥?, ×3}

ASPRS‐1990: H & V. Very simple calculations. Ease of understanding. Values on the “ground” and fixed tolerances related with scales. Proposes 3 classes of accuracy. Assumes that blunders have been corrected previously but no control process is proposed. {n≥20, ×?}

NSSDA: H & V. Simple calculations. Values on the “ground”. No classes of accuracy or tolerances. The user must interpret the result. It assumes that bias has been eliminated, errors are normally distributed and independent but no control process is proposed. {n≥20, ×?}


Algunos MECPSome details

Documents

Control de calidad en R Estándares de control posicionalcoello.ujaen.es/investigacion/.../2016_MECPosicional_IPGH_Quito_A… · Control de Calidad en R: Estándares de control posicional