1

Uncertainty budgetIn many situations we have uncertainties come from several sources.When the total uncertainty is too large, we look for ways of reducing it.The tolerable total uncertainty is our uncertainty budget, and we need to achieve it by reducing individual uncertainties in the most cost effective way.This will be illustrated by a study by Chanyoung Park on deciding between reducing uncertainties at the material level or at the structural level.1

1Chanyoung Park, Raphael T. Haftka, and Nam-Ho KimModeling the Effect of Structural Tests on Uncertainty in Estimated Failure Stress (Strength)2

Presentation based on Park, C., Matsumura, T., Haftka, R. T., Kim, N. H. and Acar.,E., Modeling the effect of structural tests on uncertainty in estimated failure stress 13th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Fort Worth, Texas, Sept. 13-15, 20102Multistage testing for design acceptance Building-block processDetect failures in early stage of designReduce uncertainty and estimate material propertiesA large number of tests in lower pyramid (reducing uncertainty)System-level probability of failure controlled in upper pyramid (certification)ELEMENTSDETAILSCOMPONENTSCOUPONSSYSTEMDATA BASESTRUCTURAL FEATURESGENERIC SPECIMENSNON-GENERIC SPECIMENS3

This presentations scope is first two stages coupon tests and element tests. The coupon tests establish the distribution of the material strength, and the element tests establish the accuracy of the failure theory used to predict element failure from material strength.

For the entire pyramid, the final goal is certifying the system by a certification test which demonstrates that the wing, for example, can carry loads that are high enough to keep the probability of failure in flight acceptably low.34Structural elements are under multi-axial stress and element strength has variability (aleatory uncertainty)Element strength is estimated from material strengths in different directions using failure theory, which is not perfectly accurate (epistemic uncertainty)Material coupon tests are done to characterize the aleatory uncertainty, but with finite number of tests we are left with errors in distributions (epistemic uncertainty)Element tests reduce the uncertainty in failure theory.If we can tolerate a certain total uncertainty we need to decide on number of coupon and element tests.

ELEMENTCOUPONUncertainty in element strength estimates

Our goal is to characterize well the distribution of element strength, typically the mean and standard deviation.

We have two main sources of error, which are both epistemic uncertainties. First, to find the distribution of material strength we perform coupon tests, and we cannot afford to have an infinite number of these, so we are left with sampling errors.

Second, we calculate the element strength from a failure theory that combines strengths in different directions, because the element has multi-axial loading, and stresses that vary from point to point.

The challenge addressed in this lecture is how to decide on the balance between the number of coupon tests and number of element tests.4Estimating mean and STD of material strengthGoal: Estimate distribution of material strength from nc samplesAssumption: true material strength: tc,true ~ N(mc,true, sc,true)Sample mean & STD: (mc,test, sc,test)Predicted mean & STD

5

tc,truemc,truemc,testtc,testtc,PDistribution of distributions!!tc,P ~ N(mc,P, sc,P)

When we estimate the distribution of material strength from nc samples, we expect to be a bit off. The red curve depicts the true distribution of material strength, while the purple curve depicts the one based on the measured mean and standard deviation.

Fortunately, we have calculated the predictive distribution of the mean and standard deviations of a normal distribution from nc samples. The mean follows a normal distribution with the same mean but with a standard deviation which is smaller by the square root of the number of samples. The standard deviation follows a chi distribution.

5Top Hat questionIn tests of 50 samples, the mean strength was 100 and the standard deviation of the strength was 7. What is the typical distance between the red and purple curves in the figure710.7

6tc,truemc,truemc,testtc,testtc,P6tc,P ~ N(mc,P, sc,P)Obtaining the predictive strength distribution by sampling? Predictive distribution of material strengthPredictive mean & STD

Samples of possible material strength distributions

Predictive true material strength distribution

How do we decide how many samples?7

SamplingSamplingmc,Psc,Ptc,Ptc,P

To obtain the predictive distribution (denoted by P) of the strength, we can use sampling. We sample a mean, then we sample a standard deviation, and then we use those to sample one strength. This is repeated until we have enough samples to get the predictive distribution of the strength.7Example (predictive material strength)Predictive distribution of material strength w.r.t. # of specimens c,P is biased but it compensates by wider distributionAre any of these results extreme compared to the expected scatter?# of samples3080c,test1.0531.113c,test0.0960.083Std. c,P0.0180.009Std. c,P0.0130.007Mean c,P1.0531.113Std. c,P0.0980.085c,true1.1c,true0.07780.60.70.80.911.11.21.31.41.51.6tc,Pnc = 30tc,Pnc = 80tc,truetc,P ~ N(mc,P, sc,P)

This is an example where the true mean is 1.1, and the standard deviation of material strength is 0.077 (7% coefficient of variation). The observed mean and standard deviations with 30 samples are somewhat extreme compared to the noise that we would expect. With 30 samples, the standard deviation of the mean with repeated samples is 0.018, so that that 1.053 is almost three standard deviations away. The standard deviation of the standard deviation is 0.013, so that 0.096 is a couple of standard deviation away.8Estimating element strengthAssumption: true element strength te,true ~ N(me,true, se,true)

Error in failure theory

Element tests used to reduce errors using Bayesian updating9Failure theoryte,true = ktruetc,trueCoupon strengthElement strengths2s1tc,truete,trueMultiaxialloadingFailure envelopektrueme,true = ktruemc,truese,true = ?me,P= (1 ek)kcalcmc,Pse,P = (1 es)sc,PErrors

To predict element strength we need to deal with the complication that the element is under multi-axial loading so that several stress components act simultaneously. This requires us to use some kind of failure theory in order to apply material strength to element failure.

For isotropic materials, we test strength only in one direction, and we can use a failure theory, such as maximum shear stress (Tresca) to predict failure under multi-axial stresses. For anisotropic materials, we test strength in several directions and use a more complex failure theory such as Tsai-Wu.

This is shown schematically in the figure for the case of two stress components. The strength in the 1 and 2 directions have been obtained from coupon tests, and a failure theory has been used to construct a failure envelope shown in red. The element is loaded in some intermediate direction as shown by the arrow.

We assume that element strength is normally distributed, that he standard deviation is not known, but that the mean is obtained from the failure theory which produces a correction k to the material mean strength. The failure theory is not perfectly accurate, so we have a calculated k, which is different from the true k. Similarly, the standard deviation is likely to be close to that of the material strength but there is some difference that we also denote as error.9Prior distribution (mean element strength)Uniform distribution for error in failure theory (10%)Used Bayesian network to calculate PDF of me,PtrueSimilar calculation for element std.10me,P = (1 ek)kcalcmc,Pncmc,testsc,testmc,Pekme,P

Before we take into account the element tests, we construct a prior distribution for the mean and standard deviation. This slides shows the process for the mean. We have the predictive distribution of the mean of the material strength mu_cP, and we have the distribution of error, which was assumed to be +-10% in the following calculations.This two can be combined by Mote Carlo sampling to yield the prior for the mean element strength. In the paper they were combined analytically using a Bayesian network and a convolution integral.

For the standard deviation we have exactly the same process.10Prior example Prior distribution (Joint PDF) for the mean and STD of element strength

ErrorDistributionBoundsError in Failure theoryUniform10%Error in estimated std.Uniform50%11200x200 gridme,Pse,Pmc,Psc,PJoint PDFme,Pse,P00.050.10.150.20.250.91.01.11.21.3Uncertainty increases due to error(epistemic uncertainty)

This is an example of assumed error distribution in our knowledge of the mean and standard deviation of element strength. The error in the failure theory is assumed to be relatively small +-10%, but the difference between the coupon and element standard deviation may be more substantial and we assume differences up to 50%.

The mean and standard deviations are independent so the joint distribution shown in the right figure is the product of the two individual distribution. The contours are based on a dense grid where we compute the joint pdf.11Bayesian updateElement test ~ N(me,true, se,true)Update the joint PDF (me,P, se,P) using ne element tests12me,Pme,truefM(m | test) = L(test | m)fM(m)Reduce epistemic uncertainty in ek

The process of using element tests for Bayesian updating is illustrated here. Each test result is used to update the distribution using Bayes rule. Unlike the previous slide this one shows the effect of Bayesian updating only on the mean.

In the illustration three test results are shown by the green, light blue and dark blue circles. The resulting updated distributions are also shown. After three tests the distribution of the mean is much narrower.

Note that this slides has animations.12Illustration of convergence of coupon mc,P & sc,P Estimated mean of (mc,P & sc,P) (single set cumulative)TestDistributionParametersCoupon testNormalmc,true = 1.1, sc,true = 0.077 True distribution of material strength

13

This is another example of the variation of the mean and standard deviation from a single set, which is different from the one shown on Slide 8. It intends to show that as we increase the number of coupon tests, the error does not reduce monotonically for a single set because of the randomness of the sampling process.13Illustrative example (coupon tests) Estimated STD of (mc,P & sc,P)

Increasing nc reduces uncertainties in the estimated parametersEffectiveness of reducing uncertainty is high at low nc

14

While the actual error does not reduce monotonically, as shown on the previous slide, the uncertainty estimates do converge well. 14Top Hat questionWhy does the convergence on Slide 14 (uncertainty estimates) look so much better than the convergence on Slide 13 (estimates of mean and standard deviations)Noise to signal ratioDifference in scalesBoth

1515Illustrative element updated distributionUpdated joint PDF of parameters

True distribution of element strength

16TestDistributionParametersElement testNormalme,true = 1.1, se,true = 0.099

se,trueML me,Pme,trueML se,Pme,Pse,P

This is an illustration how one element test changes the distribution of the predictive mean and predictive standard deviation.16Element updated distributions for 10 coupon tests

RMS error (500K instances of tests) vs. uncertainty in mean and standard deviation from a single set of tests

17STD of meanSTD of STD

These figures compare our estimate of the uncertainty in the mean and standard deviation of element strength from a single set of 10 coupon tests and a single set of element tests (in color) and what we get for the rms error of the estimate over 500,000 sets of tests results.17Element updated distributions 90 coupon tests18RMS error (500K MCS) vs. estimated uncertainty in means and standard deviation from a single set of tests.

STD of meanSTD of STD

For 90 coupon tests, the uncertainty in the mean hardly changes with number of element tests, because for our example the mean of the coupon strength is the same as the mean of the element strength (at 1.1). The standard deviations are different (0.077 vs. 0.099), so adding element tests narrows down the uncertainty and the errors.18Dependence of rms errors on number of testsFirst element test has a substantial effect to reduce uncertainty in estimated parameters of element strength19Mean of element strengthSTD of element strength

Figure illustrates the effect of diminishing returns. There is substantial reduction in errors going from 10 coupon tests to 50, but hardly any change going from 50 to 90. Similarly, there is substantial reduction in error with the first element test, but much less thereafter.19