Slide 1

Concept of Uncertainty - Resources and NRCs report Yuejian Zhu Ensemble Team Leader EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 30, 2015, Fuzhou, Fujian, China

Slide 2

Concept of Uncertainty Resources and NRCs Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRCs report fundamental education material

Slide 3

X-Z PlaneX-Y Plane Power Spectrum Lorenz Equation ( Chaos in ODE) The Lorenz model is defined by three differential equations given the time evolution of variables X(t), Y(t), Z(t) http://crossgroup.caltech.edu/Chaos_Course/Lesson1/Demos.html

Slide 4

Errors ?

Slide 5

Ensemble Forecast System Describe Forecast Uncertainty Arising Due To Chaos Buizza 2002

Slide 6

Concept of Uncertainty Resources and NRCs Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRCs report fundamental education material

Slide 7

Forecast Uncertainty Forecast Errors Uncertainty Instrument Observation Assimilation Analysis Computation Parameterization

Slide 8

Due to model imperfection One day advantage Earlier Ensemble Forecast System

Slide 9

Concept of Uncertainty Resources and NRCs Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRCs report fundamental education material

Slide 10

Ensemble Forecasts Deterministic forecast Initial uncertainty Verified analysis Forecast probability

Slide 11

Real example for bimodality

Slide 12

12 2-m temp 10/90 probability forecast verification Northern Hem, period of Dec. 2007 Feb. 2008 3-month verifications Top: 2-m temperature probabilistic forecast (10% and 90%) verification red: perfect, blue: raw, green: NAEFS Left: example of probabilistic forecasts (meteogram) for Washington DC, every 6-hr out to 16 days from 2008042300 90% 10%

Slide 13

Concept of Uncertainty Resources and NRCs Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRCs report fundamental education material

Slide 14

Uncertainties & disagreements Ensemble forecast is widely used in daily weather forecast

Slide 15

Map of PQPF and Precipitation Types: every 6 hours, 4 different thresholds

Slide 16

16 Example of forecast precipitation (next 24 hours) (assume all forecasts are bias free after bias correction or calibration) MM/24hrs 25 0 50 75 100 Fcst 1Fcst 2 Fcst 3 50 deterministic 80% mean Which forecast is relatively easier? Or more confident! Can we identify all these information? probabilistic mean

Slide 17

17 Resolution makes difference for Typhoon Morakot Ini: 2009080600 T126 ensemble T190 ensemble Ini: 2009080700 T126 ensemble T190 ensemble Most models do not make right forecasts Is this a good example for uncertainty forecast???

Slide 18

Typhoon Megi 2010101512 2010101612 20101017122010101812 1.NCEP 2.CMC 3.ECMWF 4.3-ENS Is this a good example to discuss forecast consistency and accuracy?

Slide 19

00UTC 20121022 (8 days) Opr: T254L42 (55km) Opr: T254L42 (55km) Para: T574L64 (33km) Para: T574L64 (33km) 06UTC Thick blue: ensemble mean Bimodality? Red arrow means good forecast

Slide 20

Concept of Uncertainty Resources and NRCs Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRCs report fundamental education material

Slide 21

Completing the Forecast Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts US NRC report (174 pages) The National Academic Press 2006 Coming soon in Chinese

Slide 22

Completing the Forecast Board on Atmospheric Sciences and Climate Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts

Slide 23

Task Provide guidance on how to identify and characterize needs for uncertainty information among various users of forecasts. Support this with examples of needs. Identify limitations in current methods for estimating and validating forecast uncertainty, relating these limitations to users needs and recommend improvements or new methods and approaches. Identify sources of misunderstanding in communicating forecast uncertainty, including vulnerabilities dependent on the means of communication used, and recommend improvements in the ways used to communicate forecast uncertainty. Recognizing the breadth and depth of this task, NWS advised the committee at its opening meeting to "teach us how to fish as opposed to giving us a fish."

Slide 24

Report Contents Summary Introduction Uncertainty in Decision Making Estimating and Validating Uncertainty Communicating Forecast Uncertainty Overarching Recommendations

Slide 25

Recommendations The entire Enterprise should take responsibility for providing products that effectively communicate forecast uncertainty information. NWS should take a leadership role in this effort. NWS should improve its product development process by collaborating with users and partners in the Enterprise from the outset and engaging and using social and behavioral science expertise. All sectors and professional organizations of the Enterprise should cooperate in educational initiatives that will improve communication and use of uncertainty information. NWS should develop and maintain the ability to produce objective uncertainty information from the global to the regional scale.

Slide 26

Recommendations To ensure widespread use of uncertainty information, NWS should make all raw and post-processed probabilistic products easily accessible to the enterprise at full spatial and temporal resolution. Sufficient computer and communications resources should be acquired to ensure effective access by external users and NWS personnel. NWS should expand verification of its uncertainty products and make this information easily available to all users in near real time. A variety of verification measures and approaches (measuring multiple aspects of forecast quality that are relevant for users) should be used to appropriately represent the complexity and dimensionality of the verification problem. Verification statistics should be computed for meaningful subsets of the forecasts (e.g., by season, region) and should be presented in formats that are understandable by forecast users. Archival verification information on probabilistic forecasts, including model-generated and objectively generated forecasts and verifying observations, should be accessible so users can produce their own evaluation of the forecasts.

Slide 27

Recommendations To enhance development of new methods in estimation, communication, and use of forecast uncertainty information throughout the Enterprise, and to foster and maintain collaboration, confidence, and goodwill with Enterprise partners, NWS should more effectively use testbeds by involving all sectors of the Enterprise. The committee endorses the recommendation by the NRC Fair Weather report to establish an independent advisory committee and encourages NOAA to bring its evaluation of the recommendation to a speedy and positive conclusion. NWS should dedicate executive attention to coordinating the estimation and communication of uncertainty information within NWS and with Enterprise partners.

Slide 28

Key points not to forget Broad applicability within NOAA and other agencies This will take time (turn the aircraft carrier) Lets get started now NWS evolution - time is right to incorporate uncertainty information into process Uncertainty is another component of improving the forecast its not detached from the primary mission of forecast improvement Uncertainty is not an add-on it is in the nature of the product