Major Ernest Y. Wong Department of Systems Engineering United States Military Academy

1 of 22

Introducing @RISK to Undergraduate Cadets Attending West Point:

Investing and Gambling for Active Learning

Major Ernest Y. WongMajor Ernest Y. WongDepartment of Systems EngineeringDepartment of Systems Engineering

United States Military AcademyUnited States Military Academy

2 of 22

• Goals of SE350, Systems Modeling and Design• Active Learning through Simulations• Kolb’s Experiential Learning Model (investment case study)

• Experience• Observe• Generalize• Test

• Promoting Bloom’s Taxonomy of Cognitive Learning• Challenges in Teaching Simulation • Student Feedback• Conclusions

Agenda

3 of 22

Methods CourseMethods Course

SE350SE350

Design Course

SE450

Introductory Course

SE300

Core Engineering Sequence Learning Model Overview Crawl Walk Run

Goals of SE350, Systems Modeling and Design

4 of 22


SE350SE350

Design Course

SE450

Introductory Course

SE300

Core Engineering Sequence Learning Model Overview Crawl

• Introduces non-Engineering majors to a systematic problem solving framework• Acquaints undergraduate students to engineering concepts and terminology

--Stakeholder Analysis--Problem Definition--Value Hierarchy--Alternative Generation--Cost Benefit Analysis--Pareto Principle--Functional Decomposition--Assessment & Control

Walk Run


5 of 22


SE350SE350

Design Course

SE450

Introductory Course

SE300


• Builds upon the mathematics and basic science concepts learned in the undergraduate core curriculum• Introduces non-Engineering majors to various quantitative

methods• Focuses on the application of economic, deterministic, and stochastic

models


6 of 22


SE350SE350

Design Course

SE450

Introductory Course

SE300



• Develops student teams capable of helping satisfy client needs and proposing solutions to actual problems

--West Point Cemetery --Army/Navy Game Site--Cadet Summer Training--Cadet Ethics Training--Post 9/11 Traffic Flow--Army UAV Cmd & Cntl--Soldier Pre-Deployment Tng

7 of 22


SE350SE350

Design Course

SE450

Introductory Course

SE300


• Builds upon the mathematics and basic science concepts learned in the undergraduate core curriculum• Introduces non-Engineering majors to various quantitative

methods• Focuses on the application of economic, deterministic, and stochastic

models--Decision Analysis (Risk and Uncertainty)--Engineering Economy (Time Value of Money)--Optimization Techniques--Forecasting--Spreadsheet Modeling--Monte Carlo Simulation


8 of 22

David Kolb’s Experiential Learning Model

9 of 22

Student Investment Ideas (1. Experience)

1. ING Savings Account 2. Edward Jones INDYMAC CD 3. USAA 104-Month CD

4. Fidelity Ginnie Mae Fund 5. Oppenheimer Int’l Bond 6. Dodge & Cox Balanced 7. Vanguard Target Retirement 2045 8. Franklin Templeton Founding 9. Fairholme Fund10. USAA Cornerstone Strategy11. Aegis Value12. Vanguard Wellington13. USAA S&P 50014. Vanguard Energy Admiral15. Vanguard Healthcare 16. Prudent Bear17. Cohen & Stears Realty18. USAA Extended Market

19. Wal-Mart20. Pepsico21. Starbucks Coffee22. Advanced Micro Devices23. Miami, Florida Real Estate24. Poker

“RISKLESS” ASSETS

“MODERATELY RISKY” ASSETS

“RISKY” ASSETS

ConstantConstantConstant

UniformNormalNormalNormalNormalNormalNormalNormalNormalNormalNormalNormalNormalNormalNormal

NormalNormalNormalNormal

Triangular

Triangular

BEST FITTING DISTRIBUTION Mean = 3.00%

Mean = 3.60% Mean = 5.20%

Min = -0.26% Max = 12.27% Mean = 2.31% Stdv = 3.00% Mean = 10.68% Stdv = 4.00% Mean = 4.65% Stdv = 5.00% Mean = 5.90% Stdv = 6.00% Mean = 11.08% Stdv = 10.00% Mean = 4.26% Stdv = 10.00% Mean = 18.02% Stdv = 11.00% Mean = 5.62% Stdv = 12.00% Mean = 9.60% Stdv = 14.35% Mean = 23.07% Stdv = 15.25% Mean = 15.76% Stdv = 16.00% Mean = 9.10% Stdv = 18.00% Mean = 18.00% Stdv = 19.00% Mean = 5.24% Stdv = 21.00%

Mean = 19.70% Stdv = 33.00% Mean = 11.20% Stdv = 45.00% Mean = 22.60% Stdv = 84.61% Mean = 42.60% Stdv =209.00%Min = -50.00% Mean = 20.00% Max = 150.00%Min = -100.00% Mean = 50.00% Max = 300.00%

PARAMETERS

10 of 22

Focusing on Starbucks Coffee (2. Observe)

Normal Distribution Mean = 22.60% Stdv = 84.61%

Normal(22.613, 84.610)

Valu

es x

10^-3

0

1

2

3

4

5

6

7

8

9

-25

0

-20

0

-15

0

-10

0

-50 0 50

10

0

15

0

20

0

25

0

< >39.5% 5.0%55.5%0.0 161.8

@RISK Student VersionFor Academic Use Only

Although we can expect to earn a 22.60% annual return on SBUX, what is the probability that we lose money on the stock?

Would you invest $24,000 of your own money in SBUX?

11 of 22

Simulate a Portfolio of All 24 Ideas (2. Observe) “RISKLESS” ASSETS


“RISKY” ASSETS




0000

-0.3 -0.05 0.2 0.45 0.7

7.29% 87.71% 5% 0 .3751

Mean=0.1753421 Mean=0.1753421

Distribution for Return on $1000 in All 24Assets/J33

0.000

0.500

1.000

1.500

2.000

2.500

3.000

3.500

Mean=0.1753421

-0.3 -0.05 0.2 0.45 0.7


00

-0.3 -0.05 0.2 0.45 0.7

7.29% 87.71% 5% 0 .3751

Mean=0.1753421


0.000

0.200

0.400

0.600

0.800

1.000

Mean=0.1753421

-0.3 -0.05 0.2 0.45 0.7


Wouldn’t you rather invest $1000 into each asset and accept an expected annual gain of 17.53% (vs. 22.6%) with just a 7.29% chance of losing money (vs. 39.5%)!?!

12 of 22

The Central Limit Theorem (3. Generalize) The Central Limit Theorem tells us that if enough The Central Limit Theorem tells us that if enough independent samples of almost any distribution are independent samples of almost any distribution are averaged together, the resulting distribution is normal.averaged together, the resulting distribution is normal.

Uniform(-0.26667, 12.267)

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

0.20

1 2 3 4 5 6 7 8 9 10

11

< >90.0%0.36 11.64


Normal(22.613, 84.610)

V

alu

es x

10

^-3

0

1

2

3

4

5

6

7

8

9

-25

0

-20

0

-15

0

-10

0

-50 0 50

10

0

15

0

20

0

25

0

< >39.5% 5.0%55.5%0.0 161.8


Triang(-92.026, 51.655, 302.43)

Valu

es x

10^-3

0

1

2

3

4

5

6

-10

0

-50 0 50

10

0

15

0

20

0

25

0

30

0

>14.9% 5.0%80.1%0.0 232.1

@RISK Student VersionFor Academic Use Only+ +

0000

-0.3 -0.05 0.2 0.45 0.7

7.29% 87.71% 5% 0 .3751

Mean=0.1753421 Mean=0.1753421


0.000

0.500

1.000

1.500

2.000

2.500

3.000

3.500

Mean=0.1753421

-0.3 -0.05 0.2 0.45 0.7


13 of 22

Student Investment Ideas (4. Test)




“RISKLESS” ASSETS


“RISKY” ASSETS

ConstantConstantConstant

UniformNormalNormalNormalNormalNormalNormalNormalNormalNormalNormalNormalNormalNormalNormal

NormalNormalNormalNormal

Triangular

Triangular

BEST FITTING DISTRIBUTION Mean = 3.00%

Mean = 3.60% Mean = 5.20%

Min = -0.26% Max = 12.27% Mean = 2.31% Stdv = 3.00% Mean = 10.68% Stdv = 4.00% Mean = 4.65% Stdv = 5.00% Mean = 5.90% Stdv = 6.00% Mean = 11.08% Stdv = 10.00% Mean = 4.26% Stdv = 10.00% Mean = 18.02% Stdv = 11.00% Mean = 5.62% Stdv = 12.00% Mean = 9.60% Stdv = 14.35% Mean = 23.07% Stdv = 15.25% Mean = 15.76% Stdv = 16.00% Mean = 9.10% Stdv = 18.00% Mean = 18.00% Stdv = 19.00% Mean = 5.24% Stdv = 21.00%

Mean = 19.70% Stdv = 33.00% Mean = 11.20% Stdv = 45.00% Mean = 22.60% Stdv = 84.61% Mean = 42.60% Stdv =209.00%Min = -50.00% Mean = 20.00% Max = 150.00%Min = -100.00% Mean = 50.00% Max = 300.00%

PARAMETERS

14 of 22

Re-Examining Modeling Assumptions (4. Test)Normal(42.607, 209.06)

Va

lues x

10^-3

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

-50

0

-40

0

-30

0

-20

0

-10

0

0

10

0

20

0

30

0

40

0

50

0

60

0

< >41.9% 5.0%53.1%0 386


Advanced Micro Devices (AMD)Normal Dist. Mean = 42.60% Stdv =209.00%

Has about a 42% chance of losing money.

Distribution for 12 $2000 Investments intoAMD/E15

0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

Mean=0.4259027

-1.5 -0.375 0.75 1.875 3


0000

-1.5 -0.375 0.75 1.875 3

27.28% 67.72% 5% 0 1.434

Mean=0.4259027 Mean=0.4259027

Yet when we make 12 separate purchases into AMD, does it make sense that the chance of

losing money falls to 27%?

Distribution for 24 $1000 Investments intoAMD/I24

0.000

0.1000.200

0.3000.4000.5000.600

0.700

0.8000.900

1.000

Mean=0.4256299

-1.5 -0.625 0.25 1.125 2


0000

-1.5 -0.625 0.25 1.125 2 17.77% 77.23% 5%

0 1.2098

Mean=0.4256299 Mean=0.4256299

We are still investing $24,000 but chance of losing money now drops to 18%.

The Central Limit Theorem states that if enough The Central Limit Theorem states that if enough independentindependent samples of almost any distribution samples of almost any distribution are averaged together, the resulting distribution are averaged together, the resulting distribution

is normal.is normal.

xxxx

15 of 22

Bloom’s Taxonomy on Cognitive Learning

Level 1 Goals

Level 2 Goals

16 of 22

1. Experience: Student investment ideas (ownership of familiar concepts)

Approach in SE350--Systems Modeling & Design

2. Observe: Understanding histograms (application of familiar concepts)

3. Generalize: Investment diversification (progression to new concepts)

4. Test: Modeling assumptions (understanding of modeling limitations, risks, and tradeoffs)

17 of 22

Students Insights into Modeling with @RISK

• Is there such a thing as “riskless” investments?

• What data should be used to try to determine a best fitting distribution?

• Which idealized distributions are indeed best from BestFit?

• What about modeling distributions with infinite tails? How realistic is this?

18 of 22

• Did not feel that they were equipped with adequate knowledge to interpret the simulation results (KnowledgeRecall)

• Did not know what actions to take to improve system performance (UnderstandingGrasp)

• Focused mainly on the mechanics of building the simulation model and believed the problem was solved once they ran the simulation (ApplicationApply)

• Found it difficult to go beyond just providing a single “optimal” solution (AnalysisAnalyze)

• Expressed unease with having to deal with uncertainties and coming up with open-ended recommendations (Synthesis & EvaluationSynthesize & Judge)

Challenges in Advancing Up Bloom’s Steps

However, some students complained that they:

19 of 22

• “I see a lot of potential for Excel.”• “I thought the projects were very applicable.”• “I liked learning how to use the simulation models.”• “I really liked the systems modeling and design portion of

the course—it was straight-forward and applicable.”• “I liked the projects; they gave me a chance to actually

figure out which course of action to take instead of me

knowing exactly which decision making process to use.”• “I wish I had more of these projects.”• “I wish I had majored in Systems Engineering instead of

xxxxxxxx.”

Positive Student Feedback

I hear, I forget. I see, I remember. I do, I understand. --Chinese ProverbI hear, I forget. I see, I remember. I do, I understand. --Chinese Proverb

20 of 22

Course - SE350 (Spring 2005) Answers: [5] Strongly Agree [4] Agree [3] Neutral [2] Disagree [1] Strongly Disagree

Answer [5]

Answer [4]

Answer [3]

Answer [2]

Answer [1] (no rsp)

A1. This instructor encouraged students to be responsible for their own learning. 42 (42%)

48 (48%)

9 (9%)

0 (0%)

0 (0%)

0 (0%)

A2. This instructor used effective techniques for learning, both in class and for out-of-class assignments.

37 (37%)

48 (48%)

12 (12%)

2 (2%)

0 (0%)

0 (0%)

A3. My instructor cared about my learning in this course. 43 (43%)

49 (49%)

6 (6%)

1 (1%)

0 (0%)

0 (0%)

A4. My instructor demonstrated respect for cadets as individuals. 56 (57%)

37 (37%)

5 (5%)

1 (1%)

0 (0%)

0 (0%)

A5. My fellow students contributed to my learning in this course. 36 (36%)

42 (42%)

14 (14%)

5 (5%)

2 (2%)

0 (0%)

A6. My motivation to learn and to continue learning has increased because of this course.

29 (29%)

41 (41%)

17 (17%)

9 (9%)

3 (3%)

0 (0%)

B1. This instructor stimulated my thinking. 35 (35%)

49 (49%)

12 (12%)

3 (3%)

0 (0%)

0 (0%)

B2. In this course, my critical thinking ability increased. 33 (33%)

44 (44%)

16 (16%)

5 (5%)

1 (1%)

0 (0%)

B3. The homework assignments, papers, and projects in this course could be completed within the USMA time guideline of two hours preparation for each class attendance.

32 (32%)

54 (55%)

10 (10%)

3 (3%)

0 (0%)

0 (0%)

C1. This course helped me learn to use the engineering design process to design, manage or reengineer systems or processes.

32 (32%)

45 (45%)

16 (16%)

4 (4%)

2 (2%)

0 (0%)

C2. This course taught me to communicate effectively both orally and in writing. 32 (32%)

29 (29%)

30 (30%)

8 (8%)

0 (0%)

0 (0%)

C3. This course improved my ability to solve real-world problems through quantitative techniques.

28 (28%)

53 (54%)

13 (13%)

4 (4%)

1 (1%)

0 (0%)

C4. This course provided me with practical, problem-solving experiences applicable to my future as an Army officer.

34 (34%)

44 (44%)

14 (14%)

5 (5%)

2 (2%)

0 (0%)

C5. Course exercises and designs improved my ability to model, analyze, or prototype real-world problems or systems.

30 (30%)

54 (55%)

11 (11%)

3 (3%)

1 (1%)

0 (0%)

Course Feedback (n=123)

21 of 22

Goal of Systems Engineering at USMA

“We are preparing graduates who are scientifically literate and capable of applying mathematical, engineering, and computational modes of thought to the solution of complex problems.”

--Dean, United States Military Academy

22 of 22

Questions?

[email protected]

Documents

Major Ernest Y. Wong Department of Systems Engineering United States Military Academy