1

DSTA-MIT/ESD: Armament Safety Project22 January 2007

• MIT Team– Prof. Stuart Madnick (smadnick@mit.edu) – Dr. Michael Siegel– Lynn Wu– Nathan Minami (MIT & US Army)– Allen Moulton– Mihai Lupu (SMA)– Assistance from:

• Prof. Aykut Firat (MIT & Northeastern University)• Prof. Hongwei Zhu (MIT & Old Dominion University)

© MIT, 2007

2

Safety Forecasting

Model (2)

Raw Knowledge Sources Knowledge

Repository

High Level

Safety Indicators

Detailed Indicators

Ammunition Age

Funding

Training

Training Schedule Pressure

=

Data Extraction & Semantic Reconciliation Technologies (1)

Internal sources: Assessment Reports, live firing trials data, incident investigation reports, acceptance test data, qualification trials reports, hazard database for various items, WDA, defect rates, range & training schedule, weapon & ammo procurement and ages, etc.

DSTA-MIT/ESD: Armament SafetyHigh-Level Project Overview

SEM 2007-01-13

Policy choicesand Interventions

External sources

Knowledge Re-Use

3

Diverse Raw Knowledge

SourcesKnowledge Repository

Internal sources

Knowledge Management Activities [Phase 1 and 2]

External sources

Knowledge Capture[Phase 1]

Knowledge Re-use

Knowledge Storage

(a) Raw Knowledge Retrieval (e.g., specific piece of knowledge) [Phase 1]

(b) Processed Knowledge Retrieval (e.g., comparisons and/or aggregations of knowledge) [Phase 1]

(c) Projections based on Knowledge (e.g., Forecast and/or Consequences) [Phase 2]

4

Example of Basic Knowledge Capture and Re-Use

• Goal: Demonstrate use of MIT extraction and analysis technologies

• Constraint: Difficult to get access to diverse data related to Singapore Armament Safety (so far)

• Approach: Demonstrate using public sources and interesting strategy questions

• Example issue: What is Singapore military expenditure per capita?

• How do others compare (+ or - %) with Singapore:– Israel = – USA = – UK = – Malaysia =

5

Good News – Data Publicly Available(https://www.cia.gov/cia/publications/factbook/geos/sn.html)

Population

Economy

Military

6

But it is scattered … must be located …

• … and not in exactly easy to use form …• Must do calculations:

– Military expenditures = 4.9% (of GDP) x $126.5 billion = $6.2 billion

– per capita = $6.2 billion / 4,492,150 people = $1379.85 per capita• Need to be careful about scale factors

– (one of our team members was off by a thousand)• Amount is in US$ … what if you want in SG$?

7

Easier way: Cameleon Web Extraction Tool( http://coin.mit.edu/DSTA/Mihai/cameleon/cameleon.html coin\smadnick 6671coin )

  <?xml version="1.0" encoding="ISO-8859-1" ?> - <DOCUMENT>- <ELEMENT>  <country>Singapore</country>   <population>4,492,150</population>   <gdp>126.5</gdp>   <milexpendpercent>4.9%</milexpendpercent>   </ELEMENT>  </DOCUMENT>

Results as tsv(data record)

Results as XML file

8

But also want: (1) multiple countries and (2) military per capita calculation

• First: Multiple countries.– Combine Cameleon web extraction capability with IBM

DB2 Federated Database Server• Technical details explained later

SQL Query:

Select country, population, GDP, MilExpendPercent from cia where country in ("Singapore" , “Israel”, “United States” ,“United Kingdom” ,“Malaysia”)

COUNTRY POPULATION GDP MILEXPENDPERCENTSingapore 4492150 126.5 4.90%Israel 6352117 156.9 7.70%United States 298444215 12.31 4.06%United Kingdom 60609153 1.81 2.40%Malaysia 24385858 287 2.03%

9

Now need military per capita calculation

• DB2 can provide the computation based on the data being extracted by Cameleon

Select country, MilExpendPercent * GDP * 1000000000 / population from cia

where country in (Singapore" , “Israel” ,“United States”, “United Kingdom” ,“Malaysia”)

Note: Is USA really only $1.67 per capita and Singapore $1379 per capita?

COUNTRY MilExpendPercent * GDP * 1000000000 / populationSingapore 1379.85Israel 1901.93United States 1.67United Kingdom 0.72Malaysia 238.91

10

Need to adjust for GDP “billion” and “trillion”

COUNTRY GDP_UNIT MilExpendPercent * GDP * unit/ populationSingapore billion 1379.85Israel billion 1901.93United States trillion 1674.64United Kingdom trillion 716.72Malaysia billion 238.91

Select country, MilExpendPercent * GDP * scalefactor.scale / population from cia, scalefactor where country in ('Singapore', 'Israel','United States','United Kingdom', 'Malaysia') and scalefactor.text=cia.gdp_unit

• Use an auxiliary table– scalefactor

• Now can use database “Join”

text scaleBillion 1000000000Trillion 1000000000000

11

That is great: but want to graph comparisons

• Combine Cameleon web extraction capability with Excel spreadsheet graphing capabilities

• Can put SQL queries to Cameleon in Excel

Query automatically assembled .. And sent to Cameleon

12

Excel provides nice graphics

• Excel can display results in both table form and graphics

13

Need to adjust scale factors

• Some GDP’s are stated in billion and some on trillion – by the way: using USA definition of “billion”

• Can use conditional definition for military_expenditure per capita that handles scale factor adjustment.

14

• Using Excel formula with conditional definition

Final results of “military expenditure per capita” – with scale factor adjusted

(Conditional definition)

15

Much easier to “visualize” graphically• All kinds of additional calculations and graphics possible

country population gdp gdp_unit milexpendpercent miilitary per capitaSingapore 4,492,150 126.5 billion 4.90% 1379.85Israel 6,352,117 156.9 billion 7.70% 1901.93United Kingdom 60,609,153 1.818 trillion 2.40% 719.89United States 298,444,215 12.31 trillion 4.06% 1674.64Malaysia 24,385,858 287 billion 2.03% 238.91

Military expense per capita

Singapore

Israel

United Kingdom

United States

Malaysia

0.00

200.00

400.00

600.00

800.00

1000.00

1200.00

1400.00

1600.00

1800.00

2000.00

miilitary per capita

country

US

do

llars

Singapore

Israel

United Kingdom

United States

Malaysia

16

Military expenditure in multiple currencies• The data in CIA Fact book is all in US dollars.

– Might be more helpful to view in other currencies.

• Currency conversion web sites also exist: www.oanda.com– We “wrapped” it– called it “hzoanda”

Query automatically assembled .. And sent to CameleonSelect anycurrency

17

date rate1/6/2007 1.54

country population gdpgdp_unit

milexpendpercent

Military per capita

Military in Singapore Dollars

Singapore 4,492,150 126.5 billion 4.90% 1379.85 2124.97Israel 6,352,117 156.9 billion 7.70% 1901.93 2928.98United States 298,444,215 12.31 trillion 4.06% 1674.64 2578.94United Kingdom 60,609,153 1.818 trillion 2.40% 719.89 1108.63Malaysia 24,385,858 287 billion 2.03% 238.91 367.93

Military expenditure in choice of currencies

18

These solutions can be even easier through COIN context mediation

• MIT has developed a Context INterchange (COIN) technology that can automatically:– Adjust scale factors and – Provide answers in SG$ (or any currency), etc.

• This query:Select country, MilExpendPercent * GDP * scalefactor.scale /

population from cia, scalefactor where country in ('Singapore', 'Israel','United States','United Kingdom', 'Malaysia') and

scalefactor.text=cia.gdp_unit [currency conversion not shown]

• Could be reduced to:Select country, MilExpendPercent * GDP / population from cia where country in ('Singapore', 'Israel','United States','United Kingdom', 'Malaysia')

• This part of project cannot be demonstrated today due to recent hardware problems that disrupted system and corrupted some software.

19

Information may be scattered across multiple sources

• What if want military expenditure per soldier?

• Can get size of armed forces from http://en.wikipedia.org/wiki/List_of_countries_by_size_of_armed_forces

• Need to combine this data with the information from CIA Fact Book to get military expenditure per soldier

20

Can do “Joins” across multiple sources

select cia.country, GDP, GDP_unit, armed_forces, MilExpendPercent * GDP* scalefactor.unit / armed_forces/1000 as milpersoldierfrom cia, armforces, scalefactor where cia.country in ('Singapore', 'Israel', 'United States', 'United Kingdom') and armforces.country = cia.country and scalefactor.text=cia.gdp_unit

COUNTRY GDPGDP_UNIT

ARMED_FORCES

MILPERSOLDIER

Israel 156.9 billion 168 71912.50Singapore 126.5 billion 60 103308.33United Kingdom 1.81 trillion 190 228631.58United States 12.31 trillion 1426 350481.07

21

What about Armament Knowledge?

• More good news …• There is (we believe) lots of armament

information on SAF intranets– Though we have not been able to access much yet

• There is lots on public Internet – for fee– Jane’s Information Group (http://www.janes.com )

• There is lots on public Internet – for free– Small arms, Tanks, Artillery, etc

• We would need assistance of Subject Matter Experts to determine needed information

• In meantime, here is a brief demonstration …

22

Some Example Questions?

• Firearms & RPG: How do the weights of an AK-47, AK-74, M-15A4, and M16 A2 compare ?

• Tanks: Where is the Centurion Mk13, M1, M1A2, and T-54 made and what is the range and speed of each?

23

How do the weights of an AK-47, AK-74, M-15A4, and M16 A2 compare ?

• Information scattered across multiple sites:http://www.warfare.ru/?lang=&catid=246&linkid=1814

http://www.armalite.com/sales/catalog/rifles.htm

http://www.armalite.com/sales/catalog/rifles/m15a2.htm

http://www.fnmfg.com/products/

http://www.fnmfg.com/products/m16fam/m16a2.htm

24

Can be accomplished by a single query

select weapon, weight, weight_unit from armlite where weapon in('AK-47', 'AK-74', 'M-15A2', 'M16 A2')

union select weapon, weight, weight_unit from warfareru where weapon in('AK-47', 'AK-74', 'M-15A2', 'M16 A2')

union select weapon, weight, weight_unit from fnmanufacturing where weapon in('AK-47', 'AK-74', 'M-15A2', 'M16 A2') *

WEAPON WEIGHT WEIGHT_UNITAK-74 3340 grAK-47 4300 grM16 A2 7.5 lbs.M-15A2 8.2 POUNDS

* This query could be further simplified by using a database View of the 3 db’s.

25

Where is Centurion Mk13, M1, M1A2, and T-54 made and what is the range and speed of each?

• Even when all the information is on a single site, it is hard to pull together for comparisons

http://www.fas.org/man/dod-101/sys/land/row/tanks.htm

26

But can be done with a simple query

select weapon, range, speed from tanks where weapon in ('CENTURION Mk5', 'T-55', 'M1A2', 'M1')

WEAPON RANGE SPEEDCENTURION Mk5 102 34.6M1 498 72M1A2 465 66T-55 500 50

27

Web page spec file *

How does it work? Web-Wrapper Technology

Select Edgar.Net_incomeFrom EdgarWhere Edgar.Ticker=intcand Edgar.Form=10-Q

Ticker Net IncomeINTC 1,983

User or Program (via SQL Query)

Web Wrapper Generat

or

Data record returned

* Spec file contains:Schema, Navigation rules,and Extraction rules.

SQLSide

HTMLSide

28

Recent Technology Extensions to Support DSTA Effort

• Cameleon Studio – Makes it much easier/faster to create spec files

• Incorporate Cameleon into IBM’s DB2 Federated Database System *– To handle much more complex SQL queries

• Calculations• OR and IN features

– Required creation of a general-purpose “capability handling” engine

• Web sites are more limited than databases (e.g., may only take one “key” at a time, only certain fields can be used as a “key”, etc.

* For more information see: Lynn Wu, Aykut Firat, Stuart Madnick, Tarik Alatovic, “Querying Web-Sources within a Data Federation”, Proceedings of the International Conference on Information Systems (ICIS), Milwaukee, Minnesota, December 2006.

29

Cameleon Studio tool makes spec file creation (relatively) easy

30

Previous Cameleon + POE Architecture

Data

SQL

Authentication, etc

Core Cameleon

Simple SQL Query & Output Format

Output in

Desired

Format

Retrieval

HTTPClient

Query Handling

Web or Database

Spec File Parsing

Extraction

Spec Files

Regular Expression Engine

Relational Front End

Planner, Optimizer,Executioner (POE)

Registry

Application

• Our Planner, Optimizer, Executioner (POE) facility extended Cameleon web wrapper

POE handles complex multi-source query (including “join”s):-Divides query into simpler single source queries-Dispatches the queries to core Cameleon or SQL db- Assembles results together

Problem - POE does not handle full SQL needed:- No calculations ( e.g., gdp/capita)- Allows JOINS and UNIONS but not IN or OR

31

IBM’s DB2 Federated Database System• Uses wrappers to access non-relational data sources.• DB2 first decomposes the original query into query fragments and then sends

them to wrappers.– Some generic wrappers provided (e.g., for XML sources)– For arbitrary sources (such as web sites), custom wrappers must be created

• Wrapper sends the result back to DB2 which then assembles the final results.

DB2 XML Wrapper (Adapted from IBM).

32

IBM’s DB2 Federated database system architecture

Wrapper

Web Sources

Capability

Handler

Wrapper for S1

Capability

Handler

Data

Extraction

Wrapper for S2

Capability

Handler

Data

Extraction

Wrapper for S3

S1-website

Wrapper: Request-Reply Protocol

Federation Engine

Query: Select ..from s1,s2,s3 …

IBM DB2

Data

Extraction

S2-website S3-website

For each web site (S1, S2, S3), a wrapper must be custom crafted.

33

Our Solution: (1) Cameleon as general-purpose wrapper for web sources (2) with new capability engine

Two-Layered Architecture - current IBM solution

Three-Layered Architecture - with general-purpose engines

Wrapper

Web Sources

Capability Handler

Wrapper for S1

Capability Handler

Data Extraction

Wrapper for S2

Capability Handler

Data Extraction

Wrapper for S3

S1-website

Wrapper: Request-Reply Protocol

Federation Engine

Query: Select ..from s1,s2,s3 …

IBM DB2

Data Extraction

S2-website S3-website

Wrapper: Request-Reply Protocol

Federation Engine

Query: Select ..from s1,s2,s3 …

Wrapper, Capability

Engine

S1-website

Web Sources

CameleonDataExtractionEngine

IBM DB2

Data Extraction

Engine

Query planning

with capability

declaration

CR for S1

CR for S3

CR for S2

Capability Record Declaration

DE for S1

DE for S2

DE for S3

Data Extraction Spec Files

Wrapper: Request-Reply Protocol

Federation Engine

Query: Select ..from s1,s2,s3 …

Wrapper, Capability

Engine

Web Sources

IBM DB2

Data Extraction

Engine

Query planning

with capability

declaration

CR for S1

CR for S3

CR for S2

Capability Record Declaration

DE for S1

DE for S2

DE for S3

Data Extraction Spec Files

S2-website S3-website

34

Phase 2 – Evaluation of Policy/Intervention Options for Improving Armament Safety

• Approach: Use of System Dynamics (SD) Modeling– Used by US Government/Military for many purposes– Recent example – evaluating interventions on “State

Stability” (see http://web.mit.edu/smadnick/www/wp/2005-13.pdf )

• Goal: Develop a system dynamics model for Singapore Armament Safety– Represent causalities and linkages– Uncover root causes

• Outcome: Model can be used to predict the likelihood of future accidents and help evaluate policy options and preventive measures.

35

Philosophy of System Dynamics• Every action has consequences• Often through complex feedback loops

Do you feel crowded in – and frustrated?

36

See if you can get a bit more space by pushing on that wall …

37

Oops …

38

Quick Primer: The What (and Why) of System Dynamics

Consider the domain of Software Development

• Classic “knee jerk” reaction to a project behind schedule is to add people.• “Brooks Law” noted that “Adding people to a late project, just makes it later” (because the new people must be trained, this takes productive people off the project). • Both of these points are fairly well-known by most software developers – but both are much too naïve. There are many other factors: length of project, type of project, expertise of staff available, approach to and time needed to do training, stage of project, etc. Over the years, all of these individual factors have been well-studied individually – but how do they interact ? • System dynamics makes it possible to model & study the dynamics of these interdependencies. Many non-obvious outcomes have been found (e.g., sometimes Brooks is wrong! Important to know when and Why?) Source: Software Project Dynamics: An Integrated Approach, by T.K. Abdel-Hamid and S. Madnick, Prentice-Hall, 1991.

39

History of System Dynamics

SDM used as modeling & simulation method over 30 years• Designed to eliminate limitations of linear logics and over-simplicity

• Typical human assumptions and behaviours• Based on understanding system structure, behavior patterns, interconnections of positive & negative feedback loops, and intended & unintended consequences of action

SDM has been applied to numerous domains, e.g.,• Software development projects• Process Improvement projects• Crisis and threat in the world oil market• Stability and instability of countries• … many many others …

SDM helps to uncover ‘hidden’ dynamics in system• Helps understand ‘unfolding’ of situations, • Helps anticipate & predict new modes• Explore range of unintended consequences

40

Armament Safety Example: consequences and proceeding actions

Soldier injured

Chamber explosion

Damaged ammunition

Weapon had jammed, which damaged ammunition

Weapon had overheated

Did not stop to cool off

Training was behind schedule

Training started late

Safety officer arrived late

Held up due to traffic accident

HypotheticalExample

Was busy dealing withan earlier accident

41

ChamberExplosions

Injuries

WeaponMalfunctions

Immediate ActionOccurences

+

+

FollowRegulations for IA

Incident Reporting

Level of Training

Follow regulations forammo/ WPNs use

-

Range AvailabilityTime

Ceasfires/Delays

Pressure toComplete Training

+

-

Damaged Ammo

+

+

Overconfidence &ComplacencyR1

Ammo Quality

Serviceability ofWPNs

WPN Cleanliness/Maintenance

Quality ofPre-Marksmanship

Instruction

Attention toWarning Signs

Risk AssessmentQuality

-

-

+

+

B3

-B5

Money Available forTraining, Weapons &

Ammo

Size of WarstockAmmunition

+

-

+

+

-

++

-

---

B4

Resolution ofConflicting Goals

Safety Equipment

-

+

Safety Efforts

Training Level

Preparation

Weapon Age

Availability ofReplacement Parts

+

-

+

-

Ammunition War-Stockto Training Cycle Time

AmmunitionPurchase Rate

Age of TrainingAmmunition

+

-+

-

+

+-

-

Emphasis on Safety &Process Improvement

++

+

+

+

B7

ProtectiveEquipment

B1

Ammunition

B9

Training/OperationTempo

B2

Weapons

B6

Direct SafetyImprovement

B8

Indirect SafetyImprovement

Quality of InformationManagement

+

Other CompetingBudget Requirements

-

Preliminary Example of Armament Safety SD Causal Model

Developed based upon Singapore reports provided and Subject Matter Expertise (SME) provided by US Army Major Nathan Minami, Company Commander - 25th Infantry Division and West Point instructor.

42

Plans for Phase 2

• Focus: Armament Safety Projection & Mitigation

(e.g., Understanding Consequences) - Maybe include Vehicle Safety

• Needs:– Access to Subject Matter Experts

(Plan to also explore US Army sources)• To develop causal model• To determine “tipping points” and other “non-obvious”

behaviour

– Access to Data• To parameterize model for simulation