innov manag_ProQuest_Diss19.pdf

8/17/2019 innov manag_ProQuest_Diss19.pdf

1/171

ONLINE BUSINESS SIMULATIONS: A SUSTAINABLE OR DISRUPTIVE

INNOVATION IN MANAGEMENT EDUCATION?

by

Jason Scott Earl

MARY F. WHITMAN, DBA, Faculty Mentor and Chair

MAUDIE GALLOP HOLM, PhD, Committee Member

CLARK GILBERT, DBA, Committee Member

William A. Reed, PhD, Dean, School of Business and Technology

A Dissertation Presented in Partial Fulfillment

Of the Requirements for the Degree

Doctor of Philosophy

Capella University

June 2012


2/171

All rights reserved

INFORMATION TO ALL USERSThe quality of this reproduction is dependent on the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscriptand there are missing pages, these will be noted. Also, if material had to be removed,

a note will indicate the deletion.

All rights reserved. This edition of the work is protected againstunauthorized copying under Title 17, United States Code.

ProQuest LLC.789 East Eisenhower Parkway

P.O. Box 1346

Ann Arbor, MI 48106 - 1346

UMI 3517084

Copyright 2012 by ProQuest LLC.

UMI Number: 3517084


3/171

© Jason Scott Earl, 2012


4/171

iv

Abstract

The focal goal of this research was to extend the empirical effort on business

simulations as a form of experiential learning by providing the first empirical analysis of

business acumen and knowledge application skills. Disruptions in technology are

providing more opportunities to improve the simulation gaming learning experience and

a number of pedagogical innovations are beginning to emerge which will drive the way in

which business simulations are used in the future. The purpose of this quantitative,

experimentally-based research study was to investigate the use of online business

simulations as a disruptive technology by measuring the change in participants’ business

knowledge and business acumen compared to traditional corporate training. A sample of

65 participants was randomly selected from a company population of 720 employees and

managers. This quantitative based research study demonstrated the disruptive nature of

online business simulations when it comes to gains in business knowledge by measuring

a 2.55 standard deviation difference in the normalized gains between traditional training

and business simulation training. Baseline tests against a control group and traditional

training group using MANCOVA to account for multiple variables and covariates imply

that online business simulations enhance both business knowledge and business acumen

on a staggering scale and over a very short period of time.


5/171

iii

Dedication

This work is dedicated to my wonderful wife and mother of our five young

children. Thank you, Natalie for all of your love and support over these last 15 years. I

would have never left the private equity world and gone into academia without your faith

in me and our purpose in life together. If nothing else, I have learned from this long

journey that true teaching is leadership and most teaching is bad leadership. Thank you

for your example and personal leadership in our own family.


6/171

iv

Acknowledgments

Someone once said that defeat is bitter, but only if you swallow. I would like to

acknowledge the support of Dr. Mary Whitman who believed in me and on more than

one occasion, kept me from swallowing that bitter pill. I would also like to acknowledge

my dissertation committee members who have served as great examples to me and as a

source of personal inspiration. I hope that my life can be a small reflection to my own

students of the great principles that they have taught and, more importantly, lived.


7/171

v

Table of Contents

Acknowledgments iv

List of Tables ix

List of Figures x

CHAPTER 1. INTRODUCTION

Background to the Study 2

Introduction to the Problem 4

Statement of the Problem 6

Purpose of the Study 8

Specific Variables 8

Rationale 9

Research Questions 10

Null Hypotheses 11

Significance of the Study 12

Sustaining vs. Disruptive Innovation 14

Definition of Terms 15

Assumptions and Limitations 18

Nature of the Study 19

Organization of the Study 21


8/171

vi

CHAPTER 2. LITERATURE REVIEW

Background on Experiential Learning 24

Nature of Business Simulations 28

Benefits of Experiential Learning 30

Educational Effectiveness of Simulations 31

Simulations and Corporate Training 33

Knowledge Application 35

Business Simulations as Knowledge Application Tools 36

Knowledge Application & Simulation Performance 38

Reflection 41

Reflection in Experiential Learning 43

Reflection Using Business Simulations 46

Debriefs Within Business Simulation 48

Business Simulations as a Disruptive Innovation 50

Business Simulations as a Disruptive Force Today 55

Andragogical Support for Business Simulations 58

CHAPTER 3. METHODOLOGY

Philosophy and Justification 65

Primary Research Question 66

General Linear Model for the Primary Research Question 68

Null and Alternative Hypotheses 69


9/171

vii

Research Design 70

Justification and Methodology 70

Research Design Strategy 71

Sample 75

Setting 80

Instrumentation and Measures 79

The Business Simulation 82

Dwyer and Ganster’s (1991) Autonomy/ Work Control 84

Field Testing 85

Business Simulation Performance Scores 85

Data Collection 86

Variables 87

Data Analysis 87

Validity and Reliability 92

Ethical Considerations 97

CHAPTER 4. RESULTS

Training for a Semiconductor Company 101

Participant Demographics 102

Normality Testing 103

ANOVA Analysis 109

MANCOVA Analysis 111


10/171

viii

Correlation Analysis 112

Comparison of Means 114

Hypothesis Testing 115

Alternative Hypotheses 117

Primary Research Question 118

Secondary Research Questions 118

Supporting Research Questions 119

Summary 121

CHAPTER 5. DISCUSSION, IMPLICATIONS, RECOMMENDATIONS RESULTS

Summary and Discussion of Results 124

Research Questions 124

Disruptive Innovation 129

General Discussion &Theoretical Implications 130

Practical Implications 131

Limitations and Recommendations for Future Research 135

Conclusion 137

REFERENCES 140

APPENDIX A. PRE-EXPERIMENT SURVEY & ASSESSMENT 152

APPENDIX B. POST-EXPERIMENT SURVEY & ASSESSMENT 157


11/171

ix

List of Tables

Table 1. Summary of Variables Used in This Study 21

Table 2. Instructional Methods Used by U.S. Organizations 33

Table 3. Null and Alternative Hypotheses 69

Table 4. Descriptive Statistics for Survey Items ( N = 65) 104

Table 5. Descriptive Statistics for Dependent Variables ( N = 65) 105

Table 6. Test of Homogeneity of Variances for Dependent Variables 106

Table 7. Test of Normality for Change in Business Knowledge 107

Table 8. Test of Normality for Change in Business Acumen 108

Table 9. ANOVA for Pre/Post-Test Results 109

Table 10. Tests of Between Subject Effects for Experimental Treatment 112

Table 11. Pearson Correlation Coefficients 113

Table 12. Group Statistics for Dependent Variables 114

Table 13. Independent Samples Test for Equality of Means 115

Table 14. Analysis of Variance Between Groups 120

Table 15. Summary of Hypothesis Testing 123


12/171

x

List of Figures

Figure 1. Conceptual Framework 20

Figure 2. Venn Diagram Depicting the Focus Area of this Study 23

Figure 3. Impact of Disruptive Innovation 56

Figure 4. Impact of Disruptive Innovation on Military Training Industry 57

Figure 5. General Linear Model for Primary Research Question 68

Figure 6. Research Design Schematic 72

Figure 7. Overview of the Research Design 74

Figure 8. Comp-XM

®

Comparative Standings 81

Figure 9. Foundation®

Business Simulation Performance Score. 86

Figure 10. Demographic Variable: Level of Autonomy 103

Figure 11. Normal Q-Q Plots and Boxplot for Change in Business Knowledge 107

Figure 12. Post-Test on Business Knowledge vs. Experimental Treatment 110

Figure 13. Post-Test on Business Acumen vs. Experimental Treatment 111


13/171

1

CHAPTER 1. INTRODUCTION

There has been a great divide between the gaming community and business

educators over the past twenty five years (Anderson & Lawton, 2009; Gosen &

Washbush, 2005; Wolfe, 1985). The gamers have dismissed educational simulations as

boring and irrelevant while business management educators have dismissed gaming and

simulations as trivial and pedagogically unproven (Aldrich, 2009a, xxi). Both appear to

be right and yet both may have missed an opportunity which lies within an engaging

business simulation and its potential impact on the world of management education

(Anderson & Lawton, 2009). Many business professionals have argued that the K-12 and

higher education systems are failing, myopically trapped in a nineteenth-century world of

“learning by knowing,” while the twenty-first-century world requires the judgment and

skill of “learning by doing” (Aldrich, 2009b, p. 12). Disruptions in technology are

providing more opportunities to improve the simulation gaming learning experience and

a number of pedagogical innovations are beginning to emerge which will drive the way in

which business simulations are used in the future (Faria, Hutchison, & Wellington., 2009,

p. 485). One of the major challenges with research in this field is that nobody has shown

definitively that simulation training works in the business world any better than

traditional instruction through workbooks or lectures (Davies, 2003, p. 36). The purpose

of this quantitative, experimentally-based research study was to investigate the use of

online business simulations as a disruptive technology by measuring the change in

participants’ business knowledge and business acumen compared to traditional corporate

training.


14/171

2

Background to the Study

In many respects, Bloom’s Taxonomy has been the anchor for assessing whether

learning occurs in business simulations (Bloom, Englehart, Furst, Hill, & Krathwohl,

1956). A substantial share of the early research on simulations focused on the attitudes

(affective domain) of participants exposed to the pedagogy. Much of this initial research

focused on comparing the general perceptions of students regarding cases, lectures, and

simulations (Anderson & Woodhouse, 1984; Blythe & Gosenpud, 1981). Subsequent

research expanded into attempting to assess what is learned from participating in a

simulation and almost all of these studies relied on perceptions and self-reports of

learning, rather than more objective measures (Anderson & Lawton, 2009, p.211). When

focused only on those studies that aim to examine participants’ affective reaction to

simulations, it is evident that students like simulation exercises and view them more

positively than either lectures or case discussions (Burns, Gentry, & Wolfe, 1990; Faria,

2001; Gosen & Washbush, 2004). It is worth noting that these relative comparisons have

been made by students experiencing different pedagogies within a course. There is a

dearth of studies employing experimental designs with control groups or where

comparisons are made between participants’ attitudes in one section of a course that is

solely lecture based versus those in a class that is solely case discussion based or solely

simulation based. Most business classes are taught today with one main pedagogy (e.g.,

lecture, case study or simulation) and almost no experimental studies exist that compare

learning outcomes under alternative pedagogies (Anderson & Lawton, 2009, p. 208).


15/171

3

Wolfe (1990) identified this problem over 20 years ago, yet the gap still exists.

Researchers continue to use self-assessments rather than more suitable tools because self-

assessments are much easier to employ. As a consequence, studies on the educational

merits of simulations often are measuring the affective domain, not the cognitive domain

they purport to measure (Anderson & Lawton, 2009, p. 197). Using perceptions tends to

be advantageous to those who wish to claim the superiority of simulations over

alternative pedagogies because simulations almost invariably are rated positively by

students. The downside of using perceptions is that evidence based on perceptions often

is dismissed by scholars because it lacks suitable rigor. However, studies that attempt to

go beyond perceptions to more objective measures of learning more often than not use

tools best suited for measuring lower levels of learning on Bloom’s taxonomy (Anderson

& Lawton, 2009, p. 209). The decisions required to effectively run a business simulation

often tap analytical, synthesis, and application skills of Bloom’s taxonomy (Bloom,

1956). This has led some researchers to believe that using simulations is a powerful and

disruptive form of learning because it is taking place at the higher levels of Bloom’s

taxonomy (Smith, 2006).

Disruptions often bring significant changes to an industry and these disruptions

create opportunities for those organizations willing to adopt and champion disruptive

technology (Smith, 2006). Christensen (1997) has highlighted the sometimes devastating

impact in the corporate environment of what he refers to as disruptive innovations (p. 41).

Successful, well managed firms that dominate their markets have sometimes gone into a

sharp decline or even collapsed when a new technology disrupts the pattern of their

market segment. Other firms and organizations, however, have handled such transitions


16/171

4

smoothly, maintaining their position of dominance in the market by employing specific

techniques to integrate the new and disruptive technologies into their operations

(Christensen & Raynor, 2003, p. 57). Traditional research universities enjoy a dominant

position in the higher education market, but they are beginning to feel the impact of

disruptive innovations such as online universities, distance education, and continuing

education units as semiautonomous incubators (Archer, Garrison, & Anderson, 1999,

p.137).

According to Smith (2006), changes in the underlying technology for online

simulations have improved to the point that they are now more powerful than many of the

established pedagogical tools in the field of management education (p. 8). This

disruption in the field of management education is very similar to the innovation model

that Christensen first proposed in his dissertation and built upon in The Innovator’s

Dilemma (Christensen, 1997). This theory of disruptive innovation allows for a detailed

lens to be focused on business simulations in an attempt to identify them as sustaining,

complementary, or disruptive innovations. Grüen-Yanoff and Weirich (2010) argues that

these technological and economical forces which are at work in the education industry

will create a tsunami of change throughout management education. Thus, they predict

this change will allow for the spread of more cost-effective, more powerful, and more

accessible simulations across the field of corporate training and management education

(Grüen-Yanoff & Weirich, 2010, p. 45).


17/171

5

Introduction to the Problem

Of great concern to management education scholars is the identification of factors

which will contribute to rapid changes in business and how business leaders in the future

will be taught (Smith, 2006). Although many educators do not see simulations as a threat

to traditional education programs today, there are signs showing a significant impact

from simulations already (Andersen & Lawton, 2007). The advent of flight simulators

and computer games has finally introduced a technology and learning media which is

interactive, low-cost, and scalable. Today, authors are creating “virtual velds” where

participants can repeatedly practice skills, instead of just hearing about them (Aldrich,

2009b). As individual managers and employees gain more autonomy or “work control”

over day-to-day business decisions, the ability to take advantage of these disruptive

technologies is expected to increase dramatically. An empirical study performed at a

manufacturing facility involving corporate training demonstrated that experimental

interventions that aim to augment worker control over their tasks and work environment

increased productivity and innovation (Dwyer & Ganster, 1991). It may be that

education reformers are signaling the end of the age of learning “how to know” rather

than “how to do” or “how to be” in a complex, interactive world (Aldrich, 2009a).

Davies stated that part of the problem is that nobody has shown definitively that

simulation training works in the business world any better than workbooks or lectures

(2003). The challenge is determining whether these simulations are truly disruptive

innovations to management education or simply sustaining innovations.

According to Christensen (1997), the main reason why so many successful and

dominant institutions fail due to innovation within their industry is their inability to


18/171

6

recognize the difference between sustaining and disruptive innovations. Sustaining

technologies are typically sought after by these successful institutions because they

improve the performance of established products for their most lucrative clients. This

strikes many innovative professionals as paradoxical, because the excellent business

practice of listening closely to their customers does not lead to disruptive innovation,

only incremental sustaining innovations. Technologies, in the sense that Christensen uses

the word, may refer to either “hard” technologies that result in new types of physical

goods or “soft” technologies that result in new ways of organizing work or providing a

service. Interactive simulations are referred to as veld technologies based on the

learning-to-do skills. It is possible that these technologies will successfully challenge the

institutions of management and corporate education in the near future (Aldrich, 2009a,

p.212).

Statement of the Problem

D. Goleman, R. E. Boyatzis, and A. McKee (2004) suggest that one of the largest

mistakes in management education is to assume that simply acquiring more information

(e.g., business knowledge) will automatically lead to becoming a more effective manager

or leader. They state that the development of competencies in cognitive or intellectual

ability (e.g. business acumen) leads to outstanding performance. Goleman et al also

suggests that self-directed learning is an effective method of achieving sustainable

changes in both business knowledge and business acumen (2004, p. 274). Online

business simulations are quickly becoming an effective tool for self-directed learning

while enhancing these management competencies (Segon & Booth, 2009, p. 112). The


19/171

7

problem thus becomes how do business management educators significantly increase the

level of both business knowledge and business acumen for learners today?

Because of the open-ended content of most business simulations, participants can

have a hard time articulating what they have learned during their game-play experience

(Anderson & Lawton, 2007). For example, there are no well-conducted studies that

actually investigate the learning effects of business simulations on both learners’

knowledge application skill and business acumen (Anderson & Lawton, 2009, p. 209). In

fact, almost all studies have focused on the students’ attitudes and perceptions of their

experience with the simulation (Aldrich, 2009, p. 220). The real gap in the research and

literature is the use of business simulations in the world of adult learning or andragogical

approach to learning (Anderson & Lawton, 2009). There is a need to determine whether

simulations are a disruptive innovation because “the holy grail of research within

management education is to empirically demonstrate that andragogical techniques lead to

better learning outcomes” (Knowles, Holton & Swanson, 2005, p. 235). Recent advances

in computer simulations allow potential business leaders to practice and rehearse these

business management skills in a context rich environment removed from real life, thus

allowing a participant the opportunity to strengthen their skills in an atmosphere of safety

(Sidor, 2008). Using computer based simulations, participants have the ability to review

business management skills and potentially modify their real world application skills.

This forms the following study’s primary research question: Do online business

simulations provide an increase in knowledge or business acumen for participants, which

is on the order of a disruptive innovation?


20/171

8

Purpose of the Study

The purpose of this quantitative study was to investigate the use of online

business simulations as a disruptive technology by measuring the change in participants’

business knowledge and business acumen compared to traditional corporate training.

This research was undertaken to prevent management educators from continuing to miss

the opportunity for creative learning by adopting inappropriate educational strategies that

are of little relevance to practicing managers (Burns, 1995, p. 284). Gosling and

Mintzberg (2004) have stated that business management is neither a science nor a

profession, neither a function nor combination of functions. Business management is a

practice and this research has a high level of relevance at this time because it allows

participants to appreciate the experience of making business decisions within a given

context. Management may use science, but it is an art that is combined with science

through craft (Gosling & Mintzberg, 2004, p. 19). This topic has been of repeated

interest in both simulation and gaming journals as well as business journals focused on

policy, research, and management education (Segon & Booth, 2009).

Specific Variables

Participants in the simulation group ran an online business simulation, titled

Foundation®

by Management Simulations Inc. which simulates five years of running a

company in a high-tech industry (Foundation®

, 2012). This “simulated” industry is very

similar to the semi-conductor industry where all participants work as managers and/ or

employees. A sample of 65 participants were randomly selected from a semiconductor

company population of 720 employees and managers for this research. The two


21/171

9

experimental groups (case study and business simulation) responded to prompt questions

during the three day training period. In an attempt to investigate these research questions,

this study also looked into which of the following variables contributed most to

participants’ business simulation performance:

(a) Number of years of industry experience

(b) Level of education (i.e., undergraduate, graduate)

(c) Area of expertise in the company

(d) Level of autonomy/ work control at the company

(e) Previous business simulation experience

Rationale

The following topics have been addressed by some scholars as fertile areas for

research: Do participants improve their grasp of interrelationships among the various

functions of business (marketing, finance, production, etc.) as a result of participating in a

simulation? Are the interpersonal skills of participants improved through participating in

a simulation? Do participants in simulations really develop a greater appreciation for the

difficulty of implementing what may, on the surface, appear to be rather straightforward

business concepts? Are business simulations really effective devices for integrating

participants into business programs, and are they effective at improving retention rates?

(Anderson & Lawton, 2009, p. 212).

Based on these research questions, both practical and theoretical reasons were

present for conducting this study. From a practical perspective, knowing the benefits of

engaging students in reflection activities may be very helpful in order to improve


22/171

10

business acumen and knowledge application skills for the participants. Understanding

the impact that this experiential activity can have on knowledge application and

simulation performance will provide information that is valuable to both designers and

administrators of future simulations. Furthermore, if the types of prompt questions used

in this study; (a) strategy questions for business acumen and (b) financial and/or

accounting questions for knowledge application, turn out to be effective in promoting

participants knowledge and simulation performance, then simulation designers and

business instructors will have a basis from which to make informed decisions when it

comes to designing a better learning experience. On a theoretical level, this study

integrated a number of the precepts of experiential learning, reflective learning, and

online simulations in order to provide information on the factors that promote

participants’ knowledge. This combination of instructional techniques clearly

demonstrated the impact of this teaching innovation. Past research has shown that a two

standard deviation gain in a pre/post-test of interactive-engagement compared to

traditional instruction may place this innovation on the order of disruptive scale for

education (Hake, 1998).

Research Questions

This research attempted to address the following research questions:

Primary Research Question:

1. Do online business simulations provide an increase in business knowledge orbusiness acumen for participants, which is on the order of a disruptiveinnovation?

Secondary Research Questions:


23/171

11

2. Are knowledge application skills in business positively correlated with onlinebusiness simulation performance scores?

3. How does the change in knowledge application skills with traditionalcorporate training compare with online business simulations?

Supporting Research Questions:

4. Do participants who are taught using an online business simulation gain asignificant increase in business knowledge and business acumen?

5. Do participants who are taught using traditional corporate training gain asignificant increase in business knowledge and business acumen?

6. Is the level of education or industry experience positively correlated withonline business simulation performance scores?

7. Is the level of participant autonomy/ work control positively correlated withonline business simulation performance scores?

This study used a MANCOVA technique for data analysis, based on three

experimental groups. The full general linear model and associated hypotheses are

discussed in Chapter 3. The null hypotheses based on these research questions are listed

below:

Null Hypotheses

Ho1. There is NO difference across experimental groups for business knowledgeafter adjusting for previous simulation experience and autonomy/ work control.

Ho2. There is NO difference across experimental groups for business acumenafter adjusting for previous simulation experience and autonomy/ work control.

Ho3. NO correlation exists between participants’ business acumen and business

knowledge.

Ho4. Participants who engage in an online business simulation will NOTdemonstrate higher business knowledge after experiencing the businesssimulation.


24/171

12

Ho5. Participants who engage in an online business simulation will NOTdemonstrate higher business acumen after experiencing the business simulation.

Ho6. There is NO difference between the simulation group, the case study group,and the control group based on their change in business knowledge.

Ho7. There is NO difference between the simulation group, the case study group,and the control group based on their change in business acumen.

Ho8. NO correlation exists between participants’ business simulationperformance and their level of autonomy/ work control in the company.

Ho9. NO correlation exists between participants’ business simulationperformance and their years of industry experience.

Ho10. NO correlation exists between participants’ business simulation

performance and their level of education.

Significance of the Study

This study provided key insight into what happens to participants in a business

simulation by measuring the overall increase in business knowledge and business

acumen. Several researchers have documented the benefits of using business simulations

in management education (Chapman & Sorege, 1999; Lefebvre, 1997; Segon, 2009).

The most common reported benefits included practice in an environment without risk,

increased creativity, more focused competitive analysis, increased cross-functional

understanding, and increased subject content knowledge. Although case studies and

practitioner experience support these benefits, little empirical evidence is offered on the

change process that an individual participant experiences (Scherpereel, 2005, p. 389).

Specifically, research has not studied adequately the effects of engaging students in a

business simulation and measuring the change in business acumen or knowledge

application skills. Anderson and Lawton (2009) refer to this as a “dearth of studies


25/171

13

employing experimental designs” ( p. 196). In addition to the possibility of simulations

operating on the order of a disruptive innovation, there is virtually no existing literature

investigating the order of magnitude impact that business simulations have on

participants when it comes to quickly and effectively mastering business management

skills (Anderson & Lawton, 2009, p. 211).

This study added to the body of knowledge on the nature and impact of business

simulations by employing five methods of measurement: (1) a pretest and posttest

designed to assess the participants’ knowledge application skills, (2) a pretest and posttest

designed to assess the participants’ business acumen, (3) participants’ responses to

different prompt questions, (4) business simulation performance scores, and (5) survey

questionnaires. The survey questionnaires were designed to investigate the participants’

academic and work-related background as well as their individual level of autonomy or

work control within the organization. A sampling of 65 managers and employees at a

semiconductor manufacturing facility were randomly assigned to three different groups.

These groups consisted of (a) the traditional instruction (case study), (b) the online

business simulation group, and (c) the control group. Participants in both the traditional

instruction group and online business simulation group were prompted to respond to

strategy questions which evaluated their business acumen. These same participants were

also asked to answer knowledge application questions which evaluated their business

knowledge. Participants in the control group were not prompted to respond to either

strategy questions or knowledge application questions.


26/171

14

Sustaining vs. Disruptive Innovation

Christensen’s (1997) theory of disruptive innovation was built upon the concept

of radical and incremental innovation initially proposed by Dewar and Dutton (1986).

Christensen proposed that disruptive innovations are different than radical innovations in

that they have the value-destroying characteristics of radical innovation; however, these

innovations work much more slowly and methodically up through an industry’s value

chain. Christensen’s theory states that disruptive innovations have a beginning point that

is actually much lower on the performance scale than similar existing technologies in the

same market. According to Smith (2006), the reason why low performance innovations

are so disruptive is due to their ability to meet the needs of a niche market that is

unaddressed by the current leading products and technologies (p. 4). Christensen (1997)

states that these disruptive innovations often grow in underserved markets because of

their low cost and consequently, are perceived as insignificant and less profitable to the

industry leaders due to much lower margins. This disdain held by industry leaders is

typically due to the small size of these niche markets and the small profits that are

available from them. Consequently, disruptive innovations move slowly up the value

chain and often take time to destroy the value of established products and technologies

(Christensen, 1997). In fact, almost all growth from these innovations is in completely

new markets. These disruptive innovations erode the value of formerly successful

institutions by systematically stealing away customers from the bottom of the value chain

and gaining more and more market share over time. As these disruptive innovations

improve the quality of their products or services to customers, they eventually meet the

expectations of a large customer base which is being served by the industry leaders and


27/171

15

suddenly become a real threat. While business simulations have been recognized as a

separate pedagogy for instruction, there is little empirical evidence in the research

literature on the magnitude of this innovation when it comes to measuring the increase in

business knowledge or business acumen (Faria et al., 2009, p. 485).

Accordingly, this research contributed to both the management education and

disruptive innovation literature. By investigating the order of magnitude impact that the

business simulation had on participants based on the pre-test and post-test analysis, it was

possible to determine whether online business simulations are simply a sustaining

innovation for educators or truly a disruptive technology that will change how managers

and business leaders learn in the future.

Definition of Terms

The following definitions were used in this study:

Assessment

The art and science of testing individuals to determine what they have learned or,

as is more often the case, what they have not learned (Grüne-Yanoff & Weirich, 2010).

Business Simulation

Computer-based role-playing game which makes use of high fidelity simulated

environments and involves decision-making in the research & development (R&D),

marketing, operations, human resources (HR), and finance departments of a company

(Grüne-Yanoff & Weirich, 2010).


28/171

16

Business Strategy

This is a particular game-play-strategy or group of unique business decisions

which are in complete alignment in order for participants to achieve the highest possible

score on simulation performance (Grüne-Yanoff & Weirich, 2010).

Disruptive Innovation

A term coined by Clayton Christensen and used in business and technology

literature to describe innovations which improve a product or service for non-consumers

in ways that the market does not expect. This is typically done by lowering price or

designing for a different set of consumers. Few technologies are intrinsically disruptive

or sustaining in character; however, it is the strategy or business model behind the

technology that it enables, which creates the disruptive impact (Christensen & Raynor,

2003).

Experiential Learning

A learning model which begins with the experience, followed by reflection,

discussion, analysis, and evaluation of the experience (Albert, 1970).

Game Based Learning

A learning method which combines educational content and elements of computer

games (Aldrich, 2009a).

Knowledge Application

The process of selecting appropriate business knowledge suitable to the challenge

at hand, and making connections between selected business knowledge and specific

strategies (Sarin & McDermott, 2003).


29/171

17

Online Business Simulation

An instructional method based on a representation of a physical or social business

reality in which participants compete for certain outcomes according to an established set

of rules or constraints. The competition can be (1) among themselves as individuals or

groups, or (2) against some specified standard, working as individuals or cooperating as a

group (Szczurek, 1982).

Prompt Questions

Questions that prompt participants’ reflection and guide the process of their

knowledge application during the course of the simulation. Two different types of

prompt questions will be used for this study – the Strategy Question and the Knowledge

Application Question.

Reflection

An important human ability, in which a person recaptures his or her experience,

thinks about it, mulls over it and evaluates it (Boud et al., 1985). Reflection typically

takes place after an experiential learning experience. These reflection periods are often

referred to as “debriefs” with an online business simulation.

Simulation Performance

Measured at the end of each round (or year) and defined by the company’s

Balanced Scorecard which is a combination of metrics for the Company involving

financial ratios (i.e., return-on-equity and stock price), learning and growth of people,

customer satisfaction, and internal business processes.


30/171

18

Sustaining Innovation

A sustaining innovation allows for increases in performance at typically higher

costs in the same market without effecting non-consumers from other markets.

Sustaining innovations allow a given technology to continue to improve in their own

market, but they do not directly impact other markets (Christensen & Raynor, 2003).

Assumptions and Limitations

In order for this study to proceed, certain assumptions were required. First, the

study assumed that valid and reliable data existed and could be obtained. Second, the

study assumed that all managers at the semiconductor manufacturing facility would

participate in the Capstone (Management Simulations, Inc.) business simulation. Third,

participants would accurately complete the survey questionnaire. Fourth, all three groups

of participants would have a similar level of knowledge of business management related

to managing a growing business in a highly competitive industry.

As with most empirical studies, a number of limitations began to emerge. Some

of these limitations had an effect on the validity and reliability of results, as described

below.

1. This study used a specific type of business simulation which is in the domain

of managing a growing business in the semi-conductor or electronic sensor

industry; hence it may be difficult to generalize results to other types of

business simulations.

2. This study used a relatively small sample size of 65 business managers in the

semi-conductor or electronic sensor industry, both limiting the kinds of


31/171

19

quantitative analyses that can be conducted and reducing the generalizability

of results.

3. Analyses were limited to mainly quantitative methods. While inferential

statistical procedures showed significance with numerical data, there was a

need for some qualitative analysis to shed light on the participants’ learning

experience with a business simulation. This study allowed for some limited

qualitative analysis in a short-answer post-test survey.

4. This study focused on a group of managers and employees at the same

electronic sensor manufacturing company. Consequently, it will be difficult

to generalize results to other managers or employees working in other

industries.

Nature of the Study

A true experimental study was used to investigate the possible cause-and-effect

relationships by exposing two experimental groups (business simulation and traditional

instruction) to one or more treatment conditions and comparing the results to a control

group not receiving the treatment.

Figure 1 highlights the conceptual framework that was used in this study, showing

how each of the variables relates to the Research Questions and Hypotheses.

The conceptual framework reiterates the problem that was used for this study.

There are no clear empirical studies on whether online business simulations are a

sustaining or disruptive innovation when compared with traditional instruction. The

problem established the framework for the study and allowed for both business acumen

and business knowledge to be measured in a true experimental study.


32/171

20

Figure 1. Conceptual Framework. The research questions and hypotheses in this figuretie out with the dependent, independent, and demographic variables in order to define theproblem in this study.

Characteristics of this experimental design required rigorous management of

experimental variables and conditions by direct control/manipulation or through

randomization (Isaac & Michael, 1997). The independent variables in this study were

participation in an online business simulation or participation in traditional instruction.

There was also a control group which did not participate in either. The primary

dependent variables were the change in business knowledge and change in business

PROBLEM

Are Online Business Simulations a Sustainingor Disruptive Innovation?

DEPENDENT VARIABLES

- Business Simulation Score

INDEPENDENT VARIABLES

- Online Business Simulation- Traditional Instruction

DEMOGRAPHIC VARIABLES(Impact Performance)

SimulationExperience

Autonomy/W-Control

IndustryExperience

EducationLevel

WHY?

Reflection Activity:1. Level of Engagement2. Ability to Apply Learning3. Peer Instruction4. Bloom’s Taxonomy

∆∆∆∆ in Business

Knowledge

∆∆∆∆ in Business

Acumen

RQ4

H1-H4

RQ4

H2-H5

RQ6

RQ7

RQ2

H3

RQ1

H6

H7

H9H8H1-H2

RQ3

RQ5

H10

Level of Autonomy in

the CompanyH8


33/171

21

acumen based on the pre/post-test for each group. The intent of this study was to make

ten predictions about the relationships among the variables identified in the null

Hypotheses. A summary of all variables is found in the table below (Table 1).

Table 1.

Summary of Variables Used in This Study

Organization of the Study

Following this introduction, the next chapter includes a thorough review of

relevant literature in the areas of experiential learning, online business simulations, and

disruptive innovation. This review supported the conceptual framework used in this

study and helped illuminate the gaps in existing works which represents opportunities for

future research. In chapter three, the research methodology is described

comprehensively, including the research design, experimental approach, review of

measurement instruments, and the data analysis procedures. Reliability and validity

concerns are also discussed. Chapter four reports results from the experiment as well as

from the subsequent data analysis which was conducted according to the procedures

outlined in chapter three. Finally, chapter five includes a discussion of results and their

Variable Type of Variable Measurement Level

Pedagogical Nature of Training Experimental Treatment Independent Ordinal

Change in Business Knowledge Dependent Comp-XM® Ratio

Change in Business Acumen Dependent Comp-XM® Ratio

Knowledge Application Score Dependent Foundation® Ratio

Level of Autonom y at Com pany Independent Mediating Work Control (Dwyer) Ratio

(Continuous Covariate)Simulation Experience Independent Moderator Survey Item Nominal

(Dichotomous Covariate)

Area of Expertise Descriptive Survey Item Nominal

Industry Experience Descriptive Survey Item Ordinal

Level of Education Descriptive Survey Item Ordinal


34/171

22

implications. The study concludes with recommendations for future research based on

the results from the experiment.


35/171

C

Corporations, ma

and collegiate business p

2004). Many educators a

to apply their knowledge

integrative, and iterative

empirical studies that act

participants’ knowledge

fact, almost all studies ha

experience with the simu

helps fill an important re

business simulations, exp

Figure 2. Venn diagram

23

APTER 2. LITERATURE REVIEW

agement education institutions, development c

ograms use business simulations to train and te

ssume that business simulations will help partic

to solve real-world problems based on their “in

nature” (Anderson & Lawton, 1997). However,

ally investigate the learning effects of business

pplication or increase in business acumen (Mil

ve focused on the students’ attitudes and perce

lation (Anderson & Lawton, 2006). The resear

earch gap by examining the relationship betwe

eriential learning, and disruptive innovation.

epicting the focus area of this study

nsulting firms,

ach (Summers,

ipants learn how

eractive,

there are no

simulations on

er, 1998). In

tion of their

h in this study

n online


36/171

24

Overall, the educational merits of business simulations have been subject to

considerable debate (W. Biggs, 1990). While many universities use simulations, only a

small fraction of the faculty members within those institutions integrate simulations into

their coursework (Summers, 2004). There are studies which indicate that other forms of

pedagogy are just as effective or more effective than business simulations. Anderson and

Lawton (1990) argued that only a very weak link between participation in a simulation

and learning has been shown. These researchers also pointed out that valid, reliable

instruments to assess mastery are rare, and valid measures of higher level learning of

objectives are almost nonexistent. Kayes (2002) noted that while countless management

scholars and practitioners see ‘experience’ as central to management learning, the notion

of experience has received critical attention. Kayes explains that criticisms of so called

experience-based learning arise for both empirical and theoretical reasons (p. 137).

Based on the gaps identified in the existing literature, this research study seeks a

causal relationship between online business simulations and experiential learning as a

possible source of disruption when compared to more traditional methods of corporate

training. This study therefore connects experiential learning theory from psychology

(Lewin, 1935) with the developing field of disruptive innovation (Christensen, 2008) in

order to support or refute the business case for significantly higher levels of adult

learning through the use of online business simulations.

Background on Experiential Learning

The use of business simulations as a form of experiential learning and disruptive

innovation has evolved significantly over the last six decades. The evolution and


37/171

25

development of experiential philosophy has been significantly influenced by five

individual researchers: John Dewey, Kurt Lewin, Jean Piaget, , David Kolb, and Clayton

Christensen.

Dewey's (1938) Experience and Education focuses on the conflict between

traditional and progressive education. The essence of Dewey’s work was that truth and

knowledge are not absolute but rather continuously evolving over time. Dewey

explained that experiences directly influence knowledge and what is come to be known

as truth. He further argued that experience should be incorporated into the education

process and that all education should be participatory in order to be experience-based.

Several of Dewey’s ideas have made their way into traditional educational programs over

the last sixty years, particularly at the primary and elementary levels. As individuals

grow, they find that new experiences conflict with earlier learning and knowledge.

Experiential educators in higher education are addressing these challenges by using an

innovative approach that incorporates the best of traditional and experiential

methodologies (Ruben, 1999).

Kurt Lewin is often referred to as the founder of American social psychology and

his work has laid much of the foundation for modern educational and organizational

development work. Lewin’s (1935) research on experiential learning and group

dynamics had a profound influence on the discipline of social psychology and

organizational behavior. Lewin’s studies on group dynamics and the methodology of

action research led to laboratory-training. This training is now considered one of the

most significant educational innovations of this century when it comes to the process of

learning and change (Kolb, 1984). Lewin described the change that takes place during


38/171

26

learning as a three-stage process; (1) “unfreezing” or overcoming inertia and dismantling

the existing mindset, (2) a period of confusing and transition where old ways are being

challenged but there is no clear picture of the new mindset that will take place, and (3)

the third and final stage he called “freezing.” Lewin (1947) describes this new mindset as

crystallizing and former comfort levels are returned to the learning participant.

Jean Piaget, a renowned French psychologist and epistemologist, is another major

contributor to experiential learning. The essence of Piaget's (1973) work is based on the

description of how intelligence is shaped by experience. Piaget stated that learning is the

product of interaction between the person and their environment. The growth and shape

of intelligence is impacted by decisions and the realization of consequences for each

decision. In this interaction, the ability for the person to act and experience consequences

is key. In Piaget’s studies of children, from infants to teenagers, this research

demonstrated the importance of abstract reasoning and interaction with the environment.

The ability of the child to manipulate symbols comes directly from the infant’s actions in

exploring and coping with the immediate concrete environment.

David Kolb (1984), an American educational theorist, laid the foundations of

modern experiential education theory based on the idea that knowledge is gained through

both personal and environmental experiences. Based on Lewin’s earlier work, Kolb

developed the experiential learning cycle which has been widely reproduced based on a

four-stage model of learning. Initially called “The Lewinian Experiential Learning

Model,” this model is now primarily recognized as Kolb’s (1984). Kolb states that in

order to gain genuine knowledge from an experience, certain abilities are required; (1) the

learner must be willing to be actively involved in the experience, (2), the learner must be


39/171

27

able to reflect on the experience, (3) the learner must possess and use analytical skills to

conceptualize the experience, and (4) the learner must possess decision-making and

problem solving skills in order to use the new ideas gained from the experience. This

theory offers a fundamentally different view of the learning process from that of the

behavioral theories of learning based on an empirical epistemology or the more implicit

theories of learning that underlie traditional educational methods (Kolb, 1984).

Christensen’s (1997) The Innovator’s Dilemma laid the foundational research

which helped to explain why successful companies and institutions often fail to invest

aggressively in disruptive technologies. When applying this theory to higher education,

Christensen (2008) states that the more “student-centric” classrooms become, the more

demand there will be for new technologies. Christensen’s research also suggests ways to

identify innovations which are about to disrupt entire industries. The primary method for

identifying these innovations is that they begin as products or services which are much

simpler and cheaper than the existing competition. Also, these innovations generally

promise much lower margins and very little profit. The secondary method for

recognizing disruptive technologies is based on the fact that they are typically

commercialized in new and/ or insignificant markets (Archer, 1999). The third way that

disruptive innovations are recognized is based on the perspective of the leading firms’

most profitable customers. Typically, these high-end customers, who are willing to pay

much higher prices, don’t want or can’t use such inferior products or services. In almost

every case, a disruptive innovation is initially embraced by customers who mean very

little to the industry leaders in that particular market. The great irony of this theory of

disruptive innovation is that those companies which are the best at listening to and


40/171

28

serving their most profitable and most successful customers are rarely able to build a case

for investing in disruptive technologies. When they finally do, it is often too late .

Christensen’s (1997) research highlights the fact that disruptive innovations in the

past were technologically straightforward and did not rely on a specific business model.

In fact, Christensen argues that many of the early disruptions in the computer industry

consisted of off-the-shelf components put together in a product architecture that was

simpler than prior approaches. These innovations typically offered less of what

customers in established markets wanted because they were much lower in product

performance. These disruptive innovations also began in emerging markets and

consequently, offered a different package of attributes which were considered

unimportant to the industry leaders. Christensen (2008) argues that through experiential

learning in a virtual environment, assessment–the art and science of testing individuals to

determine what they have learned–can be revolutionized . One of the potential areas for

this revolution in the virtual learning environment is online business simulations.

Nature of Business Simulations

Clearly distinguishing simulations from simulation games is quite difficult and

debatable. While a simulation imitates reality and is often used to predict what would

happen in a given scenario, the word “game” suggests playfulness and competition.

Simulation games combine all of these characteristics. Many researchers now use the

term simulation game (Jacobs & Dempsey, 1993) to describe a new class of games that

make use of high fidelity environments. Simulation games have also been defined

variously as a combination of simulations and games with competition (Heyman, 1982)


41/171

29

and as a subset of games (McGrenere, 1996). One of the more complex definitions is

proposed by Szczurek (1982). Szczurek referred to such educational tools as an

instructional method based on a simplified model or representation of a physical or social

reality in which participants compete for certain outcomes according to an established set

of rules or constraints. The competition in simulations is often against some specified

standard, where participants can work as individuals or cooperate as a team.

The origin of the business simulation dates back to 1955. In that year, the Rand

Corporation developed an exercise called Monopologs (Jackson, 1959). Monopologs

required its participants to perform as inventory managers in a simulated Air Force

supply system, thus providing decision-making experience without the risks associated

with the consequences of a wrong decision. The Air Force continued the use of

Monopologs for many years and reported it to be a highly successful training device.

One of the first practical and most successful business simulations for the masses was

Top Management by the American Management Association (AMA) in 1956. It was

used in numerous management seminars (Meier, Newell, & Pazer, 1969). Additionally,

the consulting firm of McKinsey and Company developed the Business Management

Game in 1957 for use in its management seminars (Andlinger, 1958) and the University

of Washington became the first university classroom user of a business game when a

simulation developed by Scheiber was used in a business policy course in 1957 (Watson,

1981).

The increased usage of business simulations can be measured in several different

ways. First, the number of simulations available in the market has increased dramatically

over the last ten years. Second, the number of organizations and journals devoted to


42/171

30

business games and simulations has also risen significantly (Faria, 2009). An email

survey of 14,497 business faculty members across all disciplines at American Association

of Collegiate Schools of Business member schools led to only 1,085 respondents. Of

those who responded, 30.6% were current business simulation users, 17.1% were former

simulation users and 52.3% had never used a business simulation in their coursework. In

earlier work, Faria (2004) had estimated that 95% of AACSB schools (The Association to

Advance College Schools of Business) and 86% of all business schools in the United

States were using business simulation games. Surprisingly, business simulations showed

the highest use in business policy and marketing areas.

Benefits of Experiential Learning

Experiential exercises, including role-plays and simulations, have been widely

used for educational purposes. Learning objectives are thought to be accomplished by

providing realistic, but controlled, environments in which students are guided only by

implicit rules. Although there are distinctions between different modes of simulation, the

design of most simulations allows students to be exposed to stimuli that encourage them

to acquire the key concepts of the subject area being taught (Druckman & Ebner, 2008).

Cherryholmes completed the earliest evaluations of simulation learning outcomes (1966).

He evaluated five hypothesized topics–interest, learning, retention, critical thinking, and

attitudes–with six studies using complex simulations conducted over periods of time

ranging from one day to 12 weeks. The results were clear: Only interest in the material

being learned by the simulation participants improved significantly (compared to case

study and other conventional classroom approaches); negligible changes occurred on


43/171

31

learning and attitudinal variables. However, there were convergent findings when “the

task of designing a simulation before playing it, either re-designing an existing game or

constructing one of their own” (Cherryholmes, 1966, p. 7). Based on finding from

Cherryholmes research, a design opportunity will be provided for in this study, where

participants will be able to choose the financial outcomes and ratios that are most

important to their chosen strategy while running the business simulation.

Educational Effectiveness of Simulations

Pierfy (1977) reviewed the findings obtained from comparative evaluation studies

reported during the 1960s and 1970s. With regard to learning, 15 of 21 studies showed

no difference between simulations and other instructional techniques. With regard to

retaining the information learned, 8 of 11 studies showed that the students participating in

simulations retained the information longer than those exposed to other instructional

techniques. On interest, 7 of 8 studies reported that students showed more interest in the

simulation activities than in more conventional classroom tasks. In their update of

Pierfy’s review, Bredemeier and Greenblat (1981) concluded that simulations are as

effective as, but not better than, other instructional methods on learning the subject. They

stated that simulations are more effective when used only as aids to retaining the learned

material and in instilling a positive attitude toward the subject matter. Also supporting

these results is research completed by Ellington, Fowlie, and Gordon (1998) who found

that simulations have an advantage over traditional methods in motivation, participant

involvement, and commitment. A question suggested, but not answered, by the studies is

why the learning impacts are modest. Wolfe and Crookall (1998) have also asked why


44/171

32

the field of experiential gaming has made little progress. Regarding educational

effectiveness, a variety of suggestions for significantly improving the contribution of

simulations has been given. Examples include clarifying learning objectives (Bredemeier

& Greenblat, 1981), providing more conceptual background on the subject prior to the

simulation activity (Druckman & Robinson, 1998), creating time for reflection on the

events and getting feedback (Mclaughlan & Kircpatrick, 2005), and providing

participants with conceptual maps and graphics that reflect the simulation’s purpose

(Druckman & Ebner, 2008).

In contrast to just game playing, actual simulation design contributes to analysis

by identifying critical elements (roles, goals, resources, and rules) leading to new

analytical questions (Ebner & Efron, 2005). Attention to the design process remains a

strong focus, as evidenced by the simulation-building exercise featured at the 2007

International Simulation and Gaming Association conference at Nijmegen in the

Netherlands (Durckman & Ebner, 2008). These observations suggest the hypothesis that

simulation designers learn more about the concepts being simulated than do simulation

role-players. Crookal expounded on this hypothesis when he stated the key features of

the design process: (a) Design is concrete – you can touch the results; (b) it is creative –

you develop an object, and (c) it is involving – you develop understanding in a passionate

and intimate way (1995, p.161). When participants have the ability to make changes

within the simulation and design their own experience, the learning about relations

between different concepts goes up significantly and approaches synthesis (Greenblat,

1998). In the case of an online business simulation, the participants in this research study

will have the opportunity to design their own performance metrics for their own company


45/171

33

and specific industry of competitors. This interactive engagement with the simulation

may lead to higher levels of motivation, participant involvement and commitment.

Simulations and Corporate Training

Organizations have a wide variety of methods available for training their

employees and business simulations have begun to enter this market. Table 2 reports

U.S. organizations’ usage of selected instructional materials and methods from Training

Magazine’s 2003 survey (Galvin, 2003). Computer-based games and simulations have

low usage rates, with 1% of respondents always using them, 9% often using them, 47%

seldom using them, and 44% never using them.

Table 2

Instructional Methods Used by U.S. Organizations (Galvin, 2003)

Instructional Method Often or Always Never or Seldom

Instructor-led classroom 91 9

Self-study, Web based 44 56

Performance Support 44 56

Public seminars 42 58

Case studies 40 60

Role-play 35 65

Non-computer-based games, simulations 25 75

Self-study, non-computer based 23 77

Virtual classroom with instructor 21 79

Computer-based games, simulations 9 91

Experiential programs 6 94

Virtual reality programs 3 97SOURCE: “Industry Report” (2003, p. 31)

Percentage of Respondents


46/171

34

Despite the low percentage of business simulations used in 2003, Summers (2004)

outlines three specific advantages to using computer-based simulations in corporate

training which will allow it to grow exponentially in the future. These three advantages

are; (a) specific knowledge, (b) learning on demand and (c) lower costs (Summers, 2004,

p. 226). Summers also argued that many new technology companies are introducing

business simulations to corporations based on the benefits of learning on demand (p.

228). This research poses the question: “Will the new technology companies come to

dominate or even replace the old?” If so, this would be a case of creative destruction, a

concept posed by Schumpeter (1911/1989).

Alternatively, the new technology companies which introduce these simulations

to corporate training programs may simply increase the number and variety of products.

When the theory of disruptive innovation (Christensen, 1997) is cast on this dilemma, the

outcome depends on whether the new technology is superior and whether the new

technology companies can consolidate the industry (Summers, 2004, p. 232). Online

business simulations have rapidly penetrated business schools; however, the superiority

of such experiential learning methods has not been proven in the corporate training arena.

A simple increase in the number and variety of training methods for management

education would indicate a sustaining innovation for the industry, while a significant

consolidation of the corporate training industry would indicate a disruptive innovation

(Christensen, 2008).


47/171

35

Knowledge Application

Application of theory is an on-going issue in higher education (Falkenberg,

Russell and Ricker, 2000). The fact that most of what participants learn is intended for

application to problem situations in real life is indicative of the importance of knowledge

application as a learning objective, especially in the field of management education.

Bloom, Engleheart, Furst, Hill, and Drathwohl (1959) developed a system that classified

learning into six levels. These levels arranged in a hierarchical order to reflect

progressively higher levels of learning. They are, in ascending order: basic knowledge,

comprehension, application, analysis, objective synthesis, and objective evaluation.

Bloom (1956) showed the components of knowledge application in the problem-

solving process of answering questions. The process involves six steps, in ascending

order: restructuring and classifying situations, selection of abstraction suitable to problem

type, the use of abstractions to solve a problem, and solution to a problem. This process

shows that in order to solve a problem through knowledge application, there are certain

steps to be followed. Learning how to apply knowledge means learning how to follow

these steps effectively. Bloom (1956) distinguished knowledge application from

knowledge comprehension by saying that a demonstration of comprehension shows that a

student can use the abstraction when its use is specified; while a demonstration of

application shows that he/she will use it correctly, given an appropriate situation in which

no mode of solution is specified.

Bloom (1956) stated that comprehending an abstraction does not certify that the

individual will be able to apply it correctly. Thus, participants need practice in applying

their knowledge to real-world problems in order to make their knowledge more useful for


48/171

36

real-world decision making rather than remain inert. In this study, “knowledge

application in a business simulation” refers to the process of selecting appropriate

business knowledge suitable to the problem at hand, and making connections between

selected business knowledge and business strategies to solve a complex problem. Agyris

and Shon (1974) argued that the only way for organizational effectiveness to increase

over time was through individuals learning from experience. Online business simulations

build upon this concept because the application of learning is based almost completely on

the participants’ experiences. Agryis and Shon (1974) stated that these types of

experiences provide insights during the learning process and allow for specific

knowledge application tools to be developed leading to consistent learning outcomes .

Business Simulations as Knowledge Application Tools

One major problem that comes from learning with business simulations is that it

is not always clear that learners will leave with exactly the same conclusions, mental

models, and learning outcomes. In fact, it is not clear that learners will be able to apply

what they have learned in future real-world situations. From their experiment with

business school students, Mandl, Gruber, and Renkl (1992) confirmed that students using

a computer-based simulation had serious deficits in knowledge application and problem

solving using their previous knowledge. The results of the experiment clearly show that

business school students have considerable deficits in using their own declarative

knowledge that they acquired in business school, and they are not able to use their

knowledge as a tool in the real world. This is largely due to the fact that those students

gained their business knowledge mostly through traditional methods, like lectures, case


49/171

37

studies, and textbooks, and had not had enough opportunities to practice applying their

knowledge to real-world problem-solving either in a specific situation or in a simulated

environment. As a result, their knowledge was not sufficiently conditioned to relevant

application conditions, and remained inert.

Training students to apply their knowledge requires very different methods of

instruction than training them to memorize information or understand relationships within

a business context (Reigeluth & Moore, 1999). In business education, the most

commonly used instructional methods are those of linear formats, such as

lecture/textbook format and case analysis. These linear formats can be more efficient

than the experiential learning method for communicating a large number of concepts to a

large number of students. However, these formats do not do enough to encourage

creativity, problem solving, decision making, risk taking, and knowledge application. In

addition, most knowledge that business school students acquire from their lectures and

textbooks is what Anderson (1985) called the “declarative knowledge”: various business

concepts and principles are taught in a declarative form rather than an experiential form.

Therefore, when students learn business knowledge, they often treat new information as

facts to be learned rather than knowledge to be used. As a result, many business school

students have difficulties in using their knowledge as a tool in their business decision

making (Mandl et al., 1992). Since their knowledge is not based on actual experience, it

often remains inert.

Not all business knowledge needs to be taught in a procedural form. However,

most of those management-related concepts and principles consist of knowledge about

how to do things, which is what Anderson (1985) called the “procedural knowledge.”


50/171

38

Procedural knowledge is goal-oriented performance knowledge that can be executed

efficiently. Therefore, when learning this kind of management-related knowledge,

students need opportunities in which they can transform their declarative knowledge into

procedural knowledge so that their knowledge can be more readily available for real-

world problem solving rather than remaining inert. One important instructional goal of

business simulations is to help students transform their declarative knowledge into

procedural knowledge. Business simulations are designed to help students practice in

applying their knowledge, which is mostly in a declarative form, into specific action-

oriented problems in a relatively safe, controlled, and simulated environment.

Knowledge Application & Simulation Performance

Business instructors have promoted simulations as a means of accomplishing a

wide range of learning objectives, including improving interpersonal skills

(VirtuaLeader), improving general decision-making skills (Capsim), and helping

individuals to understand themselves (Second Life). Anderson and Lawton (1997)

outlined learning objectives common to simulation exercises. These include outcomes

such as increased knowledge of facts and concepts of the business discipline; improved

analytical skills, critical thinking, decision making, and interpersonal relations; enhanced

ability to simultaneously manage interrelationships; and a better understanding of

business dynamics.

While business simulations are believed to have the potential to stimulate learning

at all levels of learning objectives, many scholars argue that simulations are best suited to

facilitate learning at the higher level of Bloom’s Taxonomy of learning objectives:


51/171

39

application, analysis, and synthesis of knowledge (Anderson & Lawton, 1997). In fact,

researchers in business education have frequently used Bloom’s Taxonomy as a

framework for guiding their thinking on the areas where simulations are likely to have the

greatest impact on learning (Bloom et al., 1959). Since business simulations require

participants to act in the role of managers, it would seem likely that, if business

simulations excel in any area they would be strong in application (Anderson & Lawton,

2002). However, objective evidence for business simulation effectiveness at these higher

levels of Bloom’s taxonomy has been lacking.

Measuring the higher levels of Bloom’s taxonomy of learning objectives has

proven to be a difficult task. A lack of reliable and valid instruments has hindered

attempts to measure the learning occurring at the higher levels of Bloom’s taxonomy

(Anderson & Lawton, 1995). Thus, while Bloom’s taxonomy provides a useful

framework for the purpose of establishing learning objectives, the framework has not

been as helpful for assessing student learning, especially in the context of business

simulations. However, Anderson and Lawton (2002) conducted a study that has been

directed exclusively at the efficacy of simulations as a pedagogy for learning about the

application of business concepts. The premise underlying their research was that if

simulation performance does reflect learning associated with analysis and application,

those students who apply the concepts that are critical to the discipline should outperform

those who do not apply the knowledge presented in the course. The results of their study

showed a significant relationship between the application of concepts presented in a basic

marketing course and performance on a marketing simulation game. The greater the


52/171

40

number of concepts that students utilized in the management of their simulation

company, the higher their performance scores.

These findings provide a powerful validation for simulations as learning and

teaching tools. In answer to the question of whether applying the principles and concepts

of a discipline results in positive results in a simulation, the answer appears to be an

emphatic “yes” (Anderson & Lawton, 2002). Their study demonstrated that simulations

are useful tools for operating at the application level of Bloom’s hierarchy, the level at

which traditional classroom lectures are thought to be weak. De Jong and Van Joolingen

(1998) point out that an appropriate design theory for instructional simulations may arise

based on this higher form of learning. These researchers also defend that “discovery

learning” with simulations can take its place in learning and instruction as a new line of

learning environments based on technology where more emphasis is being placed on the

learner’s own responsibility ( pp. 19).

Other research that investigated the link between knowledge application and

simulation performance is found in a study conducted by Wolfe and Luethge (2003).

Wolfe and Luethge investigated whether students, who were involved more in the

simulation and applied more business knowledge over time, performed better than

students who were less involved and applied less business knowledge. They examined

the following in this study; (a) the degree to which simulated companies in a business

simulation need to be run by active, knowledgeable, and engaged players and (b) the

extent to which strategic involvement affects performance. The results of this study

found that less involved students, who applied less business concepts, performed poorly

relative to more engaged students. The results of their study indicated that good


53/171

41

performance in a business simulation is not the result of luck or random guesses and that

a business simulation rewards intelligent, planned decision-making practices.

Reflection

The word “reflection” appears frequently in the literature relating to experiential

learning. Boud et al. (1985) defined reflection as an important human ability in which

people recapture their experience, think about it and evaluate it. Dewey (1910) pointed

out that all genuine education comes through experience and that reflection can assist in

this process. The importance of reflection goes back to Dewey’s early writing, but there

has been increased interest in researching and using reflective processes in adult teaching

in the last twenty years (Salmon, 2001).

Bruce (2001) stated that reflection is described as contemplating the results of a

given experience within the overall context of the impact on the individual. Boud, Keogh

and Walker (1985) argued that only when this reflective process leads to a significant

change in behavior, can it be called reflective learning. Costa and Garmston (1994)

stated that reflective learning is the ability to mentally wander through a recent personal

experience. This mental process of reflection includes the following; (a) drawing forth

cognitive and emotional information from visual, auditory, kinesthetic, and tactile

sources, (b) linking information to previous learning, (c) comparing the results that were

anticipated and intended with the results that were achieved, (d) searching for effects and

finding connections among causal factors, (e) acting on and processing the information

by analyzing, synthesizing, and evaluating, (f) applying learning to contexts beyond the


54/171

42

one in which it was learned and making commitments to plans of action, and (g) the

metacognitive process of thinking about thinking.

Some researchers argue that reflection is essentially an independent activity. An

online business simulation allows for ideal period of reflection in between rounds of

company decisions. This period of reflection or debrief allows participants to ask critical

questions about their company performance and challenge their own basic assumptions

about strategy and execution (Davies, 2003; Foreman, 2004). Other researchers stress the

importance of collaboration with others in terms of the reflection process (Rose, 1992).

Lin (1999) argued that students’ reflection can be enhanced in a reflective social

discourse, and defined reflective thinking as actively monitoring, evaluating, and

modifying one’s thinking and comparing it to both expert models and peers.

Reflection on experience is based upon the metacognitive theory developed by

Flavell (1987), who argued that becoming aware of oneself as a learner allows the student

to reflect, monitor, and revise the process and products of his own learning. The term

metacognition itself emerged from the early work of Flavell who referred to it as

knowledge concerning one’s own cognitive process and products or anything related to

them (Flavell, 1976).

J. Biggs (1985) discussed the role of metacognition in learning, utilizing the term

“metalearning” to define the application of metacognition to student learning. More

particularly, he also defined metalearning as students’ awareness of their learning and

control over their strategy selection and employment. According to J. Biggs (1988), a

metalearner is one who is aware of their motives, task demands and personal cognitive

resources and exerts control over strategies used. J. Biggs (1988) also stated that these


55/171

43

reflections invite

Documents

innov manag_ProQuest_Diss19.pdf