blended-learning pm 3.1 · Josh Bersin January 2006 Bersin & Associates Blended Learning Program Management: WhatWorks Issues, Solutions, and Platforms for the Management of Blended

Josh Bersin

January 2006

© Bersin & Associates

Blended Learning Program Management: WhatWorks® Issues, Solutions, and Platforms for the Management of Blended Learning Programs

I N D U S T R Y A N A L Y S I S

Blended Learning Program Management 2

Bersin & Associates © January 2006 Not for distribution Licensed material

Table of Contents

Table of Contents ......................................................................................... 2

Overview...................................................................................................... 3

The Case for Blended Learning..................................................................... 4 E-learning Alone Often Fails ...................................................................... 4 Defining Blended Learning ........................................................................ 5

The Process of Blended Learning.................................................................. 6 The Components of a Blended Program..................................................... 6

Sixteen Media Types for Blended Learning Programs................... 7

The Growing Role of Collaboration and Communities .............................. 10

The Challenge of Blended Learning Program Management......................... 13 What is Program Management and Why is it Important? ........................ 13 The Difference between Programs and Courses ...................................... 14 E-learning Courses and SCORM ............................................................... 14 Program Management User Interfaces.................................................... 16 Implications for Learning Management Systems..................................... 17

Today’s Solutions ....................................................................................... 18 Home Grown Solutions............................................................................ 18 LMS Providers ......................................................................................... 19 Q2Learning eCampus Platform................................................................ 19 Thomson NETg LearnFlow Platform......................................................... 21 SkillPort from SkillSoft ............................................................................ 22

Conclusions ................................................................................................ 23

Appendix 1:................................................................................................ 24 Eight Criteria for Building a Blended Learning Program .......................... 24

Appendix 2:................................................................................................ 24 Features to Consider in a Blended Learning Management Solution ......... 24

About Us .................................................................................................... 26

About This Research................................................................................... 27



Overview

As the use of e-learning continues to grow in corporate training, more and more organizations are using programs that blend many forms of media to drive high levels of mastery and retention. We have been studying best practices in blended learning since 2001 and have seen a steady increase – particularly in high-impact areas such as new hire training, sales training, management training, and executive education.

This white paper describes a key issue in blended learning: how organizations use their technology platforms to manage and measure blended programs.

Our research indicates that despite broadening adoption of blended learning, organizations find the process of managing and measuring the elements of a blended program a challenge. In this paper, we explore the issue of blended learning program management and illustrate some new approaches to solving this growing problem.



The Case for Blended Learning

As organizations adopt e-learning, they now realize that it alone is limited. The concept of blended learning, melding a wide range of instructional approaches into a single program, now is common. Our research, in summer of 2005, shows that 30% of all e-learning content buyers blend content with some form of instructor-led training, online reference materials, or live Webcasting1

E-learning Alone Often Fails

Many e-learning programs do fail – they fail to drive high rates of adoption, completion, and impact. In the early days of e-learning, programs suffered from dropout rates of 60% and higher. Many corporate managers tell us they have to continuously market their programs and still cannot get high levels of adoption.

Why do such failures occur? One of the biggest problems is the paradigm itself. Workers today are busy doing their jobs, reading email, and going to meetings. Unlike traditional training, e-learning is very easy to opt out of. There are no fringe benefits to e-learning: you do not get away from the office nor do you meet other people. It is easy to disengage.

To make matters worse, many times Internet-based content is boring, slow, and buggy. Many off-the-shelf courses are nothing more than pages of text with a few colorful graphics. We ask people to squeeze this activity into an already overcrowded day of work, meetings, email, family obligations, and commuting.

“The concept of ‘build it and they will come’ does not work. We have to continually market, evangelize, and promote our programs to remind people to complete the programs they have started.” Ceridian

“We tried e-learning alone… our e-learning programs simply did not take off. Learners repeatedly told us that they were too busy and could not take the time during the day to focus on our Web-based courseware. We found that blending was the only way to move forward from awareness to proficiency and mastery.” U.S. Insurance Company

Blended learning solves these problems.

1 “The Four Stages of E-Learning: A Maturity Model for Online Corporate Training,” available at http://www.bersin.com/stages.

http://www.bersin.com/stages



Defining Blended Learning

The term “Blended Learning” has become a buzzword – so much so that it has taken on many meanings. In “The Blended Learning Book2,” it is defined as follows:

“Blended learning is the combination of different training media (technologies, activities, and types of events) to create an optimum training program for a specific audience.

“The term blended comes from the concept that traditional instructor-led training is being supplemented with other electronic formats. In the context of this book, “Blended Learning” programs use many different forms of e-learning, perhaps complemented with instructor-led training and other live formats.”

Blended learning quite simply means taking a variety of delivery types (we call them media types) and crafting them into an integrated program designed to solve a particular training problem. Blended programs can consist of a few or many elements – usually including instructor-led training or workshops, some self-study, some Web-based content, and, increasingly, online collaboration – mixed together to meet a particular audience need.

2 The Blended Learning Book, by Josh Bersin, published by John Wiley & Sons, Pfieffer Imprint, ISBN, 0787972967, available at http://www.bersin.com/research/blended_book.asp.

http://www.bersin.com/research/blended_book.asp



The Process of Blended Learning

To understand the issues of managing blended programs, it is important to understand a bit about the process of building a blended program. There are three major steps:

1. Defining the program strategy, learning plan, and components

2. Building the content components themselves

3. Launching, delivering, and managing the program

Let us now briefly examine steps 1 and 2 before we identify the challenges in step 3. (In our research, E-Learning Program Management, What Works®3, we discuss the specific steps in marketing, launching, and managing an e-learning program. This study is an excellent review of best practices in launching, marketing, and managing all forms of e-learning programs.)

The Components of a Blended Program

To define a blended program, one must carefully look at many requirements: the business problem, the learning problem, audience characteristics, time issues, technology issues, and more. In The Blended Learning Book, we identify eight key criteria that drive creation of the right program elements. These are included in the appendix of this white paper.

Today, there are at least 16 different potential media elements you can use to build a blended program. As an instructional designer or program manager, you should consider them all. Each has its own strengths and limitations, and each brings with it a set of challenges in content development and program management.

3 E-Learning Program Management: WhatWorks®, available to Bersin & Associates research members or for purchase at http://store.bersinassociates.com/program-mgt-study.html.

http://store.bersinassociates.com/program-mgt-study.html





Sixteen Media Types for Blended Learning Programs

Media Live or Self-study

Strengths Weaknesses

1 Instructor-led training Live Expert driven, celebrity

status, social interaction Expensive to deliver, difficult to schedule, logistics

2 Webinars (Live e-learning) Live Expert driven, celebrity

status, very low cost to deliver

Lack of community, requires training by instructor

3 Courseware (Web-based) Self-study Can be highly detailed,

easy for learner to self-study

Can be dull and uninteresting to occasional learners, requires heavy content development investment

4 Simulations

Application Simulation Business Simulation Process Simulation

Self-study Highly interactive, can be very mastery-oriented

Very expensive to build and may not pay off

5 CD-ROM based courseware Self-study Highly interactive, often

with video Very expensive to build and may not be pertinent to immediate need

6 Rapid E-learning Courseware (PowerPoint-based)

Self-study Low cost, easy to build, can be SME-authored

Typically not valuable for mastery or skill and competency based programs

7 Internet delivered video Self-study Interesting, engaging Expensive to build and deliver,

difficult to edit

8 Electronic Performance Support Systems (EPSS)

Self-study Highly productive Difficult to build, only pertinent to large audiences w/ scalable problem

9 Offline videos (videotapes) Self-study Interesting and engaging

for expert-driven program Requires lots of time to view, no random access to information

10 Video conferencing Self-study Highly interactive, less $$

than classroom education Requires heavy capital investment

11 Collaboration systems Self-study and

live Engaging for self-study or live, captures history of interactions easily

Needs to be carefully designed, moderated, and monitored for each program application

12 Conference calls Live Excellent for info

distribution Can be boring/dull for long periods of time, not possible to drive skills or training materials well

13 Job aids Self-study Excellent for job support

and reference materials Not valuable for new skills and competencies, but good as support tool

14 Work books Self-study Excellent for formal

education, classroom training

Typically not used for self-study unless learner is highly motivated

15 Books Self-study Excellent media for

interesting and complex topics, reference materials

Most business people do not have time to read, often use as reference

16 On the job exercises Often a

combination of self-study and

live

Very valuable for culture building and mastery programs

Requires management to be trained in advance and monitoring to be effective

Figure 1: 16 Media Types for Blended Learning



Each of these elements has its own special role. When crafting a management training program, for example, it may be important to include group exercises that bring people together to develop peer relationships. In a new hire program, one may include a collaboration environment that lets new employees ask questions of each other and of the HR department during their initial orientation and new hire assignments.

To give you an example of a blended program, consider the Ninth House program shown below.

Figure 2: Typical Blended Program - Ninth House Example

This is a good example of a well-architected, blended program for management training that includes initial exercises, self-study, exercises, workbooks, quizzes, and supporting resources. This particular program is mostly online, but most organizations use it in the context of a workshop-based management training program, which is conducted in-person. Together, the e-learning components and the classroom exercises create a high impact result.



As you can see from this illustration, there are many moving parts to a blended program. The program manager must manage many forms of content and make sure that learners understand the program flow. The program includes some self-study, workshops, some assessments, and many supporting resources. Managing this program takes special processes, skills, and tools.



The Growing Role of Collaboration and Communities

As more and more people have online access, a fast-growing component to blended programs is the use of online communities and discussion rooms. By dividing up learners into small groups and giving them access to discussion rooms designed to address specific activities, you can create the online equivalent of a “breakout session” in e-learning programs. Typically, these breakout rooms are carefully managed and have a subject matter expert assigned as a facilitator.

If managed in this way, many companies can bring together learners from all over the world to create a community and a sharing of best practices.

Shell Oil, for example, uses online communities to bring together small groups of new hires from all over the world. Individuals meet each other in a kickoff meeting and then interact online in discussion rooms to complete exercises during their first year of new hire orientation in a small and intimate group.

CNA Insurance uses small groups which interact during online exercises (reviewing each other’s work, providing feedback) for its performance management training program.



Figure 3: Online workgroups collaborate on activities

Another example is the custom system built by AT&T to train service center representatives. IT includes an online community with an online coach to help learners progress through its programs.



Figure 4: AT&T Custom Online Learning Environment



The Challenge of Blended Learning Program Management

What is Program Management and Why is it Important?

Imagine that you are in the process of architecting a blended program. You identify the key learning strategies, create an instructional plan, and build a variety of elements that fit together into a cohesive program. But now you have to launch, deliver, and manage the program. We call these steps the “program management” phase. In many ways, this is the most important phase because it is here that you deliver value to the business and to the audience.

What are some of the program management issues you face? Let us cite some specific examples:

• Marketing and launching the program to learners and their managers.

• Recruiting and training coaches. • Identifying any role for the learners' managers (such as

developing stretch assignments or certifying that learners are applying new skills on the job).

• Notifying learners of the program, its start date, enrollment process, and why they need to join.

• Grouping learners into small groups for different scheduled sessions.

• Managing the progression of each group through a series of program elements, and making sure that groups stay together.

• Assigning subject matter experts to help facilitate and support learners from step to step.

• Monitoring subject matter experts to make sure they provide timely and consistent feedback.

• Grading exercises and providing feedback to learners in a timely fashion.

• Tracking and monitoring discussion rooms and exercises to make sure that learners get the support they need when in online communities.

• Developing reports that identify which learners are falling behind, which are completing, and which may need assistance of some kind.

• Developing similar reports for tracking coaching activities and any activities of the learners' managers.

• Providing feedback to management on the status of completion of each learner or learner group.

• Answering questions and providing daily updates to learners so they keep current on program progress.

• Reporting on completion or certification when the program is completed.



This is only a small sub-set of everything that needs to be done. In a physical university, these are activities that often are handled by teaching assistants or class administrators. In corporate training, however, resources are very limited and often there are hundreds or thousands of learners progressing through a program. How do you manage all these steps?

Unfortunately, these are problems that usually are not well solved by Learning Management Systems. Let us examine this problem in more detail.

The Difference between Programs and Courses

A key definition is needed here.

One of the most important issues you face when you adopt a blended approach is the concept of a program. In the traditional e-learning or instructor-led training approach, learners take courses, which are a linear set of activities, usually built into a strict syllabus or course outline that takes a learner from beginning to end. A program is a collection of activities, often including courses, collaboration, exercises, and other learning objects.

Our existing e-learning standards do not characterize or manage programs well.

E-learning Courses and SCORM

E-learning courses were built to mimic the outline of courses taught in the classroom. They have an introduction, a series of learning activities (i.e., lectures), assessments (i.e., quizzes), exercises, and then a final assessment and conclusion. Their structure has been defined by the e-learning standards, AICC and SCORM.

ADL/SCORM4 standards assume that a learning event is a course, and within the course there are individual chapters or modules. Each of these modules is called a sharable content object (SCO) and it must be specially designed to use certain embedded technology that allows it to be launched and tracked by a learning management system (LMS). SCORM has the concept of sequencing, which means that a course can jump from SCO to SCO depending on how well the learner is doing. If a learner does well on an initial assessment, for example, the course may skip many of the introductory SCOs.

4 Advanced Defense Language/Scalable Content Object Reference Model (ADL/SCORM) is the current industry specification for e-learning content, publishing, and metadata management.



SCORM (and its predecessor AICC5) have been very valuable for one major reason. Courseware that adopts the SCORM and AICC standards can be launched easily and tracked by a learning management system. Through the SCORM and AICC standards, each mouse click and test response can be stored in a database, thereby enabling corporate training managers and executives to track enrollments, completions, certifications, scores, and many other important training metrics. All major Learning Management Systems (LMS) adopt SCORM standards for launching, packaging, and tracking.

Unfortunately, however, SCORM misses the concept of a learning program.

In a real world blended program, many of the learning activities are not simple, trackable e-learning objects. For example, what if you have a workshop that kicks off the program? What if you send people to a discussion room to complete an assignment? What if you give learners a set of reference books and videos to watch? SCORM misses these elements for two reasons:

1. These learning activities may not be trackable. Many blended activities (workshops, exercises, Webinars, discussion rooms) do not adopt SCORM standards and, therefore, are not easily tracked and managed in learning management systems unless the LMS vendor builds special features to track these activities. This means that there is no easy way to develop a report that shows you which of these activities have been completed by whom, when they were completed, and how successfully they were completed.

2. Blended programs do not have linear or easily sequenced structures. Blended programs often involve taking multiple e-learning courses and putting them together with real world events. The whole concept of a SCORM-based course does not fit. Blended programs, by their nature, are not all electronic, so the e-learning standards fall short.

5 Aviation Industry CBT Committee (AICC) is the standard in use, now being replaced by SCORM, for launching, tracking, and scoring of CBT and Web-based courseware. AICC is widely implemented in almost every content and LMS system in the market.



Program Management User Interfaces

One of the most important issues in a blended learning program is making it easy for the learner to understand what to do and when to do it. What is needed is an easy-to-use learning map.

An excellent example of a program management interface is the learning map provided by Q2Learning.

Figure 5: Q2Learning Map

This simple, easy to use map shows a learner where they are and where they need to go. On the left pane, it illustrates the types of activities a learner will go through, enabling the learner to jump to the activity they want to focus on. In the center, they can see the whole program. Other key features, such as who is on now, are important in building the online community.



Implications for Learning Management Systems

The result of this problem is that most learning management systems are built around the concept of courses not programs. The LMS course catalog is just that: a catalog of courses. If you want to mix and match courses and learning activities into a total program, the LMS functionality is often limited.

Many LMS vendors have added features to help you manage assessments, resources, and online communities – but rarely are they well integrated enough to manage all the different aspects of a flexible blended learning program. LMS vendors are relatively small companies working hard to add many new features to their systems. However, this particular area has been lacking.

It is not enough to have a collaboration tool built into your LMS – it must be trackable and manageable as part of an integrated learning program.

For example, suppose you want to assign learners to stop after chapter 3 of a course and submit an exercise to their workgroup through the discussion room. Can the LMS enforce this and generate reports that show whether or not a learner has submitted this exercise? Can the LMS then display the grade on these exercises? Can the LMS enable different subject matter experts to be assigned to different learner groups? These are the types of features which many Learning Management Systems lack.



Today’s Solutions

So how do companies manage blended programs? We believe this is a problem yet to be solved, but there are essentially two options:

1. Build your own management platform

2. Look for an off-the-shelf solution

Home Grown Solutions

Most companies solve these problems by building Web portals and online communities that build on the LMS to provide an easy-to-use learning environment. A good example of this is the AT&T Knowledge Center. This system was built by AT&T and serves as the learning environment that front ends the company’s learning management system.

CharacterSimulationCharacterSimulation

LearningEnvironmentLearning

Environment

CharacterSimulationCharacterSimulation

LearningEnvironmentLearning

Environment

Figure 6: Custom Learning Environment



As this screen shots indicates, a home grown solution usually involves building upon a Web portal. This will have a login and authentication system, along with collaboration software and, often, a reference database of knowledge objects. The portal usually lets a user take an online course by passing login information directly to the LMS.

Companies that adopt this approach usually are those with large training organizations and support from IT. Because of the difficulty of using an LMS to manage blended programs, this is a very common approach. As blended learning continues to grow, we see a range of new vendor solutions starting to become available. These vendor solutions often are provided by content providers, consultants, or companies that have background in the collaboration market.

LMS Providers

All major LMS vendors have some form of program management in their LMS. These systems have evolved significantly in the last 12-18 months and many have built-in collaboration features. Unfortunately, most LMS-developed program management systems are still hard to use for complex blended learning programs.

Q2Learning eCampus Platform

One of the most advanced and mature blended learning systems we have seen is the eCampus solution from Q2Learning (www.q2learning.com).

Q2’s eCampus is designed to manage a large number of learners who need access to online courseware, collaboration tools, learning resources, and subject matter experts. In addition to the easy-to-use learning map interface shown in Figure 5 (page 16), Q2’s system has powerful features for activity management, coach management, and reporting.

http://www.q2learning.com



Figure 7: Q2Learning Activity

Figure 8: Q2Learning Learner Progress Reporting



Some of the unique features of the Q2 system include:

• The ability to group large numbers of learners into workgroups called “cadres.” A cadre is like a virtual discussion group that enables a group of learners to see each other, work together, and be coached as a group. This concept is one of the most advanced online blended learning features we have seen and is instrumental in managing large programs (hundreds to thousands of learners).

• The ability to automatically assign coaches to specific cadres within the program. Again, this is very important in large programs where there may be several subject matter experts who serve as coaches or instructors.

• The ability to create programs that integrate content (WBT, flash, resources) with collaboration (live Web meetings, online discussions, coached work samples).

• The ability to easily track learner and coach performance through reports designed for learners, coaches, and managers.

• A fully customizable user interface that allows you to create a very easy to follow learning map, as shown above.

• 24x7 online assistance through the company’s hosted solution.

• Strategy and design services to help clients formulate a custom blended strategy.

Thomson NETg LearnFlow Platform

Another unique solution in this space is an emerging product from Thomson NETg called LearnFlow. LearnFlow is a complete learning environment that manages a learners’ progression through a series of activities, including instructor-led programs, online activities, and resources. The product was designed to help learners complete complex IT certification programs that have many independent reading assignments, exercises, and assessments.



Figure 9: LearnFlow from NETg

LearnFlow originally was designed to manage IT certification programs that have a wide set of activities (reading, exercise, tests) that must be completed in a fixed period of time to gain a certification. The product is being opened up for custom programs and is just becoming available in Q1 of 2006. LearnFlow is available to selected customers through Thomson NETg today, (www.netg.com). We believe that LearnFlow has the potential to become a significant new platform offering to manage custom blended programs.

SkillPort from SkillSoft

SkillPort, the LMS platform solution from SkillSoft, has a newly released program management system designed to manage blended programs. SkillSoft’s system is designed to manage a wide range of SkillSoft implemented activities and content: SkillSoft courses, SkillSoft ReferenceWare, and SkillSoft Dialogue virtual classroom sessions and replays.

SkillPort’s program management interface also is being opened up to third-party systems through the company’s online learning services architecture (OLSA). This enables learning management systems, or other content providers, to use this capability.

http://www.netg.com



Figure 10: SkillPort from SkillSoft

Today, SkillPort’s program management solution is an excellent way for SkillSoft clients to build custom programs from SkillSoft content and dialogue elements. Over time, we believe SkillSoft will further open this system up to custom programs and other content providers.

Conclusions

Solutions for blended learning program management are just emerging. While most LMS vendors see this as their domain, we believe that corporate buyers with a commitment to blended programs should consider a specialized solution today. The role of collaboration, online coaching, group management, presence awareness, and online performance support is changing rapidly – and these valuable tools should be well integrated to drive an effective learning experience.



Appendix 1:

Eight Criteria for Building a Blended Learning Program

These eight criteria are described in detail in The Blended Learning Book2.

1. Program Type

a. Information Distribution

b. Critical Information Transfer

c. Skills and Competency Development

d. Certification Program

2. Culture-building Goals

3. Audience Characteristics

4. Budget Limitations

5. Access to Resources

6. Time Pressures

7. Content Complexity

8. Technology Platform

Appendix 2:

Features to Consider in a Blended Learning Management Solution

• Learning resource activities can include items such as live events, books, online courses, chat sessions, PowerPoint slides, word files, etc.

• Activities can include integrated Web meetings and can be scheduled from the interface, enabling the learner to map out a learning plan over time.

• Activities can include scheduled or mandatory discussion sessions with instructors, experts, or peers.

• Activity and results can be tracked. Reporting for all activities is enabled. Typically, it is important to know that a learner entered the activity and also important to have some measure of completion of this activity, with a score or other criteria.

• A “coach” role exists, which enables a subject matter expert or instructor to have special privileges to see all



learners, answer questions, post information available to all learners, and see scores and progress.

• Can the coaches be measured and tracked so you can tell how busy they are, how active they are, and perhaps what feedback they received?

• Manager certification: As with most LMSs, is there ability for the learning activities to be authorized by a manager? Is there the ability for the manager to “approve” completion of a step to enable the learner to progress?

• Graphical learning map. Is there a visual map that shows the various steps in the blended program that make it easy for the learner to know what to do next? Is there a calendar function that shows the learner how long each step should take and how long they have to complete the program?

• Learner profiles and presence awareness indicators: Does the system have a way for learners to see other learners who are online or in a discussion room? Is it easy for one learner to see the profile of another learner, coach, or instructor?

• Integrated chat, instant messaging: Is chat and instant messaging integrated so that learners can interact directly with each other, the coaches, and instructors without going to an outside system, such as Yahoo or MS messenger?

• Segment large numbers of learners into small groups with coaches: In a large program, it is vital to group learners into small sections, just like a physical course. Can the system group learners into small teams that may be facilitated by individual coaches? Can these teams be joined together for some activities and then separated for others?

• LMS integration supported: How much information from the blended learning system will be available in the LMS? Will the LMS be able to track completion? Scores? Progress at each major activity?

• License and hosted model: Is the system available in a hosted model or run internally? Typically hosted solutions are more cost effective for medium-sized organizations.



About Us

Bersin & Associates is the only research and consulting firm solely focused on What Works® enterprise learning research. With more than 20 years of experience in e-learning, training, and enterprise technology, Bersin & Associates provides a wide range of services, including market research, best practices, benchmarking, vendor and product analysis, corporate workshops, corporate implementation plans, and sales and marketing programs. Some of Bersin & Associates innovations include a complete methodology for Learning Management Systems (LMS) selection and application usage, an end-to-end architecture and solution for training analytics, and one of the industry’s largest research studies on blended learning implementations.

Bersin & Associates offers the industry's first enterprise learning research subscription, “The Enterprise Learning Research Center” (http://www.elearningresearch.com), which offers up-to-date research, product selection guides, white papers, and access to other experts online. This service enables corporate training managers, vendors, and consultants to make faster, better decisions about enterprise learning strategy, programs, and technology. Bersin & Associates can be reached at http://www.bersin.com or at (510) 654-8500.

http://www.elearningresearch.com

http://www.bersin.com



About This Research

Copyright © 2005 Bersin & Associates. All rights reserved. What Works® and related names such as Rapid E-Learning: What Works® are registered trademarks of Bersin & Associates. No materials from this study can be duplicated, copied, republished, or re-used without written permission from Bersin & Associates. The information and forecasts contained in this report reflect the research and studied opinions of Bersin & Associates analysts.

Documents

blended-learning pm 3.1 · Josh Bersin January 2006 Bersin & Associates Blended Learning Program Management: WhatWorks Issues, Solutions, and Platforms for the Management of Blended