CI the Experimentation Imperative 2013

The Experimentation Imperative

Innovating at the Pace of Changing Customer Expectations

Frank W. Capek

Chief Experience Officer

THE EXPERIMENTATION IMPERATIVE © 2013 Customer Innovations. All Rights Reserved Page 2

Customers’ expectations are changing faster than your organization’s ability to innovate. While this assertion may not qualify as a universal truth, it’s becoming increasingly difficult to find organizations that are confident they’re ahead of the customer curve.

For example, the integration of smart phones, search and social media has made transparency a prerequisite and impatience a virtue. In the “always on” world, customers now expect to instantly access relevant and personalized information from anywhere, comment on their experience and get an immediate response (they’re assuming you’re watching and listening, always). This has led to unprecedented levels of empowerment and organizational change that are changing the game for traditional competitors across industries.

Most mature organizations can’t get out of their own way to keep up with market changes, much less keep ahead. If customer expectations evolve in 3-‐month cycles; any organization that responds to those expectations with an 18-‐ to 24-‐month innovation cycle is going to frustrate customers and become increasingly irrelevant.

Innovation Agility measures an organizations’ ability to sense and respond to changes in the market. Innovation Agility can be described in terms of the speed and effectiveness of four processes:

• OBSERVE changes in the customer and competitive environment in real time, including latent and emerging needs that indicate the trajectory of customer expectations

• ORIENT key stakeholders to what those changes mean; where necessary, aggressively challenge and revise outdated assumptions

• DECIDE on a series of alternative actions (often “small bets”) that allow the organization to move forward, test assumptions, and adapt the direction as new insight is gained

• ACT in a coordinated cross-‐functional manner that builds shared ownership and overcomes the unwritten rules that are embedded in the culture and reinforce the status quo.

These processes are adapted from a foundational theory of maneuver warfare known as the Boyd Cycle. Colonel John Boyd, a decorated fighter pilot that served in the Korean and Vietnam wars, described the process of executing OODA loops faster and more effectively than adversaries (i.e., “fast transients”), creating an environment where adversaries’ actions become irrelevant as they respond too slowly to changing conditions.

Similarly, any competitive business situation can be considered a race to see which competitors can effectively re-‐orient themselves to the rapidly changing customer priorities and, in doing so, outmaneuver their competitors. An organization that can OBSERVE – ORIENT – DECIDE – ACT faster and more effectively than their competitors earns the right to grow their business over time.

In order to get in front of the pace of change, leading companies are embracing a more intentionally iterative, experimentation approach to market sensing, concept development and learning. Failing quickly through well-‐defined experiments help companies learn faster and avoid large, costly mistakes at full launch. As a result, experimentation improves the quality, quantity and velocity of product, service and process innovation.


THE EXPERIMENTATION IMPERATIVE

Experimentation involves a structured series of actions and a focus on incremental learning related to market and technological developments as well as new products, services, processes, or business models. When done properly, the results of a series of experiments help the business, regardless of whether any individual experiment was a success or failure. The faster and more thoroughly an idea or change is tested, the faster and better it can be refined, scrapped, or implemented to deliver business value.

Experimentation enables organizations to gather and exploit insight about successes and possibilities, to reduce cycle time, and to involve customers in the design process. It helps remove some of the uncertainty associated with addressing complex, multi-‐variable, and changing problems. Whether a company is designing consumer products or launching new services, experimentation is a key to speed and success in the marketplace.

This is certainly not new. Tom Peters and Robert Waterman highlighted the importance of experimentation decades ago in their book, In Search of Excellence:

“The most important and visible outcropping of the action bias in the excellent companies is their willingness to try things out, to experiment. There is absolutely no magic in the experiment. It is simply a tiny completed action, a manageable test that helps you learn something, just as in high-‐school chemistry. But our experience has been that most big institutions have forgotten how to test and learn. They seem to prefer analysis and debate to trying something out, and they are paralyzed by fear of failure, however small.”

Experimentation is at the heart of any company’s ability to innovate. Virtually all innovative products or services start as an idea and are shaped into reality through experimentation. Major technological innovations may involve thousands of experiments that test whether the new solution is both technically feasible and addresses market demand.

Unfortunately, as organizations mature, they often lose the ability to experiment. This is influenced by several factors including a focus on maintaining production, ensuring short-‐term performance, career paths that emphasize predictability, and a diversification of roles and resources with few intimate connections to customers.

EXPERIMENTATION MUST BE AN INTEGRAL PART OF YOUR STRATEGY

Building the capacity for experimentation is not easy; it means fundamentally rethinking the role of failure in most organizations. Experimenting with many diverse ideas is critical for innovation. Experiments that quickly eliminate unfavorable options allow people to focus their efforts on the most promising alternatives. When important projects fail late in the game, the consequences can be devastating. For example, studies of systems development projects have shown that late-‐stage problems are more than 100 times as costly as early-‐stage ones. In projects that involve large capital investments, the increase in costs can be considerably higher. Removing the negative associations with failure requires overcoming deeply entrenched attitudes. When failure is associated with incompetence, that attitude can lead to highly counterproductive behavior.

For most organizations, adopting experimentation requires change in three areas: structure, capabilities and culture.

• Structure. It’s important to be intentional about adopting a well-‐defined process for experimentation. This involves redesigning or even revamping entrenched routines; building cross-‐functional teams that overcome organizational boundaries; and aligning funding and incentives that support and encourage a more experimental approach. Getting started often requires a few “trailblazer” projects that can be done in a protected environment. It often involves leveraging small groups of key and well-‐trained people and performing experiments in parallel when time is a key factor. It’s also critical to make sure achievements are


recognized, rewarded, and broadcast (including what was learned from failing). Ultimately, the biggest benefits occur when experimentation is integrated into the business’ core processes.

• Capabilities. Staff must be trained in the process of experimentation. It’s important to develop the skill sets required to identify, build, fund, run, and analyze experiments. It’s also critical to make sure participants and key stakeholders are clear on the process to be followed, the guiding principles required for success and the need to focus on learning and the integration of learning into subsequent experiments.

• Culture. Experimentation requires shared values including an appreciation of learning and an acceptance of failure. As long as an experiment contributes to the knowledge of the organization, it can’t be viewed as a failure. A culture that does not tolerate failure, embrace new ideas and value learning will find it difficult to experiment. Adopting an effective experimentation approach often requires leaders to surface and address deeply entrenched cultural barriers.

DEFINING AN EXPERIMENTATION PROCESS

Experimentation is both a process and a discipline used to develop systematic innovation and improvement. We define experimentation as a systematic, cost-‐effective, iterative approach to learning about the potential success or failure of a new product, service or process. This definition highlights a few important characteristics:

• Systematic. There is a well-‐defined process for confirming or refuting a specific set of learning objectives that are stated before the experiment begins. An experiment begins with a hypothesis and followed by a thoughtful design to test the parameters of that hypothesis. The process is monitored, and managed to achieve its objectives; it is not a serendipitous set of activities just to see what happens. The key differentiator of a business experiment from simple trial-‐and-‐error is the control and the design centered on the learning objective. It’s this focus on learning that helps keep the activities under control.

• Cost effective. Experiments make efficient use of resources, often much less expensively than traditional designs. For example, using simulations to stress-‐test cars is a much less costly way to study the design of a car than building the automobile and destroying it to learn the tolerances. Likewise, service processes can be simulated in a laboratory environment in a much less costly manner than actually building the process and running it at the throughput necessary to find the break points.

• Iterative. Lessons from the current iteration are incorporated into subsequent iterations. Either a “success” or a “failure” can result in learning. The experiment is only unsuccessful if there is no learning. “Failure” can be a loaded term for organizations unfamiliar with experimentation, typically because a failure is seen as a waste of resources.

ADDRESSING CULTURAL BARRIERS: UNWRITTEN RULES

Culture plays a pivotal role in how willing an organization is to use experimentation. Organizations that successfully employ experimentation have cultures that support this approach. Here are a few key characteristics:

• The organization must be able to accept, and subsequently learn from, failure. Though this thinking runs counter to that of many success-‐driven organizations, there must be room for trial and error, which may not produce the desired result.

• People must love to learn. The culture needs to be able to embrace learning and value the learning that is achieved via business experiments.

• Individuals and teams must be rewarded for thinking creatively and taking initiative. When an experiment results in some significant learning, people should celebrate. When an experiment fails to produce the innovation predicted, but learning still occurs, it should still be cause for celebration. At the same time, the organization should not tolerate sloppy work, which can result in wasted resources or unnecessary mistakes.


One of the most concrete ways to understand and address these cultural constraints is to focus on the unwritten rules that drive individual and organizational behavior. These unwritten rules are often inconsistent with innovation and experimentation. They also represent the sensible ways people have learned to survive and thrive in the organization. Surfacing unwritten rule barriers is critical to creating the conditions for successful experimentation. Addressing these rules involves shifting the management systems and structures that reinforce them.

COMMON CHALLENGES TO MAKING EXPERIMENTATION WORK

There are understandable challenges in adopting a more structured experimentation approach. Some of the most common include:

• Loading the experiment to get desired results. “Stacking the deck” to ensure a specific result is likely to undermine the viability of the deliverable and to taint the learning that comes from the experiment. We’ve found including participants with conflicting points of view in experiment design is valuable.

• Succumbing to pressure to succeed. Similar to “loading” an experiment, yielding to pressure to realize a certain outcome can undermine the results and taint the learning of the experiment.

• Refusing to accept the results. This most likely occurs when the experiment’s results are unanticipated or surprising, disprove a key hypothesis, or challenge cherished working assumptions of the business.

• Failing to secure support. There must be adequate support for the experiment by key stakeholders – both business and IT. Stakeholders need to understand and commit to the goals and process for the experiment.

• Picking the wrong experiments. If experimentation is new to your organization, choosing a mission-‐critical project may be the wrong place to start. Likewise, picking a project that has already committed to certain outcomes is a bad idea, since it violates a principle of experimentation – that the results can be thrown away.

• Picking the wrong location. If an experiment is done at the wrong location, the results may suffer from poor technical or logistical conditions or from customers not receptive to participating.

RECOMMENDATIONS FOR ADOPTING EXPERIMENTATION

If experimentation is new or immature in your organization, we urge you to start now and learn how to make experimentation work for your business. A few key steps to help get started include:

• Start with a few small business experiments to prove that learning occurs regardless of the outcome of the experiment, thus building momentum.

• Identify a senior executive who will champion the experiments and help remove roadblocks that “nonbelievers” may put in the way.

• Identify a location and business unit that is on board and interested in the experiment’s learning objectives. Don’t start in a hostile environment.

• Fund the project using a stage model. Build stages based on a set of learning objectives, and fund the experiment to the next stage as long as the learning objectives are being met.

• Make sure that those who embark on the early experiments are not penalized for taking the risk. Make sure there are incentives in place to reward success, including in the form of learning.

If you need help adopting or improving the effectiveness of your business experimentation efforts, please let us know. We can be reached at: [email protected]

Documents

CI the Experimentation Imperative 2013