test & learn goes mainstream' (iq, fall 2013) pdf

The Innovation Quarterly (Fall 2013) | 1

‘Test & Learn’ Goes Mainstream

By Brian Christian

uch of the strategic innovation work that Inovo has done over the years has been at the front end of innovation, or in what we call the Discovery phase. Oftentimes, having helped our clients

to discover a set of attractive opportunities, we are asked to stick around and assist with post-‐Discovery phase work. For strategic innovations, the post-‐Discovery phases are commonly referred to as Incubation and Commercialization. Recently, several of our clients have inquired about an approach to Incubation and Commercialization called ‘Test & Learn.’ In this Innovation Quarterly article, we would like to offer our perspective on the emerging Test & Learn movement.

Before we dive in, it may be helpful to remind our readers what we mean by strategic innovation. (For a more detailed understanding, please see our white paper on strategic innovation at www.theinovogroup.com.) Others refer to this type of innovation as breakthrough, disruptive or radical. At Inovo, we prefer the descriptor ‘strategic’ because it connotes the degree of long-‐term importance this type of innovation has internally for the innovating organization. Not all strategic innovations are radical or disruptive and neither do they need to be. Others (such as Govindarajan and Trimble in 10 Rules for Strategic Innovators) have also chosen the label ‘strategic innovation,’ and the world has come to recognize that strategic innovation requires a very different set of principles, methods and skills than does the more incremental, sustaining type of innovation. It is also appropriate to view a strategic innovation as the large company version of a startup.

Chief among the proponents of Test & Learn are Steve Blank (‘Customer Development’), Eric Ries (‘The Lean Startup’), and Alexander Osterwalder (‘Business Model Generation’). For the purposes of this article, let’s call their collective methodologies the Test & Learn Approach, or TLA. TLA goes something like this: The startup entrepreneur has a vision of a specific innovative offering and/or business model in mind, as well as an approach for achieving that vision. The precise manifestation of

M


the actual offering and its related business model emerges through an iterative process of testing with potential customers in an approximation of the eventual marketplace. The entrepreneur learns from these tests and adapts (or ‘pivots’) as necessary to optimize the ‘product/market fit.’ In essence, each test is aimed at one or more assumptions or hypotheses, and the startup learning journey will succeed or fail according to how well one tests these assumptions. The entrepreneur does not declare failure until the vision has been proven wrong – or the clock or funding runs out. This is contrasted against the traditional startup approach in which more thorough planning and analysis are conducted with the expectation that the first version tested in the market is essentially the right one. In the highly uncertain world of startups, this traditional approach often leads to disappointment, missed expectations and investor fatigue.

To provide a deeper perspective on TLA, let’s conduct a brief ‘Lean’ primer. One of the foundational principles underlying Lean Thinking is the elimination of waste (muda in Japanese). The original application of Lean Thinking at Toyota was in manufacturing. In Lean Manufacturing, waste is reduced with a variety of specific methods and tools such as value stream mapping, kanban inventory control, and cellular manufacturing. The TLA proponents point to Lean Manufacturing as the inspiration for their thinking. For some reason, they have chosen not to cite Lean Product Development (as explored by Kennedy, Liker, et al.), which is a more recent application of Lean Thinking. This is also sometimes referred to as Lean Design or Lean Innovation.

In Lean Product Development, waste is reduced with methods such as customer-‐defined value and set-‐based concurrent engineering. In the Lean set-‐based approach, multiple design options (a set) are tested in parallel, and options are closed out only as knowledge is gained and assumptions are confirmed or refuted. Eric Ries calls this ‘validated learning.’ In Lean Product Development, testing occurs according to the iterative Plan-‐Do-‐Check-‐Act or Shewhart/Deming cycle. Ries has labeled this the Build-‐Measure-‐Learn cycle. In Lean Product Development, a critical means for eliminating waste is to include end user or customer testing in the PDCA cycle (customer-‐defined value) to ensure design fit to purpose. Steve Blank calls this Customer Development. As you can see, the foundation for TLA is more directly found in the principles and methodologies of Lean Product Development than in those of Lean Manufacturing.

While waste elimination would not seem to be an essential element of a strategic innovation process, in fact it is quite critical. Its value is less in cost reduction in the normal operational sense and more in the conservation of precious money and people resources allocated to strategic innovation projects. When confronted with a vast white space of opportunity, an organization must be able to discover and exploit its hidden opportunities within restrictive money and people constraints. This would not be possible without a Lean set-‐based approach. Also, as these opportunities are often new to the world and/or new to the company, uncertainty remains high right through to commercialization of the new offering or business model. The greater the degree of uncertainty, the more important is a


Lean set-‐based approach. This is in contrast to highly predictable situations typical of sustaining innovations, for which the traditional linear stage-‐gate model is a reasonable approach.

The proponents of TLA have smartly recognized the relevance of Lean Thinking (as addressed by Ohno, Womack, et al.) for startups and entrepreneurs. By doing so, they have broadened the application of Lean Thinking and made it understandable to an audience that probably has not been much exposed to Lean concepts. In addition, through Osterwalder’s contributions, they have broadened the application of Lean beyond the offering to encompass the entire business model.

In addition, the TLA proponents have asserted that the mindset and methodologies, originally developed for startups, are translatable to the world of large-‐company strategic innovation. As Blank stated in an article in the May 2013 edition of the Harvard Business Review, “…despite the methodology’s name [‘The Lean Startup’], in the long term some of its biggest payoffs may be gained by the large companies that embrace it.” We agree with this assertion and are happy to see that large companies are starting to adopt and adapt TLA principles and methods for Incubation and Commercialization.

However, there are also some caveats to be made, in our opinion, regarding the application of TLA for large-‐company strategic innovation.

1. Business model diversity: As we move from intangible products distributed through virtual channels to tangible products distributed through physical channels, the methods and tools required to support TLA vary greatly. In fact, business models are becoming increasingly complex with a blend of intangible, tangible, virtual and physical. The strategic innovator must be able to select the right methods and tools for each situation from a vast array of options.

2. Large company complexity: TLA emerged as a framework for startups, either VC-‐funded or bootstrapped. Startups are relatively simple organizations. In translating TLA to large-‐company strategic innovation, it is important to complement TLA with other methods and tools that help to manage a complex process in a complex organization.

The first caveat relates to the vast difference across business model types and how this affects the

way that one designs and conducts a TLA program. Consider the differences between the Incubation and Commercialization of a mobile device gaming app like Fruit Ninja and that of a human drug like Lipitor, or that of a new commercial jetliner like the Boeing 787 Dreamliner. As business models move from intangible to tangible offerings and from virtual to physical channels, the types of methods and tools to be used to support a TLA program differ greatly.


In addition, how these methods and tools are assembled into an overall TLA program design will vary greatly across these diverse business model types. Here are some of the factors that vary across business model type and which have an important impact on the design of a TLA program for Incubation and Commercialization.

1. Degree of invention: When the required invention level is high, iterations on the product development side take a long time. However, market-‐side assumption testing need not come to a halt during these periods. Valuable input into the invention process can be gained while waiting for the minimum viable product (MVP) or a more advanced prototype. The ecosystem that determines adoption is much broader than the customer or end user, and valuable information about the ecosystem (e.g., regulatory constraints) can be gained during these extended invention periods. Finally, the expected invention time can often be short-‐circuited by scanning the landscape for available technologies or technology development partners. Recently, Inovo helped a large CPG client (which typically spends years developing its strategic innovations) to identify a material technology that significantly advanced the client’s development of a very promising new product. (This high-‐potential offering is launching in the marketplace this month, so we will share more about it in the near future.)

2. Cost of prototype construction: When the cost of prototype construction is high, as with complex systems like electric vehicles, it is often practical to gain customer and ecosystem feedback in advance of a full system investment to reduce investment risk. In these situations, it can make sense to develop in silico models of adoption to correlate design variations to customer and ecosystem needs and desires. A few years


ago, Inovo built an adoption model for the electric vehicle market to support the product design efforts of an electric vehicle battery manufacturer. It is also possible to test individual elements of a complex system and piece together the market responses to build a complete picture. This has obvious risks but can be superior to the alternative of taking one large risk with a complete system prototype. This is not so much a minimum viable prototype as a set of minimum viable subsystems. You can be sure that for its Dreamliner, Boeing was market-‐testing seat design with passengers, cockpit design with pilots, galley design with flight attendants, and cargo bay design with baggage handlers all while developing and testing the aircraft itself for flight worthiness.

3. Cost of in-‐market testing: Sometimes in-‐market testing is difficult, costly or even impossible. This is true in many business-‐to-‐business situations as well as in the life sciences. In these situations, it is often practical and sometimes necessary to alleviate investment risk by getting end user and broader ecosystem input in lieu of a minimum viable product. Several years ago, Inovo helped a Fortune 500 client test assumptions related to the use of a proprietary plastic composite to replace wooden railroad ties. By engaging with experts across the railroad industry ecosystem, we were able to help the client discover that several large railway operators were potentially interested in this new composite offering for certain use cases. More importantly, however, we also helped them discover that their proposed offering was insufficient along a key performance dimension and therefore would not be adopted. The client quickly made the decision to cease its product development efforts, and we were told that this saved them a large investment.

One effective way to manage the risk associated with the three factors discussed above is to seek financial, product development, and/or commercialization partners who can share the risk and perhaps quicken one’s time to market. For example, Inovo is just winding down a project in which we are helping a carbon fiber part manufacturer build an entirely new business in the automotive sector by identifying, screening and developing partnerships with OEMs and Tier 1 suppliers who are well positioned to co-‐develop their first trial offerings.

The second caveat regarding the translation of TLA to the world of corporate strategic innovation relates to the real cultural and business process differences between a true startup (VC-‐funded or bootstrapped) and a large, established company. As Blank likes to say, “A startup is not a small version of a big company.” Here are some important factors that explain why TLA alone may not be adequate to the task of large-‐company strategic innovation.

1. Coordination: Startups are simple organizations with few layers and shallow functional divisions. Large companies are complex structures with many layers and


deep functionalization. Orchestrating a ‘startup’ activity within a large company is a complicated task, and doing it quickly enough to achieve adequate speed-‐to-‐market is a significant challenge. Inovo has found that constructing cross-‐functional and multi-‐level teams with clear accountabilities and active involvement is the key, but it does require a lot of effort and patience to manage this team structure.

2. Decision-‐making: Startups have one or few decision-‐makers, and they are well aligned around a simple goal. Large companies have many decision-‐makers, often in a matrix consensus structure, and goals are not always well aligned. Decisions to adjust, pivot or proceed within a large company are difficult to make effectively while maintaining alignment. At Inovo, we have found that the use of online voting tools to gather opinions and preferences in advance of decision-‐making events are a great way to cut through politics and posturing and also to broaden the conversation. We believe that for strategic innovation, the reliance on the judgment of senior executives or scientific gurus is dangerous, because their intuition is derived from the existing business. It helps to apply decision meeting facilitation tools that interrupt the normal way in which administrative or technical seniority tends to bias the outcome of conversations.

3. Portfolio management: Startups generally have one offering and one business model to manage. Large companies have many offerings and possibly multiple business models. Large companies are not just making decisions about one opportunity but must make ‘go/no go’ decisions across multiple opportunities on an ongoing basis. Inovo uses an objective assessment tool to rate opportunities along two axes – ‘should’ we do it and ‘could’ we do it – to assist in making portfolio decisions across multiple opportunities. We also use opportunity canvases to monitor and maintain a healthy distribution of opportunities across the white-‐space domains that we are exploring.

As with the business model caveat, this organizational caveat does not invalidate the application of TLA for large-‐company strategic innovation. However, it does mean that TLA alone is probably inadequate.

In summary, having a Lean set-‐based approach such as TLA is an important component of a large-‐company strategic innovation capability, but it is not sufficient. It requires an appreciation for the wide variety of business model types that exist in the world and the need to tailor one’s Test & Learn program with the proper methods and tools. It also requires additional methods and tools to allow TLA to be effective in a complex, large-‐company setting.

About The Inovo Group

Founded in 2001, Inovo is an innovation consulting firm based in Ann Arbor, Michigan, that helps the world’s leading organizations succeed at strategic innovation.

About the Author

BRIAN CHRISTIAN is President of The Inovo Group. He has 30 years of diverse business experience in product development, marketing, sourcing, and information technology across multiple industries and geographies. The common theme throughout his career has been the development of new technologies, new products, and new capabilities.

Prior to the formation of The Inovo Group, Brian was founder and president of DASO, a consulting firm with a mission to help companies build innovation culture and capabilities. Brian also worked for Whirlpool Corporation for eleven years in multiple management positions, the last five years as Vice President of Global Product Development. Earlier in his career, he worked for the management consulting firm Booz & Company in their Chicago office.

Brian holds an MBA from the University of Chicago and a Chemical Engineering degree from the University of Michigan.

Copyright © 2013 by The Inovo Group, LLC All rights reserved.

www.TheInovoGroup.com