Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  1  

INTRODUCING AGILE CUSTOMER-CENTERED DEVELOPMENT IN A LEGACY SOFTWARE PRODUCT LINE

JAN BOSCH & PETRA M. BOSCH-SIJTSEMA

Abstract. The ability to rapidly respond to customer interest and to effectively prioritize development effort has been a long-standing challenge for mass-market software intensive products. This problem is exacerbated in the context of software product lines as functionality may easily fall over software asset and organizational boundaries with consequent losses in efficiency and nimbleness. Some companies facing these problems in their product line respond with a new development process. In this article we discuss the developments made from a single case study, Intuit’s Quickbooks product line that combined agile software development, design thinking and self-organizing teams in a successful approach that provided a significant improvement in terms of responsiveness and accuracy of building customer value. Keywords: software product lines, agile software development, compositional software engineering, design thinking

1 Setting the Context Software product lines (SPLs) present the most successful approach to intra-organizational reuse and many companies have managed to establish or expand their market leadership, often by disrupting competitors, through the successful adoption of a SPL [1, 2]. Although SPLs can be planned, often the company starts off with a single wildly successful product that then grows into a family of products. Over time, however, software product lines may become victims of their own success and start to suffer from a number of challenges, including high coordination cost, slow release cycles and high system-level error density [3].

The cause of these challenges is what we refer to as integration-oriented software engineering, i.e. treating the entire software product line as one, often large, tightly coupled set of software assets that require virtually continuous coordination between development teams and a sizeable central organization. Starting with the road mapping and planning stage, through requirements management, development, testing and, finally, integration, teams need to interact, adjust to each other’s interfaces and changes in functionality behind those interfaces. This approach assumes a slow release cycle, often yearly and/or even slowly, as the cost and unpredictability of the integration phase is so high that the organization naturally seeks to minimize the pain associated with that stage of development [cf. 4].

The typical answer to the above is introduction of an agile development approach [5]. Although agile development approaches are very valuable, these are not sufficient in the context of large-scale software development in software-intensive product companies [6]. The scale of development often requires decoupling mechanisms that further decrease the dependencies between teams, such as independent deployment techniques that allow teams to release their code independently of other teams. In the case of software-intensive product companies, actively involving its customers, which may range in the millions, in the prioritization and implementation of functionality is challenging as any direct customer input is a small sample that may easily be biased whereas any

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  2  

accumulated, averaged customer data is often colored by product management’s assumptions about the typical customer.

The topic and contribution of this paper is twofold. First, we present the main issues of SPL and Agile methods for large-scale software product line engineering. Second, we present a novel approach to software product line engineering that encompasses agile, design thinking and self-organization principles and illustrate the approach by means of a company case study.

The paper is organized as follows. In section two we present the theoretical background why compositional product line engineering matters. After that we present the research methodology applied for collecting data in a single case study. In section four we discuss the case study in which this approach has been applied. Section five discusses a novel approach to software product line development combining elements from agile, design thinking and self-organization and present the specific implementation of this approach in the case study. The paper is concluded with a discussion and conclusion section.

2 Why Compositional Product Line Engineering Matters The topic addressed in this paper is pervasive and touches many aspects of software product line development. As a consequence, the problem statement that forms the fundament for the paper also has multiple dimensions. Specifically, we focus on three aspects, i.e. software product lines, agile software development and organizing for large-scale software development.

2.1 Software Product Lines Software product lines present the single most successful approach to intra-organizational reuse of software assets [1, 2, 7]. As we discussed in the introduction, companies have established or expanded market leadership positions in a variety of industries through the successful introduction of a software product line. However, there are inherent risks associated with successful product lines that may cause the company to turn the initial advantage into a liability. This liability expresses itself [3] in three symptoms:

1. Coordination overhead: Successful software product lines naturally increase the scope of functionality as well as products over time. This can occur both in an organic fashion, through the introduction of new products, or inorganically through mergers and acquisitions or reorganizations where previously unrelated products are brought into the software product line. With increasing scope of the software product line and the significant increase in the number of people working on it, the cost of coordinating the efforts of teams, individuals, product derivation efforts, road mapping, integration of the platform, etc. increases exponentially to the point that most staff spends most of its time coordinating through meetings, email exchange or other mechanisms and has rapidly decreasing amounts of time dedicated to adding value to the products and shared assets in the software product line.

2. Slow release cycles: In case studies presented in earlier research we described that the shared part of the software product line, i.e. the platform, was integrated and released periodically to the product teams that derived new or evolved

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  3  

existing products built on top of the platform [4, 6]. With the increasing size and complexity of the overall SPL as well as the platform, the cost of verifying all functional and quality requirements tends to become very high and as a consequence, the release frequency to decreases and slows to a point that in some companies was below the “speed of the market”. This caused the benefit of the SPL to turn into a liability as the SPL caused delays in product releases.

3. High system-level error density: One of the main reasons for slow release cycles was that many errors were only found during the integration stage as the complexity of the platform and the SPL as a whole had reached a point where teams and their architects were unable to predict the implications of their design decisions and extensions on the overall system. Hence, the negative implications were found late in the development cycle and could cause time-consuming rework in the various components. This caused the integration stage to be long and painful, putting a strong pressure on the organization to perform integration as infrequent as possible, reinforcing the aforementioned problem of slow release cycles.

Although the problems of large, legacy software product lines discussed above are more serious and complex in a product line context, large-scale software engineering in general is suffering from the above problems as well as from other. In response to this, the last decade has seen an increasing and broad adoption of agile software development. The next section discusses the challenges associated with agile approaches in large-scale, legacy software development for product companies.

2.2 Agile Software Development Over the last decade, several agile software development process approaches have been developed, including XP [8], lean software development [9] and scrum [10, 11]. In the context of smaller scale software development projects, agile development projects [12] have shown significant success.

Inspired by the agile approaches, especially for web applications and services, software teams now focus on small team size, short release cycles, ranging from weeks to several times per day, and experimentation in the market place, i.e. the notion of perpetual beta. Agile development has been widely documented [5, 8] as working well for small (<10 developers) co-located teams. From agile software literature it becomes clear that agile teams work mainly co-located, have frequent face-to-face contact and highly motivated team members work in self-organized teams. Techniques such as pair-wise programming, daily standup meetings and sprint planning meetings are relying to a large extent on the team being co-located. As a consequence, agile development has shown success especially in smaller software development projects. Recently more researcher are studying product line complemented by agile methods in software development [13, 14] Specifically for large-scale software development in product companies, there are specific challenges in using agile development methods:

• Customer representation: Companies serving mass markets typically have a product management function to represent the customer internally. However, although product management may be able to identify what features are relevant

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  4  

to the customer, providing specific input about the specific implementation of new functionality is often something that product development is left with on its own. Agile methods assume active customer involvement as a key ingredient to make sure that the right functionality gets built. In the case of product companies, alternative mechanisms are required to make sure that the customer is sufficiently involved in the development process. This problem is exacerbated in the context of a software product line as there are multiple products, serving different segments of the market, as well as a platform used by customers only indirectly.

• Architecture evolution: Several authors [e.g., 15] have studied the relationship between architecture and agile development, but most of this work has focused on the upfront efforts around architecture. Agile methods focus on the notion of architecture refactoring. Local refactoring works well independent of scale. However, especially global refactoring where adding, changing or removing a design decision has a system-wide effect is very complicated and goes against the culture that agile teams seek to build.

• Managing scale: Finally, as an overall concern, the scale of the code base and the associated development organization requires operating mechanisms that are not covered and may even conflict with agile development methods. Agile methods are best suited to projects of modest to moderate size [5, 16].

2.3 Organizing For Large-Scale Software Development One of the topics that has received more in the software engineering research community in recent years, is how one organizes large-scale software development where tens or hundreds of engineers all work on a multi-million line code base.

Most organizations the authors have worked with or for in the past use the notion of component teams. Using the software architecture of the system as a blueprint, teams of engineers, possibly extended with an architect, product manager and/or designer, are responsible for a component, maintain its quality and extends its functionality in response to requests from component teams whose components depend on this component. The component team approach works very well in small to medium sized software systems, but is the cause of several of the problems discussed earlier in this section. In particular, using component teams as the organizing principle may, among others, cause the following challenges:

• Coordination overhead: A customer-facing feature is often broken down into new requirements on several components. This requires the teams to coordinate the precise implementation and interface changes as well as the timing and sequencing of the implementation of these requirements in order to maintain system stability. In addition, the teams jointly work to build and integrate the overall system periodically and need to coordinate their work, often related to bug fixing in response to interface mismatches and incorrect assumptions about the implementation by other teams. Consequently, teams need to connect both with the central development organization and with other teams during the requirements stage, the development stage as well as the integration stage and the amount of overhead associated with this easily becomes staggering.

• Disconnect with customer: As teams are focused on their component, few team members have a good understanding of the customer’s use of the overall product

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  5  

and their role in delivering an optimal experience for the customer. This inward focus easily leads to disconnection and points of stress in the organization between the customer facing product managers and designers and the engineers that are responsible for maintaining the quality of their component.

• Slow response times: The breakdown of a customer-facing feature into requirements for multiple components in the system combined with the lack of a holistic understanding by many of the engineers, results in a slow response time for software development. The feature is not finalized and ready for shipping to the customer until all teams have developed their functionality and the integration of that functionality has been completed successfully. Mismatches in sequencing, misunderstandings between teams, etc. easily cause delays and unpredictability.

The above challenges cause organizations to put strict, top-down process in place in order to cope with the lack of predictability. Unfortunately, this will only increase the overhead in the system.

As we discuss later in the paper, there is an alternative organization principle, i.e. the use of feature teams that addresses these concerns. Although it does introduce its own set of challenges, especially in the case of large-scale development, this approach can have fewer evils associated with it.

3 Research Methodology We performed a single case study [17] in which we held 14 semi-structured interviews with participants of Intuit, a Fortune 1000 company, who were part of the development process. These interviewees were on levels from senior management, project management to senior software engineers. All interviews were taped and transcribed and we collected secondary data in terms of documentation, presentations, and publications around the development process. One of the authors was a participant of the organization (complete participation method of Adler and Adler [18]) and could therefore gain insight in the respondent’s world. Understanding the setting is of critical importance in establishing research relationships, trust, rapport and in evaluating both the information obtained and the interaction that occurs with in-depth interviews [19]. In order to increase validation of the interview data, the data was coded by both authors separately and compared. Furthermore, the data analysis and results were presented to the case company (respondent validation).

The interviews and secondary data were coded, labeled and categorized iteratively through systematic stages [20] and we found several themes. These themes are discussed in the remainder of the paper.

4 Intuit Quickbooks Product Line Intuit is Fortune 1000 software product and services company that serves consumers, small businesses, accountants, financial institutions and healthcare providers with primarily finance and accounting related solutions. Intuit is organized in 10 business units that address different customer segments or provide solutions in specific domains. The small business group consists of four business units. The largest business unit, Financial Management Solutions, owns a flagship product line, Quickbooks that serves

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  6  

small businesses with an accounting solution. The other three business units focus on payroll, payments and customer management, and to a large extent operate in the Quickbooks ecosystem.

The product line Quickbooks is market leader in its domain in the United States with a market share of more than 75%. The product line supports hundreds of use cases and offers a very rich set of functionality that serves small businesses in a host of industries and ranging from no-employee businesses to businesses with tens of millions on revenue and dozens of employees.

Although Quickbooks is often presented as a product, it is a software product line with several products: Simple Start, Pro, Premier, Quickbooks Canada, Quickbooks UK, Payroll, Point of Sale, Mac and Online. Further variability is provided by support for specific types of businesses and industries. The product line is largely organized as a configurable product base [Van der Linden et al., 2004], meaning that the different products can be automatically derived. However, for some of the products, the platform code is manually integrated with product specific functionality. The Quickbooks product line also forms the basis for a rich software ecosystem consisting of thousands of independent solutions providers (ISVs) as well as application developers that build and sell functionality extending Quickbooks for specific domains.

Up to now, Quickbooks employed a yearly release schedule where next year’s version is released in the September timeframe. Development was geared towards this release date and could roughly be divided in three periods, i.e. feature selection, development and integration, including quality assurance.

The size of the code base for the main set of products in the product line is between 5 and 10 MLOC, and the oldest part of the code is 20 years old. The size of the development organization has fluctuated significantly over the years, but is currently around 100 people divided over multiple locations in the US and India.

Although significant effort has been put into architecture refactoring, the code base has a significant level of design erosion that complicates the introduction of significant new functionality.

4.1 Product-line challenges Although not all the symptoms described above in section 2.1 (coordination overhead, slow release cycle and high system level density) were present at the case study presented in this paper, there were several product line related problems that needed to be addressed. The interview data categorizes the problems and issues with the original approach into four main areas:

• Lack of customer feedback: The original development approach assumed a yearly release cycle where during the fall the features to be added were selected, development took place in the January-June timeframe, integration and testing started in June and ran through August and the products were released in September. As a consequence, the product managers and developers involved in a selected feature would only get customer feedback on the feature and its implementation after the release of the product. Furthermore, customer feedback was difficult to incorporate as the development organization had already committed to a new set of features for next year’s release by the time the feedback became available. 23% of the interviewees raised the lack of customer

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  7  

involvement, influence and feedback as a key disadvantage of the original development approach.

• Heavy, top-down development process: Due to the complications of the integration phase in terms of required effort as well as its unpredictability, over the years more and more processes and process steps were added, resulting in a heavy, top-down process that left engineers feeling disempowered and significantly decreased the efficiency of development due to all the checks and balances. Of the interviewees, 52% raised the development process as a key inhibitor for more efficient and effective development as well as innovation.

• Inefficient use of engineering resources: Due to the process, development effectively took place only during 50% of the year. During the fall, the engineers were recovering from a very intense integration phase, did some minor refactoring and fixed and released bugs. During the integration stage, engineers were primarily concerned with bug fixing and adjusting their components to other, dependent components. Although all engineers were very busy, the efficiency of work was very low. Of the interviewees, 18% considered the inefficient use of resources to be a key challenge with the original development approach.

• Low engagement of team members: The lack of contact with customers, the constraining process and the inefficiency of resource use negatively affected engagement and motivation of the engineering population. The engineers felt constrained and not in control of their work as they lacked autonomy. As a consequence, initiative by engineers decreased, resulting in lower productivity and quality. Of the interviewees, 7% raised engagement as a key challenge of the original development approach.

Although it was clear that the original development process had reached the end of its useful life, there were some positive aspects of it that were mentioned by some the interviewees. First, the process had been in place for many years and consequently the whole organization knew it inside out. Second, some engineers appreciated knowing all functionality that had to be developed at the beginning of the development effort. Finally, as the development organization was organized around component teams, there was significant expertise that had been accumulated in key individuals that acted as a resource for the component teams.

5 Compositional Product Line Engineering The goal and contribution of this paper is to present a novel approach to large-scale software product line engineering. This approach combines elements from agile, design thinking and self-selecting, -directing, and –managing teams to achieve customer-focused development by small, independent teams. We refer to this approach as compositional product line engineering as the intention is to the largest extent to facilitate teams to (semi-) automatically insert the fruits from their work into the code baseline when certain milestones have been met The approach consists of two iterative processes, i.e. the product release process where all contributions by the various teams are combined, tested and released to customers and the concept development process where functionality identified as important by customer is developed in an interactive fashion. Figure 1 gives an overview of the new process.

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  8  

--------------------------------- Insert figure 1 about here

--------------------------------- The remainder of this section is organized as follows. First, we introduce the concept development process of the team. Then we describe the product release process for the organization. Finally, we summarize the approach and discuss the key benefits of the compositional product line engineering approach for the case organization.

5.1 Concept Development Process The concept development process of the software team is focused on developing new functionality that is prioritized by customers and implemented in a way that customers care about. The concept development process consists of four main phases, i.e. the solution jam, the code jam, development iterations and, finally, solution validation.

5.1.1 The solution jam The solution jam is organized at the product development organization level. It is a full-day event where 10-15 customers are invited and all PM and PD staff currently not full-time assigned to development projects attends. Also, senior PM and PD management is present. In total, somewhere between 20 and 50 Intuit staff attend the event.

The customers invited to the solution jam are selected based on their potential interest in the areas addressed by the customer pain statements. For instance, if a customer’s pain statement addresses support for multi-location inventory management customers are selected whose business has multiple locations.

Before the solution jam, the coordinator connects with individuals and already created teams to facilitate the formulation of “customer pain” statements, i.e. hypotheses about areas that customers would prioritize development to take place in. These customer pain statements are quite concrete and specific and are used as the starting point for conversations and reviews with the customers present at the solution jam.

Teams can be formed before as well as during the solution jam. Feature teams are formed that are see their own copy of the code base line allowing them to make changes necessary for implementing their feature, anywhere in the system. The team formation is completely bottom-up and is not influenced by management. The average size of the team is three members, but it can fluctuate from a single individual to seven people. It is important to note that the teams are self-selected, i.e. management does not influence their formation, self-directed, i.e. the team selects the customer pain that it wants to work and are self-managed, i.e., there is no standard division of tasks. The latter is interesting as team leadership can land with individuals in different roles and may even change over the course of the project as different skills become predominant.

During the solution jam, the team fleshes out the customer pain statement and starts to design a solution. This solution is not implemented in code, but rather is expressed in terms of “mock-ups” and sketches and complemented with verbal commentary. Once a first rough draft design has been created, it is presented to a few of the attending customers for feedback. Based on the feedback, the team continues to iterate the design.

The customer feedback at this early stage in the process is invaluable to refocus teams and to make sure that teams are prioritizing the right aspects of the solution.

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  9  

Although Intuit engineers collectively spend around 10.000 hours per year in “Follow Me Home” activities where customers are followed and observed in their daily routine (a form of ethnography), product developers still do not have the same level of understanding of the needs and wishes of customer than the customers themselves. Receiving customer feedback at the very start weeds out the well-intended but ill-conceived hypotheses of customer “pains” before any significant investment is made. The second reason is that even if the customer pain is correctly identified and significant, engineers may still focus on the wrong aspects initially rather than fleshing out the design of the most important aspects. Immediate feedback and input from customers will cause the engineers to focus on the right aspects of a solution immediately.

During the day, the team iterates on their concept, receiving input and feedback from customers. During the final hour, all teams get the opportunity to present for a panel consisting of engineering and product management as well as customers. The panel decides which of the presented concepts are considered to be the most viable. Typically, the panel accepts around 50% of the concepts to proceed to the next stage.

5.1.2 Code Jam The next stage in the development process is the code jam. Often organized in the same week, with perhaps a day in between, a 2-day code jam is organized where the teams with accepted concepts from the solution jam get the opportunity to build a skeleton implementation of the solution.

As discussed earlier, the Quickbooks code base is around 2 decades old and hence significant design erosion has occurred over the years. This has caused the creation of a number of beliefs in the development organization about what things are hard to change and which are easier. Rather than relying on the beliefs in the organization, the code jams validate the technical complexity through the implementation of a skeleton solution. The skeleton provides a partial implementation of the most basic functionality while focusing on assessing the highest risk areas of the implementation.

The skeleton implementation provides deep insight into the complexities associated with a full implementation of the concept and highlights the key areas where difficulties can be expected. The skeleton implementations are also used for taking a first peek at what the solution will look like for the customer. Even if a concept is visualized during the solution jam using mock-ups, seeing the implementation in software is still significantly more informative.

The code jam ends with a panel presentation from each of the participating teams. The panel consists of product and engineering management, and assesses the estimated value of each of the concepts as well as the predicted cost and complexity of implementation and integration. Based on this information, the panel selects the highest ROI (return on investment) concepts. In a typical code jam, the panel selects around half of the presented concepts.

5.1.3 First Development Iteration The teams whose concepts were accepted at the code jam receive the go-ahead for a first two-three week iteration. The team now has much more time than during a solution or code jam to start developing a proper implementation of the concept in the context of the Quickbooks product line. Depending on the target audience of the concept, it is either

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  10  

implemented directly in the platform or it is implemented in a specific Quickbooks product.

The purpose of the design iteration is to proof a specific hypothesis about the value and usability of the concept. As this obviously requires the feedback from customers, the team is expected to have collected customer feedback before the end of the iteration. The customer feedback is often collected through telephone meetings with customers where a screen-sharing tool is used to show customers the implementation of the concept in order to receive feedback.

At the end of the first iteration, the current implementation of the concept as well as the customer feedback is presented to product and engineering management. The decision at this point is three-fold. First, the team can get the go-ahead for a subsequent iteration. Second, the team can be stopped and dispersed if the results of the implementation and the customer feedback are not satisfactory. Finally, although unlikely at the end of the first iteration, the team may have finalized the implementation to the extent that the concept can be integrated in the Quickbooks code base. The acceptance rate for concepts is much higher once the team has been allowed to enter its first iteration, so in practice most teams proceed with implementation of the their concept until it complete.

5.1.4 Subsequent Development Iterations Once the team can proceed after the first iteration, the process is similar and repeats the described process. For every iteration, development takes place to test a specific hypothesis related to the concept with customers, customers provide feedback and at the end of the iteration, the results are presented to product and engineering management.

For larger and more complicated concepts, though, there is an additional mechanism for collecting customer feedback: the feature alpha. Although customers can provide a certain level of feedback through screen sharing and telephone conferences, the concept cannot be used in their day-to-day practice. For this purpose, the notion of a feature alpha is introduced. The feature alpha consists of the most recent stable and released version of Quickbooks extended with the concept implementation. As the feature alpha contains mature Quickbooks code, the customer can use the feature alpha in their day-to-day operations. The customer can also use the functionality associated with the concept, but with the understanding that the code is not yet of production quality and hence may experience stability issues.

The feature alpha allows the customer to provide feedback on the concept in the context of their real business. Also, customers are selected because of their specific interest in the functionality. For instance, to evaluate multi-site inventory functionality, customers are selected that already today have multiple sites where inventory is held and hence are keen to see that aspect of their business better supported by Intuit’s solution.

Once several iterations of have been invested in the concept, the concept will most likely be included in the product line. So, the typical exit point for the concept and the associated team is the end of an iteration where the concept has received positive customer feedback and is considered complete by product management. At this point, the concept is integrated into the product line and the team disbands and returns to the pool of engineering resources. Of course, the engineers immediately start to look for new

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  11  

“customer pains” to bring to the next solution jam and to form teams around ideas based on their passions and interest.

5.2 Product Release Process The previous section described the bottom-up part of the product development process. In the bottom-up process, the teams identify, present and development new concepts. There is also a top-down part of the process where product management identifies a small number of typically larger features that need to be developed in addition to the concepts identified as part of the bottom-up process. The engineering resources for the top-down part are selected and assigned by management, rather than following the principles of self-organization.

The top-down process, as described earlier, is how Quickbooks was developed earlier and one could assume that this is relic from earlier days. However, there is broad agreement in the organization that there is a need for both types of development in the context of Quickbooks in the future as well. The top-down development process has, however, been extended with customer feedback in different stages of the development.

As part of the overall push to collect more and earlier customer feedback, the development team creates several product alphas during the year. A product alpha is similar to a feature alpha, except that it combines several features and concepts into one release that is distributed to selected customers. As a product alpha contains so many new features, it is significantly less stable than feature alpha and hence customers in their daily operations cannot use the release. Instead, the product alpha is released embedded in a virtual machine (to avoid installation issues) and operates on a copy of the customer’s Quickbooks data file. In this fashion, the customer can test the product alpha in the context of his or her own business without risking corruption of the business data.

At this point, the Quickbooks desktop versions are released once per year. As a consequence, towards the end of the development cycle more and more resources are transitioned from development to testing, validation and bug fixing. As part of the future evolution of Quickbooks, the intention is to release new functionality quarterly or monthly. The capabilities for achieving this high a frequency of releasing production quality code are being developed as part of the transition to the new development model.

5.3 Benefits of the new approach From the interviews and secondary material from the case study we found a number of benefits for the new development approach. These benefits are discussed below. • Customer feedback: Throughout the new approach, the customer is deeply involved

in all aspects of the development, starting from the earliest ideation stages throughout the maturation of new functionality. The solution jam, customer feedback during development iterations, the feature alpha and the product alphas all contribute to continuous customer feedback. In the earlier stages, the customer actively helps shape the development of new functionality whereas in later stages the feedback is more focused on qualitative and quantitative feedback on what has been built. From the interviews 22% of the interviewees mentioned the importance of customer feedback. Especially the customer engagement and feedback was highly valuated. Other aspects mentioned during the interviews were the fact that customers were able to test or

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  12  

experiment features early on and many mentioned the positive responses from customers on the developed solutions as validation and motivation for their work.

• Development process overhead: The new process is heavily focused on minimizing the dependencies between different teams as well as between the central development organization and the development teams. This has resulted in a situation where teams have a great deal of autonomy between the milestones in the process, i.e. at the end of the solution jams and the code jams, at the end of each development iteration and throughout the integration of their code into the software baseline. A major supporting factor in this case is the transition from component teams to feature teams. As feature teams effectively have their own development code line in the SCM system, each team can make the necessary changes in all of the components without having to coordinate with other teams until the integration stage. The feature team solution was discussed during the interviews as empowering and motivating for team members.

• Use of engineering resources: Development of new functionality now takes place throughout the year, rather than just during the winter and early spring of the year. This means that development, QA and integration resources now to a large extent work in parallel rather than in a sequential fashion. In addition to significantly increasing the effective use engineering resources, it also prepares the organization for its transition to a continuous release model. In the interviews many interviewees mentioned the increase in speed (9% of interviewees) in development.

• Engagement: The amount of autonomy, the avoidance of a yearly peak during the traditional integration stage and the frequent customer interactions have significantly increased the engagement by engineers. Team members and managers mentioned during the interviews the increase in engagement and motivation of team members. Other aspects mentioned during the interviews were: team members can have ownership of features; work in cross-disciplinary, small and organic teams that can learn from each other.

In the next section, we evaluate the approach with respect to the key challenges identified for the original approach for software product line engineering in general.

6 Discussion and Conclusion In the sections above, we have introduced a novel approach to building large scale software in the context of software product lines that uses elements from design thinking, agile software development and self-organizing teams. The approach was developed in response to a number of key challenges that software product lines start to suffer from over time, i.e. high coordination overhead, slow release cycles and increasing error density.

In the case of the case study company, the generic problems in legacy software product lines translated themselves into four main challenges, i.e. the lack of customer feedback during development, a heavy, top-down process that had evolved in response to lack of predictability, this process had in turn resulted in inefficient use of engineering staff which negatively affected the engagement levels of engineers negatively. The new approach resulted in four important key elements:

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  13  

1. Self-selected, directed and managed teams: A major part of R&D is organized in small teams that self-select their members, self-direct (based on customer input) their work and that, to a large extent, manage themselves.

2. Continuous customer involvement: Customer involvement in this approach is raised to a new level. In most products, customers were especially used to provide feedback after development was complete. In this case, customers are involved throughout the development process.

3. Solution and code jams: One and two-day jams are an integral part of the development process and have a dedicated purpose, i.e. converting an idea into a real, partially validated, concept and exploring the complexities of implementing it in the legacy code base.

4. Feature alphas: The ability to present the customer with a stable version of the most recent release of a product in the software product line but extended with a specific feature that is under development has proven to be a particularly powerful customer feedback mechanism.

The topic and contribution of this paper is twofold. First, we presented the main issues of SPL and agile software development methods for large-scale software product line engineering. Second, we presented novel approach to software product line engineering that encompasses agile, design thinking and self-organization principles and illustrate the approach by means of a company case study.

Although the compositional software product line engineering approach presented in this paper has addressed the key issues that it intended to solve, there still are a number of areas of for future discussion and research. The compositional product line approach helps to deal with the development of large scale software through agile and design thinking principles, however, these process changes also have an important impact on the software development organization, the way its management is set up as well as a variety of other topics including human resourcing issues such as performance management.

One topic for future research is the long-term implications of a transition to self-selected, self-directed and self-managed teams. How are team members selected, having the right team members in your team, dealing with available people resources, and management expectations need to be studied more in the future. Furthermore, employee performance as well as managing the individual’s perception of adding significant value especially when being part of several solutions jams without being selected for subsequent iterations is an important topic of study.

Another important topic for future research is the management of architecture evolution and refactoring without sacrificing the principles of agility and self-managed teams. Agile methods often tend to de-prioritize architecture, leaving architecture erosion as a consequence. However, for compositional software product line engineering to be effective, the architecture plays a central role.

The customer engagement and involvement early on in the development process has been very beneficial for this particular case study. However, an important aspect for future research is how software companies receive input from the right customers. Selecting the right customer for a particular concept or feature at the right time in order to optimally benefit from the customer input and validation is a complicated problem. Finally, in the context of large-scale software product line development for mass-

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  14  

markets, translating the input from a small selection of customers into accurate insights that apply to the entire customer base is potentially complicate problem of induction.

References [1] Bosch J. Design and Use of Software Architectures: Adopting and Evolving a Product

Line Approach. Addison-Wesley: London, 2000. [2] Clements P, Northrop L. Software Product Lines: Practices and Patterns. Addison-

Wesley: Boston, 2001. [3] Bosch J. Toward Compositional Software Product Lines. IEEE Software 2010; 27(3):

29-34. DOI: ieeecomputersociety.org/10.1109/MS.2010.32. [4] Bosch J, Bosch-Sijtsema PM. From integration to composition: On the impact of

software product lines, global development and ecosystems. Journal of Systems and Software 2009; 83: 67–76. DOI:10.1016/j.jss.2009.06.051.

[5] Boehm B, Turner R. Balancing Agility and Discipline - A Guide for the Perplexed. Addison-Wesley, 2004.

[6] Bosch J, Bosch-Sijtsema PM. Coordination between global agile teams: From process to architecture. In Šmite D, Brede Moe N, Ågerfalk P (Eds.). Agility across time and space. Implementing agile methods in global software projects. Springer Verlag: Berlin, 2010; 217-233.

[7] Linden F van der, Bosch J, Kamsties E, Kansala K, Obbink H. Software Product Family Evaluation. In: Proceedings of the Third Conference Software Product Line Conference (SPLC 2004, LNCS 3154), Springer Verlag: Berlin, 2004; 110-129.

[8] Beck K. Extreme programming explained: Embrace change. Addison-Wesley: Boston, 1999.

[9] Poppendieck M, Poppendieck T. Lean software development: An agile toolkit. Addison-Wesley: Boston, 2003.

[10] Rising L, Janoff NS. The Scrum software development process for small teams. IEEE Software 2000; 17(4): 26–32. DOI: 10.1109/52.854065.

[11] Schwaber K. Agile software development with Scrum. Prentice Hall: New York, 2001.

[12] Larman C. Agile and Iterative Development: A Manager's Guide. Addison-Wesley: Boston, 2004.

[13] Babar MA, Ihme T, Pikkarainen M. An industrial case of exploiting product line architectures in agile software development. Proceedings of the 13th International Software Product Line Conference (SPLC09) 2009; 446: 171-179.

[14] Hanssen GK, Fígri TE. Process fusion: An industrial case study on agile software product line engineering. Journal of Systems and Software 2008; 81(6): 843-854. DOI: 10.1016/j.jss.2007.10.025.

[15] Kruchten P. Software architecture and agile software development: a clash of two cultures? Proceedings of the 32nd ACM/IEEE conference on Software Engineering 2010; 2: 497-498. DOI: http://doi.acm.org/10.1145/1810295.1810448

[16] Tian K, Cooper K. Agile and Software Product Line Methods: Are They So Different? 1st International Workshop on Agile Product Line Engineering (APLE'06). 2006, IEEE Computer Society: Baltimore, Maryland, USA.

[17] Eisenhardt KM. Building theories from case study research. Academy of Management Review 1989; 14(4): 532-550.

Bosch, J. & P.M. Bosch-Sijtsema (2011). Introducing agile customer-centred development in a legacy software product line. Software: Practice and Experience, 41(8): 871-882. DOI: 10.1002/spe.1063  

  15  

[18] Adler PA, Adler P. Observational Techniques. In: Denzin NK, Lincoln YS. Handbook of Qualitative Research. Sage: Thousand Oaks, 1994; 377-393.

[19] Miller J, Glassner B. Interviews. The Inside and the Outside, Finding Realities in Interviews. In Silverman D. (Ed.) Qualitative research. Theory, Method and Practice. Sage: London, 1997; 99-112.

[20] Corbin J, Strauss A. Basics of qualitative research, 3rd ed. Sage: Thousand Oaks, 2008.