reporting and build statistics using business intelligence by naga sowjanya karumuri build team,...

Reporting and Build Statistics Using Business Intelligence

By

Naga Sowjanya Karumuri Build Team,

VMware,Cambridge

Summer Internship 2008

1. Problem Background

• The Release Engineering management team needs meaningful views into build-related data in order to make informed tactical and strategic decisions.

• For this purpose, they started generating reports using Perl/python scripts as shown below.

• Two basic problems with this method:

- Script-generated output presents values but not the trends; as a result decisions cannot be made based in this data.

- Manually generating the reports is a tedious process.

2. Manual Method

Database Perl/Python Scripts

Spreadsheet / Word Document

Querying Database

Manually Copying Script Output

3. Solution

• This project sets a Reporting system using Pentaho Business Intelligence Suite, an open source equivalent of Crystal Objects.

• Business Intelligence (BI) refers to technologies, applications and practices for the collection, integration, analysis, and presentation of business information to support better business decision making.

• Why this solution ?

- Reports present the data in tables and graphs in a way that improves understanding of the data.

- Reduces labor of manager in writing the scripts.

4. Pentaho BI Suite

Report Designer• Helps us set the layout of the report. • Connects the reports to a result set of a query.• Displays data in the result set using text and number fields.• Display data in the form of a chart.• Display date and page numbers.• Publish the report to the server

Design Studio• Assign author name, description and version number to the report.• Helps us set the output type of the report.• Allows user to give input parameters to the report.• Helps to set the cron string so as to run the report automatically at regular intervals.

5. Architecture of the Project

Backend is a reporting database which is a collective representation of BuildDB, Bugzilla, ContinuousBuild data files. This is generated by extracting data from different data sources and transforming them to tables and loading them into the reporting database.

Middle Layer generates requested reports using Report Designer and Design Studio.

Front End is a web portal that presents the list of reports that the user can generate with a single click.

6. Database and Report Designer

Source data Store Report Template from

Report Designer

Continuous Build Data files

Publish to server

Perl/python scripts

7. Adding User Interaction

Design Studio Web Portal

Add title

Add author name

Setup recurring schedule

Add User interaction to the webpage

Report

Click to view the output

8. Reports Generated

Product Statistics : The success/failure rates presented indicates which products needs more attention.

Build Summary : Presents a detailed view of the reasons for build failures, e.g. infrastructure, merge, dependencies, during the current reporting period tells us if an area requires immediate intervention.

Build Summary for 1 yr : Comparing failure trends with other trends over a similar period, to spot causal relationships.

Branch Deltas : Tracks the trend over the number of branches built upon.

9. Reports Generated: Description (cntd..)

Official Build Server Utilization : Graph will help with the prediction of when new servers need to be purchased due to low server availability. The graph can also indicate the greater need for capacity between Linux and Windows servers.

Branch Summary : Shows the trend in branches that can be used to compare to other resources over the same period.

Official Stats Summary : Provides a snapshot into the current resource usage to answer the question, “Do we have sufficient resources applied to the official build subsystem?”.

10. Official Stats Summary for 1 Year

The results of this report are shown below. From this we can learn :

• Trends among various build resources. For example, did an increase in active branches result in increased build failures.

• Trends in Success Rate of Official Builds. Abnormalities will indicate other issues (like inadequacy of pre-checkin tests) to be investigated.

• The current resource usage to answer the question, “Do we have sufficient resources applied to the official build subsystem?”.

11. Report Generated

12. “Wins”

Proof-of-Concept

Meaningful output can be produced with minimal expense and a reasonable amount of effort.

Preliminary Reports Completed

The build team can make decisions based on the first set of reports

Automation

Reports are automatically generated and posted.

Extensible

The groundwork has been laid so that new data sources and new reports can be easily added into the reporting system.