irishdata&analytics&landscape&survey&2014: 2015& … · introduction!...

Irish Data Analytics Landscape Survey 2014-‐2015 Analysis

APRIL 2015

TABLE OF CONTENTS

CONTENTS

Executive Summary ______________________________________________________________________________________ 1

Introduction _______________________________________________________________________________________________ 3

Survey Method ____________________________________________________________________________________________ 3

Analysis of Survey Responses ____________________________________________________________________________ 4

Summary ________________________________________________________________________________________________ 14

Contact Information ____________________________________________________________________________________ 15

EXECUTIVE SUMMARY

© The Analytics Store 2015 1

EXECUTIVE SUMMARY

Analytics and Big Data have been receiving a lot of press attention recently -‐ ranging from attempts to describe and define them, to assessments of how organizations can monetise their data. Within an Irish context questions such as the following are being asked:

• How big is the average Irish analytics team? • What industries in Ireland are using analytics? • Are companies in Ireland planning to expand their analytics teams in 2015?

While there have been a number of European and international surveys of the analytics industry there has not been a specifically Irish survey taking the pulse of the analytics industry in Ireland. For this reason The Analytics Store launched the Irish Data Analytics Landscape Survey 2014-‐2015 in December 2014.

SURVEY HIGHLIGHTS The survey was conducted online in December 2014. Overall there were 94 survey responses, 75 of which completed the survey. We omit incomplete responses, so include 75 responses in our analysis.

Survey Par*cipant Profile • Almost 70% of participants use analytics either frequently or almost always in their decision

making processes. This shows that the majority of participants in the survey are from organizations currently using analytics.

• The survey responses came primarily from people in large companies (56%) , with 15% coming from medium sized companies and 29% from small companies.

• The majority of participants (over 75%) came from the banking and finance, IT, and services sectors.

Organising Analy,cs Highlights • Most participants are using analytics to ensure greater accuracy in decision making (68%)

and/or to remove "gut instinct" from decision making (45%). • Implementing successful data analytics projects is not without its challenges, and the most common

challenge reported is the difficulty in hiring suitably qualified staff (45%), followed by insufficient relevant data (35%).

• Where to house an analytics team is a perennial question at analytics conferences. In this survey 34% of participants worked in organisations where analytics belonged to a line of business, 21% worked in organisations in which analytics belonged to IT, and 10% worked in organisations in which analytics belonged to a specific analytics team.

• It is promising for the Irish analytics industry that over half of the people surveyed work in companies that plan to hire new analytics staff in the coming year.

EXECUTIVE SUMMARY


Analy&cs Applica&ons Highlights • Sales and marketing dominate the use of analytics, followed by finance and operations, which is

consistent with international surveys. • It is interesting to see that 15% of the survey participants worked in organisations at which

analytics was being used in human resources. This is a growing trend that is also seen internationally.

• Almost all participants used reporting, and over two thirds performed statistical and exploratory analysis.

• Over half of the participants reported using some aspects of advanced analytics (for example predictive modelling or forecasting), which speaks to the maturity of the practice of analytics in Irish companies.

• Participants primarily reported using transactional data in their analytics work. • The use of unstructured data (for example text, audio, images or video) is growing with 20% of

respondents reporting its use. • SQL and Excel are the most commonly used tools, and remain workhorses on most analytics

projects. After this there were a wide spread of tools, with the open source programming language R being the next most commonly used tool.

• For advanced analytics tools there seems to be an even balance between GUI-‐based tools (for example IBM SPSS, SAS Enterprise Miner and RapidMiner), and programming languages (for examples Base SAS, R and Python).

• 28% of respondents had used one or more Big Data specific tools. The most commonly used Big Data tools were Hadoop, Hive, and Spark.

INTRODUCTION


INTRODUCTION

How big is the average Irish analytics team? What industries in Ireland are using analytics? Are companies in Ireland planning to expand their analytics teams in 2015? While there have been a number of European and international surveys of the analytics industry1,2,3 there has not been a specifically Irish survey taking the pulse of the analytics industry in Ireland. For this reason The Analytics Store launched the Irish Data Analytics Landscape Survey 2014-‐2015 in December 2014. This document describes an analysis of the results of this survey.

SURVEY METHOD

The survey was conducted online in December 2014 through the SurveyMonkey platform, and promoted primarily using social media. Figure 1 shows a screenshot of the survey interface. The survey contained 16 questions, only a small number of which were mandatory. Depending on the answers that were given to certain questions participants were guided through different routes through the survey. Overall there were 94 survey responses, 75 of which completed the survey. We omit incomplete responses and so include 75 responses in our analysis.

Figure 1: A screenshot of the survey interface.

1 PwC’s Global Data & Analytics Survey 2014: Big Decisions https://www.pwc.com/gx/en/issues/data-‐and-‐analytics/big-‐decisions-‐survey/index.jhtml 2 NewVantage Partners’ 2014 Big Data Executive Survey http://newvantage.com/thought-‐leadership/publications-‐and-‐executive-‐surveys/ 3 BARC’s Big Data Analytics 2014 Survey http://barc-‐research.com/research/big-‐data-‐analytics-‐2014/

ANALYSIS OF SURVEY RESPONSES



This section analyses the responses of participants to the survey. The survey was broken down into thematic sections and responses within each section are analysed separately.

CHARACTERISTICS OF SURVEY PARTICIPANTS The first four mandatory questions in the survey were designed to characterise the survey participants, and the companies in which they worked. Figure 2 and Figure 3 characterise the companies in which participants worked. Two interesting things stand out. First, the survey responses came primarily from people in large companies. Second, the majority of participants (over 75%) came from the banking and finance, IT, and services sectors. This is mostly due to the networks within which the survey was promoted, but is also indicative of the fact that analytics is primarily practiced within larger organisations and in particular industries. The other industries represented in the survey responses were hospitality and events, media, and gaming.

Figure 2: Question 1 -‐ What size is your organisation? We defined small companies as having less than 20 employees, medium companies as having between 20 and 249 employees and large companies as having 250 or more employees. (Responses: 75)

29.3%

14.7%

56.0%

0%

10%

20%

30%

40%

50%

60%

Small Medium Large

What size is your organisation?



Figure 3: Question 2 -‐ In which sector does your organisation operate? (Responses: 75)

The third question in the survey asked participants to characterise their own roles at the company in which they worked. Figure 4 summarises participants’ responses. Half of the participants were in roles that mixed technology and business, with the remainder split between solely business roles, solely technology roles and other roles (which included finance, communications, and education). We believe that this means that the survey respondents were in a position to offer an interesting mix of views from both business-‐focused and technology-‐focused points of view.

2.7%

4.1%

4.1%

6.8%

6.8%

23.0%

24.3%

28.4%

0% 5% 10% 15% 20% 25% 30%

Manufacturing

Telecommunications

Retail & Wholesale

Public Sector

Other

Services

IT

Banking & Finance

In which sector does your organisation operate?



Figure 4: Question 3 -‐ What best describes your role at your organisation? (Responses: 75)

The final question in the first part of the survey asked users to characterise the use of data analytics at their companies. Figure 5 summarises participants’ responses. Only 7 of the survey participants (9.3%) worked in companies where data analytics was not used at all. These participants were redirected to the end of the survey and did not answer any of the other questions. The remainder of the survey participants were using analytics, at least to some extent, which validated their participation in the survey.

Figure 5: Question 4 -‐ Does your organisation use data analytics techniques to drive decision making? (Responses: 75)

50.7%

20.0%

9.3%

20.0%

0% 10% 20% 30% 40% 50% 60%

Business & Technology

Business Technology Other

What best describes your role at your organisation?

9.3%

24.0%

41.3%

25.3%

0%

10%

20%

30%

40%

50%

Never Rarely Frequently Almost always

Does your organisation use data analytics techniques to drive decision making?



ORGANISING ANALYTICS The next section of the survey asked participants about how analytics was used at their organisations. This began by asking what the major drivers for using analytics were. Figure 6 summarises responses to this question (participants were allowed to select multiple answers). Most participants were interested in improving decision making, either selecting the Ensuring greater accuracy in decision making option or the Removing "gut instinct" from decision making option4. This is in line with international survey responses.

Figure 6: Question 7 -‐ What are the major drivers of the use of data analytics for decision making in your organisation? (Responses: 66)

Implementing successful data analytics projects is not without its challenges and Question 8 in the survey addressed this. Figure 7 summarises the responses to this question. It is interesting that the most common challenge faced is the difficulty in hiring suitably qualified staff. Although there has been a growth in third level analytics courses (for example at UCD5 and DIT6) qualified, experienced analytics practitioners are still thin on the ground. It is likely that this will continue for a number of years until the current pool of graduates gain relevant experience. It is also interesting that, in spite of all we hear about the deluge of data facing us, 35% of participants saw insufficient relevant data as a challenge to their analytics projects. Relevant is the key word here. It is very often the case that although masses of data are available in organisations, there is a dearth of clean, recent, appropriate data for analytics projects. Finally, it is worth noting that only 7 participants (11%) mentioned data protection issues. We expect this to become a bigger

4 Of the 28 participants (42%) who selected the Removing "gut instinct" from decision making option 17 also selected the Ensuring greater accuracy in decision making option and 11 did not. Taking these 11 participants together with the 45 participants (68%) who selected Ensuring greater accuracy in decision making this means that 56 participants (85%) were interested in improving decisions making. 5 www.ucd.ie/mathsciences/graduatestudents/onlinecoursesindataanalytics/ 6 www.dit.ie/postgrad/programmes/dt228adt228bmscincomputingdataanalytics/

20%

23%

38%

42%

55%

68%

0% 20% 40% 60%

Demonstrating our capacity for innovation

Regulatory compliance

Competitive advantage

Removing "gut instinct" from decision making

Exploiting our data resources

Ensuring greater accuracy in decision making

What are the major drivers of the use of data analytics for decision making in your organisation?



challenge in the future as awareness of data protection and privacy issues grows in the public consciousness.

Figure 7: Question 8 -‐ What challenges did you face in undertaking data analytics projects this year? (Responses: 65)

Questions 9 and 10 in the survey asked participants about the size of the analytics teams in which they worked and whether they intended to grow these teams in the coming year. Figure 8 summarises the responses to these questions. It is interesting that in the survey participant pool 26 participants (38%) work in organisations with large analytics teams (more than 10 team members). Having unpacked this responses a little, most of these respondents work either in analytic services companies or at large companies in the banking and finance sector. This is not unexpected as most banking and finance organisations have large risk teams, which do a lot of analytics work.

It is promising for the Irish analytics industry that over half of the people surveyed work in companies that plan to hire new analytics staff in the coming year. A word of caution is required, however, given the fact that access to qualified staff is the number one challenge faced by practitioners.

5%

11%

22%

23%

25%

35%

45%

0% 10% 20% 30% 40%

Lack of compelling business case

Data protection issues

Lack of corporate sponsorship

DifZiculty accessing suitable tools

Cost

InsufZicient relevant data

DifZiculty hiring suitable staff

What challenges did you face in undertaking data analytics projects this year?



(a) (b) Figure 8: (a) Question 8 -‐ What is the size of the data analytics team at your organisation? (Responses: 68) (b) Question 9 -‐ Do you plan to hire additional data analytics staff in 2015? (Responses 67)

Where to house an analytics team is a perennial question at analytics conferences, and Question 11 in the survey explored this. Figure 9 summarises the results. 34% of participants worked in organisations where analytics belonged to a line of business. In fact most of the other responses actually referred to a line of business so this number is really closer to 45%. This is the classic siloed approach that tends to emerge as a company begins to use analytics, and is also common internationally. Analytics also remains something associated with IT, and 21% of respondents work in companies in which this is the case. It is promising, however, that in 10% of organisations analytics belonged to a specific analytics team, which suggests that these companies are moving towards a scenario in which analytics is recognised as a function in its own right.

Figure 9: Question 11 -‐ Which department at your organisation controls data analytics? (Responses: 68)

13%

19% 15%

38%

15%

0%

10%

20%

30%

40%

1 2 -‐ 5 6 -‐ 10 > 10 None

What is the size of the data analytics team at your organisation?

43%

37%

12% 7%

0%

10%

20%

30%

40%

No 1 -‐ 5 5 -‐ 10 > 10

Do you plan to hire additional data analytics staff in 2015?

4%

10%

15%

16%

21%

34%

0% 10% 20% 30%

Data Architecture Team

SpeciZic Analytics Team

Other

BI Team

IT

Line of Business

Which department at your organisation controls data analytics?



ANALYTICS APPLICATIONS The next section of the survey asked participants about the ways in which they were using analytics at their companies. The first question in this section, Question 12, asked about the departments that utilised the outputs of analytics efforts. Figure 10 summarises these results. Sales and marketing dominates the use of analytics, followed by finance and operations. The prevalence of sales and marketing departments as consumers of analytics is consistent with international surveys. It is interesting to see that 9 of the survey participants (15%) worked in organisations at which analytics was being used in human resources. This is a growing trend that is also seen internationally.

Figure 10: Question 12 -‐ Which departments at your organisation utilise the outputs of data analytics projects? (Responses: 60)

It is also interesting to consider the types of analytics that people are doing and the types of data they are working with. Questions 13 and 14 dealt with this, and the results are summarised in Figure 11 and Figure 12. Almost all participants used reporting, which is not surprising, and over two thirds performed statistical and exploratory analysis. It is encouraging to note that over half of the participants reported using some aspects of advanced analytics (for example predictive modelling or forecasting). This speaks to the maturity of the practice of analytics in Irish companies.

15%

22%

23%

27%

30%

62%

62%

75%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Human Resources

Other

Production

IT

Research & Development

Operations

Finance

Sales & Marketing

Which departments at your organisation utilise the outputs of data analytics projects?



Figure 11: Question 13 -‐ Which of the following data analytics techniques has your organisation used in projects this year? (Responses: 60)

Participants primarily reported using transactional data in their analytics work. Again this is in line with international results. The use of unstructured data (for example text, audio, images or video) is growing in the international analytics community and was relatively well represented here with 12 respondents (20%) reporting its use.

3%

23%

23%

55%

57%

58%

65%

68%

92%

0% 20% 40% 60% 80% 100%

Other

Association analysis

Text analytics

Segmentation

Forecasting

Predictive modelling

Exploratory data analysis

Statistical analysis

Reporting

Which of the following data analytics techniques has your organisation used in projects this year?



Figure 12: Question 14 -‐ What types of data did you use in data analytics projects this year? (Responses: 60)

Question 15 in the survey asked respondents which tools they were using for analytics projects -‐ their responses are summarised in Figure 13. SQL and Excel were the most commonly used tools, and remain workhorses on most analytics projects. After this there were a wide spread of tools, with the open source programming language R being the next most commonly used tool. For advanced analytics tools there seems to be an even balance between GUI-‐based tools (for example IBM SPSS, SAS Enterprise Miner and or RapidMiner), and programming languages (for examples Base SAS, R and Python). The tools mentioned more than once under the other category were Qlikview and Teradata.

7%

15%

20%

28%

32%

85%

0% 20% 40% 60% 80% 100%

Other

Sensor data

Unstructured data

Social media data

Log data

Transactional data

What types of data did you use in data analytics projects this year?



Figure 13: Question 15 -‐ Which of the following data analytics tools or languages did your organisation use in projects this year? (Responses: 62)

The final question in the survey asked participants whether they had used so-‐called big data tools in an analytics projects this year (a list of the following tools was included: BigML, Cassandra, Giraph, Hadoop, HBase, Hive, Mahout, Pig, and Spark). Figure 14 summarises the responses. 21 respondents (28%) had used one or more of these tools. This suggests that people are starting to work with datasets outside of the typical small transactional data that characterises early use of analytics. The most commonly used big data tools were Hadoop (15 respondents), Hive (5 respondents), and Spark (3 respondents). It will be interesting to see if the use of these types of tools increases in future iterations of this survey.

3%

3%

6%

6%

6%

6%

6%

15%

18%

18%

26%

29%

32%

34%

37%

58%

74%

0% 20% 40% 60% 80%

Weka

KNIME

SAP

Rapidminer

SPSS

Matlab

Oracle Data Miner

IBM SPSS

Other

Python

Tableau

Microsoft SQL Server

SAS (Enterprise Miner)

SAS (Base)

R

SQL

Excel

Which of the following data analytics techniques has your organisation used in projects this year?

SUMMARY


Figure 14: Question 16 -‐ Were "big data" specific tools or languages used by your organisation in projects this year? (Respondents: 75)

SUMMARY

The purpose of this survey was to capture the data analytics landscape in Ireland in 2014-‐2015. The number of survey respondents was not high enough to say that the results are fully representative of the entire analytics industry, but they are sufficient for interesting analysis and present a snapshot of the current state of the industry, and a baseline for comparison of future studies.

The survey results paint a picture of a reasonably mature analytics industry, with many participants reporting analytics applications moving beyond descriptive analytics to the application of advanced analytics techniques (such as predictive modelling), which in some cases use interesting unstructured data sources and high-‐end big data toolsets. There is also some evidence for the emergence of strong, centralised analytics teams -‐ although in most cases analytics work is siloed in a line of business.

The main challenge facing analytics practitioners is a lack of experienced, qualified candidates to fill analytics roles. It is expected that this will continue to be the case for the immediate future. In-‐house training of existing staff is likely to be a solution to this issue for many companies. It is somewhat surprising that, in spite of the data deluge we constantly hear about, a lack of relevant data continues to be an issue for some practitioners. This is likely to continue to be the case and highlights the importance of strong data management and governance.

As we repeat this survey in the coming years it will be interesting to benchmark against these results

Yes 28%

No 72%

Were "big data" speciWic tools or languages used by your organisation in projects this year?

CONTACT INFORMATION


CONTACT INFORMATION

Company: The Analytics Store Address: 12 Merrion Square, Dublin 2. Tel: 01 631 6007 Website: www.theanalyticsstore.com

irishdata&analytics&landscape&survey&2014: 2015& … · introduction!...

Documents