got big data? get big people
Post on 25-Feb-2016
Embed Size (px)
DESCRIPTIONGot Big Data? Get Big People. Apple Maps vs Google Maps. Source: http://theamazingios6maps.tumblr.com/post/32055905836/yup- lexington -avenue-is-now-in- brooklyn -and-no. Source: Google Maps (2013 -09-25). Bad business because of data. Make your customers very unhappy. - PowerPoint PPT Presentation
Got Big Data? Get Big People.Apple Maps vs Google MapsSource: http://theamazingios6maps.tumblr.com/post/32055905836/yup-lexington-avenue-is-now-in-brooklyn-and-noSource: Google Maps (2013-09-25)
Explode your support operations
Start a battle with your partnersBad business because of data
Market starts questioning the company
Make your customers very unhappyBad business because of data
Recommend competition to customers
Forced to reorganize & dump talent
Keep paying to fix your mistakes
Googles approach a different way of thinking about data
Googles approach clear data policies with partners
ProblemData is getting bigger, faster, in more shapes and formats, from more sources more complex to control more important for business, both for operational and analytical purposesBusiness wants to keep focusing on the businessData is in databases and servers, so IT is covering itComplexity increase makes current approach tedious, error prone and unsustainable
Meer met minder houdt een keer op7SolutionMake slogan data is an asset actionable:Stop trying to convince the business with it is the right thing to doStart involving the business in a way that is easy to understand, practical and effectiveMake the data management process a sustainable business processIT cannot focus on technology alone and needs to facilitate business taking ownershipShow & communicate that data drives business and business drives data
Where to start?Thousands and thousands of different data elementsIt takes significant time and effort to define, categorize, relate to business terms, establish governance, etc.GovernanceData Architecture and infrastructureData Accuracy and IntegrityCompletenessTimelinessAdaptability Reporting accuracyComprehensivenessBasels PrinciplesNot all data elements are equally important to business or corporate functionsRegulatory pressure is increasing and much more data points are required at higher frequencySome data elements are required by auditors, counterparties, etc.Find out what is critical
and use it to drive changeMove from report based delivery to data element based delivery
Select Critical Data Elements based on business line and corporate function needs, regulatory and external requirements Streamline internal (e.g., MA) & external reporting
Show proof of control over data: regulatory forbearance & compliance as well as overall increased internal efficiencyDeliver faster at lower costPositioning
Applying Data Governance
FinanceJohn FisherData Steward
Data Governance CouncilBob BrownData GovernanceOfficerMary SmithChief Steward(Finance)Mike JonesChief Steward(Sales)
Judy ClarkBusiness StewardSalesData Governance Working Group
Data GovernanceManagerEmma MillerData GovernanceManagerLet's zoom in on this a little bit and see who the stakeholders for Data Governance are.
We need to understand who they are, what they expect, and how they can contribute.
The example that I am using is a typical one, even for organizations with a lower level of maturity:- on top, we have some kind of council with decision makers, ideally harvested across functional groups, and a top champion who sponsors the initiative- at the bottom, we have the different areas and various kinds of stewards. These are business users who in their daily work produce and consume data.
I would say that the top people are our "one percenters" of DG and that the bottom people are the "five - ten" percenters of DG. Data Governance is not their main job. They contribute to it. They contribute a percentage of their time to Data Governance.
In the middle we find the full time Data Governance people. Data Governance is their job. They live and breathe it. This is typically a core team which co-ordinates all the data governance efforts within the organization.
How does the Collibra software help all these people? How can the software help them increase your data maturity?14
Search and interact
They look at reports: in specialized reporting environments, on shared portals, in shared drives, inside spreadsheets.As soon as something in there seems not quite right - you know, the numbers seem wrong, what is behind them?, is the data correct?, is the report showing what it should? ... - the trust in your data gradually disappears. Data Governance guarantees trustworthy data .
From any Windows application where you can select text (office, excel, IE, ...) our target user can simply highlight what he or she needs, hit CTRL+ALT+S and Collibra delivers contextualized search results. These provide the first answers: there is more than one kind of customer, what are the differences, what are the rules, where is the data stored, ... And from here people can directly use the "Business User Portal" to get access to all the data assets and easily search, filter, browse, navigate information as well as access the people with Data Governance responsibilities to re-establish and maintain trust.15Business Glossary
Reference DataManage, mapping & hierarchy
A characteristic of reference data is that a lot of it will be outside of your control. One of the simplest examples here are the ISO Country Codes. The International Standards Organization (ISO) manages the list of countries as well as the list of codes (2 or 3 character codes) to represent those codes. And those codes are sprinkled all over your data: which customers are in which countries, operating locations, transactions between countries, ... And yet, you do not control that standardized list in any manner. You get what is given.
That doesn't mean that what you get is immediately useful. For example, "AS" represents "American Samoa", one of the "countries". Except it is not a country. It is a dependency, and its parent country is the US. That could mean for example that any transactions made with AS needs to follow US regulations. And that you should listen to the FDA for clinical trials in AS. This is critical.
You will also need to "hook up" this reference data to your own internal reference data. Let's zoom in on the region "North America". Inside Collibra we can relate that region (which might be internal reference data) with those countries (which is external reference data): "North America" consists of "Canada", "The United States" and many others. Drilling down on "The United States" shows us other relations as well:- two direct mappings: US as in the ISO 3166-2 list, and USA as in the ISO 3166-3 list, and- according to the countries by the "Bank of England" the United States is part of "USA". This is because they use the reference code "USA" to include the United States, as well as American Samoa, Guam and the Northern Mariana Islands.
Next to those mappings & relations, you also need hierarchy management of your reference data. Let's take products as an example. We have Swaps (LO2000) & Loans (LO1000). Then we find Fixed Rate Interest Loans (LO1100), Floating Rate Interest Loans (LO1200) and the mixed bag: Fixed/Floating (LO1300) and so on. Like the Basic Line numbers of the old ages (Line 10 do this, Line 20 do this so if I need to do something in between I still have lines 11 - 19 as a buffer). People will use the specifics of the coding scheme to hide that hierarchy in their databases. Collibra takes care of this as well.18Timetravel
Because the only constant is change, and because not everybody (or every system) can handle change at the same speed, you also need the ability to travel through time. In Collibra you have snapshots to make sure you can access the relevant reference data at the relevant point in time. Need the latest and greatest for your new Big Data project that you are gearing up? Check. Need the countries that you were doing business with in 2011 for audit & compliance purposes? Check.
Like I have shown before, the accessibility (via search, the BUP, ...) of these data assets is valuable. People who do data entry can make sense of the obscure values in dropdowns or figure out the exact code they have to type in field X on the screen.
This reference data can then be provisioned to other systems.19Data Stewardship ManagementThe process of data managementI put my heart and my soul into my work, and have lost my mind in the process. Vincent Van GoghIntake / ProposalReview, approval, votingEscalationIssue managementData SharingSecurity classificationPolicy audit
The Data Stewardship Manager enables the process of data management. Let me show you how that works with two specific processes. These two processes will illustrate the reusable components within the Data Stewardship Manager which will support any Data Management process within your organization.
Picture by Andrew AveleySource: http://photo.getaway.co.za/2012/03/26/andrew-aveley-hidden-danger/ My first process example: Issue Management. Even if you believe that all is well, there are always data related issues. 22Detect and research the issue
I think I have a problem with my dataLet's zoom in on "John Doe" who depends on data for his daily business activities. He can't quite put his finger on it, but he believes that something is wrong with the data he is working with. After some investigation, John figures out that there really is bad data involved. 23Communicate the issue
Guys, I have a problem with my dataJohn communicates the issue.24Hoping for feedback in the communicationYup. There seems to be a problem with his data
People respond ... emphasis on his. 25Processing the issue
/dev/nullThis way of treating the problem takes the issue from John to his colleagues and from there straight to