inspire 2013 - alteryx and the teradata unified data architecture
Post on 20-Oct-2014
899 views
DESCRIPTION
With so much to be gained from slicing and dicing the vast amount of data being collected, why aren’t more companies further along in their efforts to exploit all of their available data? In a word: complexity. The volume, velocity and variety of data coming from transactional systems, web, text, social media, and machine generated data make it too complex to be analyzed in an ad-hoc manner. And with the variety of environments like data warehouses, Hadoop and other analytical platforms available, what tools should be brought to bear to take advantage of it all? The Teradata Unified Data Architecture™ helps make sense of these massive, unruly data sets so organizations spend time analyzing information rather than gathering and managing data, letting users leverage these powerful resources transparently to unlock new and valuable business insights. The net result: higher productivity, lower costs, and a broadening of opportunities. See how Alteryx provides a visual workflow to blend and move data plus create and publish analytics within and across the different environments supported in the Teradata Unified Data Architecture™, and learn how you can get started immediately using Alteryx with Teradata.TRANSCRIPT
Getting your data to “play nice with others” using
Alteryx and the Teradata Unified Data Architecture
Bruce Johnson, Teradata
Jim Schattin, Alteryx
Technology track
March 7, 2013
1:15 – 2:00 pm
Person with the hose?
Person trying to get the hose?
Dog waiting for the bath?
OR
The Data
The Analyst
The Problem to be solved
Which one are you?
Data (Big & Small) Provides Sources For Insight
• Organizations compete on analytics
So are the challenges to figure out how to best enable this capability
Why?
• Narrowly focused analytics against
narrow data sets produce narrow insights
• Enriching and putting all data to work
makes for smarter decisions and data-
driven business models
• Opportunities for organizations to
benefit from analytics across more of
their data is greater than ever before…
• Organizations are not making use of all the
data they possess NOW!
• Undiscovered nuggets of information about
customers, products and performance
hidden in hard to reach places – e.g., ERP,
legacy systems, web logs, social media
Challenge = Opportunity
Data should be easily accessible and usable
Platforms and tools should be flexible and scalable
Business should be able to “ask any question at any time”
Need to understand the role of new technologies (e.g., Hadoop, MapReduce)
Need to understand the value in Big Data, how it can improve what we know today!
• Teradata’s response to this opportunity is
the Unified Data Architecture™ (UDA) to
deploy available technologies to unleash
the value of data
• Creates a strong analytic foundation by
embracing existing, new and emerging
technologies in a cohesive manner
Take advantage of it!
• Manage all the data with workload specific engines and a consistent set of tools
> Apply the right technology to the right analytical opportunities
> Isolate intelligent signals in a world of noise
> Turn invisible opportunities into actionable decisions
Three capabilities working in conjunction with one another…
• Teradata Data Warehousing: Integrated and shared data environment serves as the foundation
for any analytic environment, provides a single source of centralized data for reuse, delivers
strategic and operational analytics to the extended organization
• Aster Data Discovery: Pre-packaged SQL-MapReduce capabilities for data-driven discovery,
helping unlock insights from big data, performed with a technology with rapid exploration
abilities via a variety of analytic techniques and accessible by mainstream business analysts
• Hadoop Data Staging: Preparation for analytics in a low-cost
technology proven to be very effective for loading, storing
and refining data
Data, answer-sets, and insights passed seamlessly among
the architecture capabilities
Synchronized components with transparent management,
access and analysis of all of the data
Teradata Unified Data Architecture™
Teradata Unified Data Architecture™
ADVANCED ANALYTICS
Enable any analysis against any type or volume of data at any time…
Discovery and
exploration platform
that enables agility
with limited
constraints
Data warehouse for
insight deployment
into a reliable
production
environment…
So you’ve got big and little data, a discovery
platform and a flexible architecture…. Now what?
Analytics Ecosystem Strategy That Fits
Business Users Executives Analysts Data Scientists
NUMBER OF USERS
Str
ate
gic
Goals
and
Init
iati
ves
Feedback
KPIs
Insight
Opportunity
Advanced Analytics
EDW
Guided Analytics
Guided Analysis
CO
MPLEXIT
Y
Low
H
igh
Low High
Analytics Lab
Develop Analytics to Solve Business Problems…
Statistics
Forecasting
Geospatial Graph Analysis
Augment traditional analytic
approaches
with new approaches
Text Analysis
… And Make Your New Insights Operational
Marketing Service Provider Digital Marketing Attribution
• Benefits:
- Marketing analysts more productive with Aster
- Lower cost - storage and batch refining done on
Raw Web Logs
Analytic Tools
Teradata Aster
Co
ok
ie-
level d
ata
Arc
hiv
al
Hadoop (on AWS) (Storage, aggregations,
cleansing)
Ad Server Logs
Media Data (Aggregated)
Custom Data by Client
• Segmentation:
Custom SQL-MR algorithms to match
and create centralized identifiers
• Sessionize by client
• nPath identifies segment path analysis
(behavior after ads)
Big Box Grocer – Initial Use Case Affinity Analysis
What is the customer
buying in key categories?
Does this affect other
categories and how?
Analytics Architecture
Teradata
Aster Lab/Test
Teradata EDW Prod
Hadoop Store/Cleanse
• Reporting
• 2 – 3yrs data
• Productionization
• Prototyping
• Fail Fast
• Analytics Lab
• Aggregations / ETL /API • Data Hub – Active Archive • Cleansing Multi-Structured
Guided Analytics
Layer Prototyping Layer Application Layer
Reports Mobile Analyst Data Scientist Social
Data
base
Layer
Data
Layer
Enterprise Systems Transactional Data Documents Social/Text/Log Audio Images IT/OT Video
Affinity Analysis
Current Method UDA Method
Tool SQL SQL-MapReduce
(Collaborative Filter operator)
Dataset Time Span 13 weeks 8 years (32X time span)
Affinity Calculation One category against
others
All Categories vs. all others
Calculation Time 4 hours 48 Minutes
2.4 Minutes - same calculations against same data
Affinity Analysis: Shelf Stable Juice Seasonal Affinity with Other Categories
0
0.005
0.01
0.015
0.02
0.025
1 4 7 10 1 4 7 10 1 4 7 10 1 4 7 10 1 4 7 10 1 4 7 10 1 4 7 10 1 4 7
2004 2005 2006 2007 2008 2009 2010 2011
Collab.
Score
Year/Month
Alcohol
Cereal
Frozen - Ice Cream
Laundry Detergent
Other Cheese
Paper Towels
Pizza
Shredded Cheese
Sliced Cheese
String Cheese
Affinity Analysis:
Analyzing Affinity of items over a long duration (6-10yrs) will provide key insights into running better promotions,
planogram and price planning using affinity of items. Affinity Analysis on 8 Years of Data for All Categories against All Other Categories
Consumer Migration:
Analyzing declines in consumer segments over large timeframes. Determine the items missing from declining baskets and why
How much time best (Platinum, Gold) consumer is spending in different segments before becoming unengaged?
Pricing Affinity:
Analyzing item price movement and its impact on basket size and affinity of items over a long duration (6 years). Determine individual and multiple item their price movement impact on total basket
Competitor Impact:
Analysis of various competitor impacts over time Understand impact of competitor store opening on basket size and consumer loyalty (trips per month)
Determine if the effects are temporary or permanent
Social Media :
Integrating consumer online data (Social Media - Facebook) with existing transaction data to understand loyalty Understand the number of fans by demographic
Understand social media behaviors of best consumers (Platinum and Gold)
Differences in behavior of consumers in categories who are Facebook fans versus non-Facebook fans
Retail Use Cases
• Understanding Customer Service Interactions key to product ‘fit’
• Churn, Adoption, Attrition, Path to product cross sell
• Combine Check Image, Voice, Social, Web, Transaction data
• Web Analytics (90% of data)
• Combine and Sessionize HTTP raw, HTTPS and XML Application logs
• Find Golden path to Application Submit
• Execution requires discovery across multiple channels to see patterns and paths in
data to use as new variables for propensity scoring
• 4 steps
1) Customerization (ID the custom in data)
2) Sessionization
3) Sequencing Analytics (Discover behavior across a period of time and all channels)
4) Productionize in Models, Events and Campaigns
Financial Services Use Cases
Financial Services - Churn Prevention Hadoop captures,
stores and transforms social,
images and call records
Aster does path and sentiment analysis
with multi-structured data
Data Sources
Multi-Structured Data
Call Center Voice Records
Check Images
Traditional Data Flow
Analysis +
Marketing
Automation
(Customer
Retention
Campaign)
Capture, Retain and
Refine Layer
ETL Tools
Hadoop
Call Data
Check Data
Teradata
Integrated DW
Dim
en
sio
na
l
Data
Analy
tic R
esu
lts Aster Discovery
Platform Sentiment
Scores
SOCIAL
FEEDS
CLICKSTREAM
DATA
BRANCH, ATM
DATA
Email/Survey Data
Events Preceding Account Closure
Interactive Analytics
Reducing the “Noise” to find the “Signal”
SELECT * FROM npath ( ON ( SELECT … WHERE u.event_description IN ( SELECT aper.event FROM attrition_paths_event_rank aper ORDER BY aper.count DESC LIMIT 10) ) … PATTERN ('(OTHER|EVENT){1,20}$') SYMBOLS (…) RESULT (…) ) ) n;
Events Preceding Account Closure
Events Preceding Account Closure
Closed Accounts
Fee Reversal Seems to Be a “Signal”
SELECT * FROM nPath ( ON (…) PARTITION BY sba_id ORDER BY datestamp MODE (NONOVERLAPPING) PATTERN ('(OTHER_EVENT|FEE_EVENT)+') SYMBOLS ( event LIKE '%REVERSE FEE%' AS FEE_EVENT, event NOT LIKE '%REVERSE FEE%' AS OTHER_EVENT) RESULT (…) ) n;
Paths to Attrition (Version 2)
Multiple Fee Reversal and Viewing Product/Rates and
Offers happens in the last mile for Account Closure
Telco Use Case with Teradata UDA and Alteryx Problem A Global Communication Service Provider is interested in preventing customer churn by identifying at-risk customers and then
providing special offers that reduce the likelihood of churn in a profitable way. This requires use of predictive analytics.
Joint Solution Alteryx loads call records of customers that churned over the last 5 years into Aster to identify a golden path
Alteryx moves output into a Teradata Data Lab to combine with customer data from Teradata DW to drive in-database analytics
Alteryx performs detailed geospatial engineering/network analysis and then provides to Business Analysts for review
Results • Ability to identify key customers that are likely churn candidates
• Visualization of problem spots on the network (cell sites, network elements, ...) that are driving churn
• Understanding of other key reasons for churn – performance, competitive offers, ...
• See what offers have avoided churn by similar customers in the past
• Able to identify which offers will work and to evaluate a least cost offer to prevent churn
• The ability to make offers to keep customers from churning
• Deeper understanding of customer behavior
Perform predictive analytics to identify customers most likely to churn
Retail Use Case with Teradata UDA and Alteryx Problem Improving customer insight by identifying customers most likely to shop at a competitor, to drive better marketing campaigns,
to bring social media analytics into decision making, and to get a 360 degree view of product demand requires use of a strong
analytics solution that is accessible to business users.
Joint Solution Alteryx loads customer purchase history into Aster to identify golden paths for purchase and churn
Alteryx moves output into a Teradata Data Lab to combine with customer data from Teradata DW to drive in-database analytics
Results enhanced by applying Alteryx Drive Time analysis to understand which customers drive by competitors to shop them
Results • Ability to identify key customers that are likely churn candidates
• Gain customer insight and identify key attributes for purchase
• Combine ratings, reviews, mobile, and interaction data, and apply predictive model clustering, to determine products
with the most and least demand
• Integrate social media content to enable business units to understand market perception and to analyze sentiment
values in decision making
• Integrate drive time geospatial with data such as weather, sensor, economic, competitive, traffic, logistics and other
data sources to improve labor costs and maximize customer service
Enable business users to access a single analytics platform for a variety of requirements
E-MAIL CUSTOMER
SEGMENTS COMPETITOR ON-LINE STORE DATA IRI / NIELSEN PRICING
Affinity analysis
Consumer migration
Price elasticity
Competitor Incursion
X-promotion affinity analysis
Influencer analysis
Sessionize
Golden Path Determination
Fraud Sentiment Analysis
Channel Hopping
Attrition Paths
Fraudulent Paths
Productionize insights
Dashboards and reporting
Vendor managed inventory
Assortment optimization
Analytical Scoring
Event Triggers
Customer Behavior Analysis
Spend Analysis
Performance Analysis
Customer Segmentation
Risk Analysis
Customer Profitability
Portfolio Analysis
DISCOVERY
PLATFORM
CAPTURE | STORE | REFINE
INTEGRATED
DATA WAREHOUSE
TERADATA UNIFIED DATA ARCHITECTURE
LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS
Engineers
Data Scientists
Business Analysts
Front-Line Workers Customers / Partners Quants
Operational Systems Executives
Aggregations / ETL /API
Data Hub –Archive
Cleansing Multi-Structured
Consumerize
Voice to text; ID keyword
Image
X-Platform Aggregation
Demo
• Leverage Alteryx to direct analytics across Teradata and Teradata Aster to
gain consumer insight for your retail brand.
• Append 3rd party content to your customer records and load into Teradata
• Execute statistical algorithms in database via the Teradata R package
• Run nPath MapReduce function in Teradata Aster to reveal how online
customers navigate your web site to make a purchase
• Perform product correlation calculations in Teradata to understand
purchasing behaviors at brick and mortar locations
>>> Merchandise e-Commerce and Physical locations for optimal results
Retail Analytics
Let’s get started!
Ready to Play?