building practical analytics at the wall street journal ... · the wall street journal: rapid...
TRANSCRIPT
![Page 1: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/1.jpg)
#AnalyticsXC o p y r ig ht © 201 6, SAS In st i tute In c. A l l r ig hts r ese rve d.
Building Practical Analytics at The Wall Street Journal: Rapid Development Using Amazon Redshift
Jeff ParkinsonVice President, Customer OperationsDow Jones
![Page 2: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/2.jpg)
Deploying Customer Analytics in the Cloud
Jeff ParkinsonVice President, Customer Operations
![Page 3: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/3.jpg)
Speaker Background – Jeff Parkinson
o Vice President of Customer Operations at Dow Jones (7 years)
o Over 20 years of CRM experience across multiple industries- Fidelity Investments – Director of Marketing Operations- Accenture – Marketing Transformation practice- JP Morgan – Program Management Office
o Deep usage of IBM, Oracle, SFDC and SAS operational and analytical stacks
![Page 4: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/4.jpg)
o We explain the world and the world of businesso We are authoritative journalism and smart technologyo We provide a window on events, clarify issues, inspire new thinking and give readers and business
customers the insight they need to make informed decisionso We are newspapers, newswires, websites, apps, newsletters, magazines, proprietary databases,
conferences and more o Our premier brands include The Wall Street Journal, Dow Jones Newswires, Factiva, Barron’s,
MarketWatch and Financial News
Company Background
![Page 5: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/5.jpg)
Product Focused to Customer Centric
Product
Customer
Organizational Paradigm Shift
![Page 6: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/6.jpg)
Challenges - People
Product
Customer
o Customer channel data stored across
multiple external vendors and applications
o Aligning centralized data vision across
product groups
o Internal legacy systems requiring complex data transformations
o Various versions of the truth
o Competing data groups
o Lack of a true customer master record or single view of the customer
o Intuitive easy to use reporting interface
![Page 7: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/7.jpg)
Challenges - Process
Product
Customer
o Multiple Analytics teams working on the
same analysis
o Different data stores producing different
results
o Complexity of standard data pulls require heavy data mining skill set
for simple requests
o Turn around time weeks to market as a result
o Inability to scale resource due to advanced skill set needed
![Page 8: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/8.jpg)
Data Architecture Current State
Data Warehouse
Reporting Cubes
Usage Data
WSJ Barron’s MarketWatch
Data Mart
Legacy Systems
CRM Applications
![Page 9: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/9.jpg)
Challenges - Technology
Data Warehouse
Reporting Cubes
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
o Legacy systems have different data models
Usage Data
Data Mart
![Page 10: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/10.jpg)
Challenges - Technology
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o Reporting cubes hard coded and out of date
Usage Data
Data Mart
![Page 11: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/11.jpg)
Challenges - Technology
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o No feedback from channels
Usage Data
Data Mart
![Page 12: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/12.jpg)
Challenges - Technology
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o No connectivity between commerce and content systems (online activity
Usage Data
Data Mart
![Page 13: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/13.jpg)
Challenges - Technology
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o Channels lack a hub to record all interactions
Usage Data
Data Mart
![Page 14: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/14.jpg)
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o Tagging suboptimal and not gathered at an individual level on a regular basis
Usage Data
Data Mart
Challenges - Technology
![Page 15: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/15.jpg)
Personalizing Every Touchpoint
Mobile Print
Direct Mail Social Media
Telemarketing Web
Customer Service
Tablet
![Page 16: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/16.jpg)
This project will provide the following business benefits:
- Expansion of the Data Provisioning team’s scale of resource to be applied to ad hoc requests
- Access by non-technical users to simple data visualization of key metrics
- Ability to scale resources as datamart will be accessible to all levels of data miners
- Migration to a SAS analytical stack enabling the team to increase their skill set in the world -class data tool
- Documentation of all business rules
- Enablement of vendor-supplied analysis/marketing via access to subscription data – we currently struggle to do this
o Create an accessible campaign datamart allowing quick and agile access to account-level data agnostic of source system
o Implement SAS applications to provide functionality required for segmentation, campaign management and reporting
Program Goals:
Initial Phase - SAS Datamart to build Customer Master
![Page 17: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/17.jpg)
Key Challenges We Encountered
o Addressing lack of AWS knowledge within company
o Optimizing the movement of large data volumes
o Complexity of coordinating incremental updates versus full data overwrites (PC SAS)
o Outlining ETL process (SAS vs Lambda functions for master controller)
o Adhering to stricter data encryption requirements
o Storing Personally Identifiable Information (PII) data in the cloud
o Shifting resource mind sets entrenched in PC SAS to Server SAS
![Page 18: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/18.jpg)
Project Scope
o Infrastructure phase was a little harder than expected
o Internal resources gaining experience with AWS and all major connection points are working together
o More Information Security hurdles than were expected
o Good news is that all delays took days not months and we now have a core team that is making rapid progress each day
o Static data set in AWS Red Shift
o Visualization being connected to data set (these cover Phases 1.1-1.2, 1.3 coming swiftly)
Next Steps
o Nightly Updates currently being worked on
o Extension of current resources
o Prioritization of Phase 2
![Page 19: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/19.jpg)
✓ Scalable (up or down scale)
✓ Easy to administer (distribution, sort keys)
✓ Secure
✓ Good disaster recovery
✓ Integrates well with SAS
✓ Cost-efficient
✓ Easy to pick up skillset wise (SQL)
✓ Complements other components of solution S3 and EC2
Why AWS Redshift?
![Page 20: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/20.jpg)
SAS Office AnalyticsSAS Enterprise Miner
SAS DataFlux
Possible Cloud Data Mart Configuration
x Prodx Dev
Potential Future Migration
Extract Staging BucketDelivery Staging Bucket
Data Warehouse
Conceptual Design
![Page 21: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/21.jpg)
What does Maturity look like? - Peopleo Embedded analysis function in each key
stakeholder group to drive strategy from actionable insights
o Self-serve visualization tools to empower users such as Office Analytics and Visual Analytics
o A centralized data infrastructure and governance function producing consolidated data
o Appropriate communication and training to ensure usability and adoption
o Auditability process to ensure validity and instill user confidence
![Page 22: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/22.jpg)
What does Maturity look like? - Process
o Encryption must be incorporated into all data transfers and when data is at rest
o Privacy Statements adjusted to allow for responsible insight gathering with subscriber permission
o All data transformations must be audited regularly
o Data Quality must become an obsession and a permanent group within the Data Management function both for new data streams and existing data stores
o Tagging and recording the correct activity data is essential (embarrassing to be wrong)
![Page 23: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/23.jpg)
What does Maturity look like? - Technology
o Connectivity between systems must be solved
o Centralized customer record essential starting point
o Build activity and interest data on top of customer record
o Record and store historical snapshots of customer record for modeling and regression analysis
o Connect firmographic and demographic information to customer master for 360 degree view
![Page 24: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/24.jpg)
What does Maturity look like? - Technology
![Page 25: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/25.jpg)
What does Maturity look like? - Technology
![Page 26: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/26.jpg)
What does Maturity look like? - Technology
![Page 27: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/27.jpg)
What does Maturity look like? - Technology
![Page 28: Building Practical Analytics at The Wall Street Journal ... · The Wall Street Journal: Rapid Development Using Amazon Redshift Jeff Parkinson Vice President, Customer Operations](https://reader030.vdocuments.mx/reader030/viewer/2022041016/5ec7eeb8c42d770bb1016ef3/html5/thumbnails/28.jpg)
C o p y r ig ht © 201 6, SAS In st i tute In c. A l l r ig hts r ese rve d.
#AnalyticsX