microsoft data platform airlift 2017 rui quintino machine learning with sql server 2016 and r...
TRANSCRIPT
Data PlatformAirlift
Rui QuintinoData Research, [email protected]
Machine Learning withSQL Server 2016 and R Services
24 de fevereiro \\ Microsoft Lisbon Experience
Your feedback is important!
Keep in touch with Microsoft Azure
Try Azure for FREE now: https://azure.microsoft.com/free/
Step by step guide to setting up Azure SQLhttps://aka.ms/adc_guidetosettingupazuresql
Web Apps on Microsoft Azurehttps://aka.ms/adc_webappsazure
E-book | Azure Overviewhttps://aka.ms/adc_azureoverview
Free Manual | Microsoft Azure https://aka.ms/adc_manualgratisazure
Understanding Data Technologies on the Cloud
https://aka.ms/adc_datatechnologiescloud
E-book | Microsoft Loves Linuxhttps://aka.ms/adc_microsoftloveslinux
Agenda•Machine Learning•R – What & Why?•R, Microsoft & SQL Server•SQL 2016 R Services•Q&A
Machine Learning ?Using existing structured or unstructured data to:
1. Predict unknown/future data
2. Create intelligent & automated agents/services
3. Advanced Data Insights -> Why? Drivers? Root causes?
4. Content producers & “creative” agents
Agent allocation
Warehouse efficiency
Smart buildings
Predictive maintenance
Supply chain optimization
Churn/segmentation
Personalized offers
Product recommendation
Fraud detection
Risk management
Sales forecasting
Demand forecasting
Sales lead scoring
Marketing mix optimization
Sales and marketing
Finance and risk
Customer and channel
Operations and workforce
Investment modelling $$$
Example use cases (classic)Revenue Forecasting$$$Portfolio optimization
What is
• A statistics programming language• A data visualization tool• Open source
• 2.5+M users • Taught in most universities
• Thriving user groups worldwide
• 10.000+ free algorithms in CRAN• Scalable to big data
• New and recent grad’s use it
Language Platform
Community
Ecosystem• Rich application & platform integration
R & R Studio
Demo
http://blog.revolutionanalytics.com/2017/01/cran-10000.html
??
Lack of Commercial
Support
Inadequate/Limited
Performance
Complex Deploymen
tProcesses
Limited
Data Scale
Challenges posed by open source R
• R & SQL Server• SQL Server is one of the most widely used SQL databases• R is the most widely used statistical and advanced
analytical language • Complications From Using R with SQL Databases• Requires Data Extraction• Bottlenecks in Performance• Data Sizes Limitations• Increases Security Risks• Increases Duplication Costs• Poor operationalization support
Before SQL Server 2016 & R Services
April 6, 2015
SQL Server 2016 EESQL Server 2016 SE
Growing Beyond Revolution Analytics
Red HatSUSE
Pre Acquisition
Microsoft R Server
Azure HDInsights
Azure
Expanding Product Family
SQL Server R Services
Post Acquisition
Continued Support of Enterprise R Solutions
Expanding Support for Open Source R
Cortana Analytics Suite
Open
Introducing Microsoft R Server
High-performance, Scalable R100% open source RCRAN, Bioconductor, MRAN, GitHub compatibilityBig-data connectivityScalable analyticsMulti-platform In-database, in-cluster scalabilityChoice of IDE (R Studio, R Tools for Visual Studio)
Linux, Windows, SQL Server, Hadoop & Teradata
Open Source Components
Licensed Components
CRAN
Micr
osof
t R
Open
DistributedR
ScaleR
ConnectR
DeployR IDE
R Server Technology
Introducing SQL Server 2016 R services
Included in SQL Server 2016
Reuse and optimize existing R codeEliminate data
movement
In-database deployment
Memory and disk scalability
No R memory limitsWrite once, deploy
anywhere
Enterprise speed and scale
Near-DB analytics Parallel threading and
processing Reuse SQL skills for data engineering
Cost effectiveness
Scalability and choice
Simplicity and agility
SQL Server 2016SQL Server R ServicesIntegration Facilities:• Component
Integration• Launchers• Parameter
Passing• Results Return• Console Output
Return• Parallel Data
Exchange (RTM)• Stored Procedures• Package
Administration
SQL Server 2016 & SQL R Services
SQL ServerQuery
Processor
Algorithm Library
• Data Prep• Descriptive Stats• Sampling• Statistical Tests• Predictive Models
• Variable Selection• Clustering• Classification• Custom APIs for R +
CRAN• Parallel Scoring
Fast, Parallel, Storage Efficient Algorithms
Revolution R Open• 100% Open Source R• Fully CRAN
Compatible• Accelerated Math
Open Source R Interpreter
Using Parallel Algorithms, Remote Contexts
Ingest
Set Context
Use
Scalable algorithms
• 5+ hours to 40 seconds:
R - Portability & investment assurance
Write Once – Deploy Anywhere
R Server portfolio
Cloud• Windows• Linux• HDInsight
• SQL Server 2016 EE• SQL Server 2016 SERDBMS• Windows• LinuxDesktops & Servers
Hadoop & Spark • Hortonworks• Cloudera• MapR
EDW • SQL Server 2016• Teradata Database
CRAN
Micr
osof
t R
Open
DistributedR
ScaleR
ConnectR
DeployR IDe
R Server Technology
SQL Server 2016 R Services
Demo
Additional Notes/References•Azure Data Science Virtual Machine
Additional Notes/References•SQL 2016 R Services Virtual Labs
Additional Notes/References•Free ebook: Data Science with Microsoft SQL Server 2016
Additional Notes/References•New R packages•olapR•MicrosoftML •LightGBM (“xgboost” by Msft)
Additional Notes/References•SQL 2016 Machine Learning Templates
SQL Server 2016 R Services
Q&A
Obrigado!Rui QuintinoData Research, DevScope
twitter.com/rquintino
rquintino.wordpress.com
www.devscope.net
Free AzureTrial
Try SQL Server 2016http://aka.ms/trysql2016
http://aka.ms/tryazure
Try Power BIhttp://powerbi.com
Cortana Intelligence Serviceshttp://aka.ms/cortanaintelligence
© 2016 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.