big data meetup_10_9_2013
DESCRIPTION
TRANSCRIPT
Click to edit Master title style
This presentation, and the information contained therein, is the proprietary and confidential property of ComLinkData, powered by
Comniscient Technologies LLC, and may only be viewed by the individual to which it was initially provided by ComLinkData. By viewing this
presentation, you agree that you will not copy, reproduce or distribute the presentation in whole or in part, without the prior written consent
of ComLinkData. You agree not to forward this presentation (by email or otherwise) to anyone inside or outside your organization, and you will
not save this presentation on a network to which anyone other than you has access (except for information technology professionals who are
subject to confidentiality provisions, in the ordinary course of their responsibilities). You agree to maintain the confidentiality of all of the
information and data in the presentation that is not already in the public domain. You further agree to use the presentation only for the
purpose of evaluating the services offered by ComLinkData, and will not use any of the confidential information in the presentation for any
other purpose, including for the purpose of analyzing aspects of the telecommunications market.
Turning Big Data into Business Applications: An introduction to R Shiny@tanyacash21, @JoshuaWalker_
10/9/2013
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.2
The Opportunity: Real-time Telecom Insights
50,000 transactions a day18MM+ a year
2.2 Terabytes Disk Space3 years of data and growing
Demographics
Advertising spend
Events
Social Media
Distribution
Device
Real-time Telecom Switching Data
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.3
Data Flow and Architecture
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.4
The Front End Framework
Web Application (Front End)
jQuery
AngularJS
d3.js
Twitter Bootstrap
Font Awesome
Javascript, HTML5, CSS3 Java Servlets API(Service Layer)
1. Parse queries2. Fetch data
3. Construct JSON
SQL Server / Amazon Redshift
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.5
R is Gaining Major Visibility
http://r4stats.com/2013/03/19/r-2012-growth-exceeds-sas-all-time-total/
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.6
Testable, Reproducible, Scalable
• Testable – RUnit package
• Reproducible – Commented scripts / multiple ways to ingest data (flat files, Excel files, ODBC/JDBC, Rcurl)
• Scalable – doSNOW, doMPI, doMC, foreach packages
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.7
Developers and Data Geeks Tend to Have Mutually Exclusive Sets of Skills
Web Developers / Designers / Engineers
Data Scientists / Analysts / Researchers
!=
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.8
Solution: Shiny!
+ =
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.9
Shiny Features
• Easy to use – readily available widgets, no HTML/Javascript or jQuery knowledge required
• Fast bidirectional communication using websockets
• Uses Twitter Bootstrap
• Reactive Programming Model
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.10
Reactivity
Inputs on a web page Plots / Tables
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.11
rCharts
Generates d3.js code
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.12
googleVis
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.13
Define HTML UI Instead of ui.R for Even More Flexibility
index.html
ui.R:<application-dir>|-- www |-- index.html|-- server.R
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.14
Interactive Shiny Workshop
Start up R!
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.15
Demo of Full Shiny Applications
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.16
New Method of Collaboration and Product Development in Data-Centric Companies
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.17
References
1. http://rcharts.io/2. http://rcharts.io/gallery/3. http://cran.r-project.org/web/packages/googleVis/index.html4. http://cran.r-project.org/web/packages/websockets/index.html5. http://rstudio.github.io/shiny/tutorial/#welcome6. https://groups.google.com/forum/#!forum/shiny-discuss7. http://r4stats.com/2013/03/19/r-2012-growth-exceeds-sas-all-time-total/8. http://www.businessinsider.com/excel-partly-to-blame-for-trading-loss-2013-2