the open analytics platform · •data storage –ms azure blob storage, amazon s3 •databases...
TRANSCRIPT
![Page 1: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/1.jpg)
© 2017 KNIME AG. All Rights Reserved.
What’s New
Bernd Wiswedel
KNIME
![Page 2: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/2.jpg)
© 2017 KNIME AG. All Rights Reserved. 2
Outline
• “What’s new“ presented in two use cases, presented by the team
• Questions/Discussions/Concerns: Find us!
• Demo booths in the registration area
![Page 3: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/3.jpg)
© 2017 KNIME AG. All Rights Reserved. 3
What’s new - Agenda
• Too much to show it all.
• Some highlights:
– Audio Processing
– Text Processing
– Analytics & Scripting
– KNIME Server
– Cloud Offerings
– Guided Analytics
Use Case 1: IMDB Reviews
Use Case 2: Census Data
![Page 4: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/4.jpg)
© 2017 KNIME AG. All Rights Reserved. 4
KNIME Software Pieces
KNIME®AnalyticsPlatform
Open Source
Extensions
Community&
PartnerExtensions
KNIME® Server
![Page 5: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/5.jpg)
© 2017 KNIME AG. All Rights Reserved. 5
What’s new - Agenda
• Too much to show it all.
• Some highlights:
– Audio Processing
– Text Processing
– Analytics & Scripting
– KNIME Server
– Cloud Offerings
– Guided Analytics
Use Case 1: IMDB Reviews
Use Case 2: Census Data
![Page 6: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/6.jpg)
6© 2017 KNIME AG. All Rights Reserved.
KNIME as an Open (Source) Platform
![Page 7: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/7.jpg)
© 2017 KNIME AG. All Rights Reserved. 7
KNIME Analytics Platform – Source Code on GitHub
• Source Code available on GitHub & BitBucket
• Ongoing effort, enough to get started
https://github.com/knime/
![Page 8: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/8.jpg)
© 2017 KNIME AG. All Rights Reserved. 8
SVN To Git – Fun Facts
• Switched from SVN to Git earlier the year
• ~3 weeks conversion time(parallelized over 10 physical machines)
• Split into 70 repositories (60 open source)
• Code Base:
– ~20GB (includes text models, R, etc)
– ~1.7 Mio Lines of Code (open source only)
– 1500 integration tests per version (excl. unit tests)
![Page 9: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/9.jpg)
© 2017 KNIME AG. All Rights Reserved. 9
KNIME Analytics Platform – Nightly Builds
• Nightly Builds publicly available:
![Page 10: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/10.jpg)
© 2017 KNIME AG. All Rights Reserved. 10
KNIME Personal Productivity – free & open-source
• Part of the KNIME Analytics Platform 3.4
• What is it?
– Workflow Diff Viewer
– Metanode Templates
– Workflow Linking (“Call Local Workflow”)
– “Local” Workflow Coach
![Page 11: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/11.jpg)
© 2017 KNIME AG. All Rights Reserved. 11
KNIME Personal Productivity – Workflow Diff
![Page 12: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/12.jpg)
© 2017 KNIME AG. All Rights Reserved. 12
KNIME Personal Productivity – Metanode Templates
![Page 13: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/13.jpg)
© 2017 KNIME AG. All Rights Reserved. 13
KNIME Personal Productivity – Workflow Linking
• Workflow Orchestration by calling out to other workflows
![Page 14: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/14.jpg)
© 2017 KNIME AG. All Rights Reserved. 14
KNIME Personal Productivity – free & open-source
• Part of the KNIME Analytics Platform 3.4
• Big Data Extensions in December (3.5)
![Page 15: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/15.jpg)
15© 2017 KNIME AG. All Rights Reserved.
Speech Recognition– Tobias Koetter –
![Page 16: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/16.jpg)
© 2017 KNIME AG. All Rights Reserved. 16
KNIME Audio Processing
• New audio cell type
• Listen to audio files within KNIME
• Extract acoustic features
• Speech recognition
https://www.youtube.com/watch?v=1L6zwLMvA-U
https://pixabay.com/en/microphone-silver-metal-sound-waves-1074362/
![Page 17: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/17.jpg)
17© 2017 KNIME AG. All Rights Reserved.
Text Processing– Kilian Thiel –
![Page 18: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/18.jpg)
© 2017 KNIME AG. All Rights Reserved. 18
Example Use Case
• Sentiment classification of movie reviews
– PDF parsing
– Feature vector creation
– Predictive modeling
![Page 19: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/19.jpg)
© 2017 KNIME AG. All Rights Reserved. 19
Example Use Case
19
Goal
• Build predictive models to predict sentiment labels of reviews, “positive” or “negative”.
• “Ah, Moonwalker, I'm a huge Michael Jackson fan …”
• “This film has a very simple but somehow very bad plot. …”
Positive or negative?
![Page 20: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/20.jpg)
© 2017 KNIME AG. All Rights Reserved. 20
Data Set
• The Large Movie Review Dataset v1.0
– English movie reviews
– Associated sentiment labels “positive” and “negative”
– http://ai.stanford.edu/~amaas/data/sentiment/
• 2000 PDFs
– 1000 positive reviews
– 1000 negative reviews
![Page 21: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/21.jpg)
© 2017 KNIME AG. All Rights Reserved. 21
Textprocessing
• Tika integration
• StanfordNLP NER Learner & Tagger nodes
• Document Vector Hashing
• Word2Vec integration
![Page 22: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/22.jpg)
26© 2017 KNIME AG. All Rights Reserved.
Analytics & Scripting– Bernd Wiswedel –
![Page 23: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/23.jpg)
© 2017 KNIME AG. All Rights Reserved. 27
• Logistic Regression - more options through new solver
• New Integrations – more on this tomorrow
– DeepLearning via Keras
– H2O AI Platform
Analytics
![Page 24: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/24.jpg)
© 2017 KNIME AG. All Rights Reserved. 28
Scripting – Python (Labs)
• Supports Python 3
• Major Speedup
• Used for new DeepLearningIntegration
![Page 25: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/25.jpg)
© 2017 KNIME AG. All Rights Reserved. 29
Type Extensions in Java Snippet
• Java Snippet: Swiss Army knife for data manipulation
• Read/write support for 3rd party types:SVG, XML, JSON, Images, Molecules, …
https://www.knime.org/blog/using-custom-data-types-with-the-java-snippet-node
![Page 26: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/26.jpg)
© 2017 KNIME AG. All Rights Reserved. 30
Date & Time Handling Nodes revised
• Richer data types
• Timezone support
• Duration “math”
![Page 27: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/27.jpg)
© 2017 KNIME AG. All Rights Reserved. 31
Three new helpers!!
• Generate a file path based on folder, file name and extension type
• Simple changes to string based variables
• ... and for number variables as well.
![Page 28: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/28.jpg)
32© 2017 KNIME AG. All Rights Reserved.
... and now for something completely different– Rosaria Silipo –
![Page 29: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/29.jpg)
33© 2017 KNIME AG. All Rights Reserved.
E-Learning
![Page 30: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/30.jpg)
© 2017 KNIME AG. All Rights Reserved. 34
Free E-Learning Course: Web Page
34
• Hands-on e-learning course
• Data Access, ETL, Analytics, Control Structures, Visualization
• Around 50 small units
• … with exercises
• … and with solutions on the EXAMPLES server
• Final exercises to test your knowledge!
https://www.knime.org/knime-introductory-course
![Page 31: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/31.jpg)
© 2017 KNIME AG. All Rights Reserved. 35
KNIME Course Slides – now available as pdf
![Page 32: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/32.jpg)
© 2017 KNIME AG. All Rights Reserved. 36
E-Books
At the KNIME Press:
https://www.knime.com/knimepress
![Page 33: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/33.jpg)
37© 2017 KNIME AG. All Rights Reserved.
On-line Learning
![Page 34: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/34.jpg)
© 2017 KNIME AG. All Rights Reserved. 38
On-line Courses
• On-Line Courses vs. On-Site Courses with Teacher
• First Experiment with Joe Porter’s group in November/December
• Basic Course on KNIME Analytics Platform
• With Homework and Homework Correction
Send us an email: [email protected]
![Page 35: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/35.jpg)
© 2017 KNIME AG. All Rights Reserved. 39
Webinars
In 2018 Series of Webinars: • Data Science Cycle• Introduction to KNIME Analytics Platform• Data Access• Database• Logistic Regression• Ensemble Models • Text Processing• …
Keep an eye on our web site www.knime.com
![Page 36: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/36.jpg)
40© 2017 KNIME AG. All Rights Reserved.
Learning @Meetups
![Page 37: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/37.jpg)
© 2017 KNIME AG. All Rights Reserved. 41
Learnathons
Data Science Cycle: ETL, Model Training, Deployment
LondonBerlin
ZurichMilanNew York
Austin
Somewhere on WC
Sao Paulo
![Page 38: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/38.jpg)
42© 2017 KNIME AG. All Rights Reserved.
![Page 39: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/39.jpg)
43© 2017 KNIME AG. All Rights Reserved.
KNIME Server– Jon Fuller–
![Page 40: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/40.jpg)
© 2017 KNIME AG. All Rights Reserved. 44
KNIME Software Pieces
KNIME®AnalyticsPlatform
Open Source
Extensions
Community&
PartnerExtensions
KNIME® Server
![Page 41: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/41.jpg)
© 2017 KNIME AG. All Rights Reserved. 45
KNIME Server
Shared Repositories Access Management Web Enablement
Flexible Execution
![Page 42: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/42.jpg)
© 2017 KNIME AG. All Rights Reserved. 46
Deploy to Server
![Page 43: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/43.jpg)
© 2017 KNIME AG. All Rights Reserved. 47
Open in WebPortal
![Page 44: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/44.jpg)
© 2017 KNIME AG. All Rights Reserved. 48
Authentication
• Authentication with LDAP/Active Directory.
• Single Sign On (SSO) with Kerberos
![Page 45: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/45.jpg)
© 2017 KNIME AG. All Rights Reserved. 49
KNIME Server – Admin made easy
![Page 46: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/46.jpg)
50© 2017 KNIME AG. All Rights Reserved.
KNIME in the Cloud– Jon Fuller –
![Page 47: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/47.jpg)
© 2017 KNIME AG. All Rights Reserved. 51
Why cloud?
• Bring analytics to the data
• Work fast, work agile
• Pay for what you need
• New ways to work
![Page 48: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/48.jpg)
© 2017 KNIME AG. All Rights Reserved. 52
Cloud Platforms
• AWS and Azure
• The two leaders in Cloud (see Gartner 2016)
![Page 49: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/49.jpg)
© 2017 KNIME AG. All Rights Reserved. 53
KNIME Cloud Products (Marketplace)
• KNIME Cloud Analytics Platform
• KNIME Server (5-User)
• KNIME Server + Big Data Extensions (5-User)
• KNIME Server (+ Big Data Extensions) BYOL
![Page 50: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/50.jpg)
© 2017 KNIME AG. All Rights Reserved. 54
KNIME Server in the Cloud
![Page 51: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/51.jpg)
© 2017 KNIME AG. All Rights Reserved. 55
Cloud Connectors
• Enable working directly with data on the AWS or Azure clouds
• S3
• Blob Storage
• SQL Server
AmazonS3
Microsoft Azure Blob
storage
Azure SQL DB
![Page 52: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/52.jpg)
© 2017 KNIME AG. All Rights Reserved. 56
KNIME Cloud Connectors
• Amazon S3/Azure Blob Storage:Save (large) unstructured data in the cloud
• Connect through KNIME’s filehandling nodes
– List
– Upload/Download
– Delete
– URL access
![Page 53: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/53.jpg)
© 2017 KNIME AG. All Rights Reserved. 57
Cloud Connectors Workflow Example
![Page 54: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/54.jpg)
© 2017 KNIME AG. All Rights Reserved. 58
Azure SQL Server Connector
![Page 55: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/55.jpg)
© 2017 KNIME AG. All Rights Reserved. 59
More connectors: Amazon Redshift
• Fully managed data warehouse
• SQL interface
• Distributed and parallelized queries
• Amazon Redshift connector integrates seamlessly with existing KNIME Database connectors
Amazon Redshift
![Page 56: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/56.jpg)
© 2017 KNIME AG. All Rights Reserved. 60
Amazon Athena
• Serverless, interactive query service
• SQL interface
• Amazon Athena integrates seamlessly with existing KNIME Database connectors
![Page 57: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/57.jpg)
© 2017 KNIME AG. All Rights Reserved. 61
Amazon Athena
![Page 58: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/58.jpg)
© 2017 KNIME AG. All Rights Reserved. 62
S3/Blob Storage support for Spark Data Source Nodes
![Page 59: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/59.jpg)
© 2017 KNIME AG. All Rights Reserved. 63
Cloud Resources
• Data storage– MS Azure Blob Storage, Amazon S3
• Databases– Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc)
• Data warehouse– Amazon Redshift, Amazon Athena, Azure SQL-DW
• Hadoop/Spark– Azure HDInsight, Amazon EMR
Amazon Redshift
Azure SQL Server
Azure Blob Store
AmazonS3
Azure HDInsight
Amazon EMR
Amazon Athena
![Page 60: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/60.jpg)
© 2017 KNIME AG. All Rights Reserved. 64
KNIME in the Cloud: Summary
• KNIME Analytics Platform available on Azure Marketplace
• KNIME Server available on Azure and AWS marketplaces
• Cloud connectors for S3, Blob Store, Azure SQL server
![Page 61: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/61.jpg)
65© 2017 KNIME AG. All Rights Reserved.
Guided Analytics– Greg Landrum –
![Page 62: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/62.jpg)
© 2017 KNIME AG. All Rights Reserved. 66
Reminder of what we’re talking about here
KNIME expertSomeone who needs data / analytics
![Page 63: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/63.jpg)
© 2017 KNIME AG. All Rights Reserved. 67
What often happens
KNIME expertSomeone who needs data / analytics
![Page 64: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/64.jpg)
© 2017 KNIME AG. All Rights Reserved. 68
What we’d like to provide
KNIME expertSomeone who needs data / analytics
Guided Analyticsworkflows
![Page 65: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/65.jpg)
© 2017 KNIME AG. All Rights Reserved. 69
Guided Analytics workflows
• Enable people who are not KNIME experts to work with/explore/analyze/learn from data
• Typically deployed via KNIME WebPortal
• Heavy use of interactive JavaScript views for the user experience
![Page 66: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/66.jpg)
© 2017 KNIME AG. All Rights Reserved. 70
Guided Analytics workflows
• Enable people who are not KNIME experts to work with/explore/analyze/learn from data
• Typically deployed via KNIME WebPortal
• Heavy use of interactive JavaScript views for the user experience
• Bonus for everyone: the JavaScript views that we’re building for Guided Analytics are also really useful in “normal” KNIME workflows.
![Page 67: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/67.jpg)
© 2017 KNIME AG. All Rights Reserved. 71
So what’s new?
• JavaScript Views:– Parallel coordinates plot– Table view– Decision tree view– Network view– Stacked area chart– Sunburst chart
• Interactivity: Selection and filtering– Range filter widgets– Linked views– Accessing views in wrapped metanodes, composite view– Layout editor
![Page 68: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/68.jpg)
72© 2017 KNIME AG. All Rights Reserved.
But let’s start with a demo
![Page 69: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/69.jpg)
© 2017 KNIME AG. All Rights Reserved. 73
The workflow behind the demo:
Page 1
Page 2
Page 3
![Page 70: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/70.jpg)
© 2017 KNIME AG. All Rights Reserved. 84
JavaScript Stacked Area Chart
![Page 71: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/71.jpg)
© 2017 KNIME AG. All Rights Reserved. 85
JavaScript Network Visualization
![Page 72: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/72.jpg)
© 2017 KNIME AG. All Rights Reserved. 86
JavaScript Sunburst Chart
![Page 73: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/73.jpg)
87© 2017 KNIME AG. All Rights Reserved.
(Short) Coffee Break- all of us -
![Page 74: The Open Analytics Platform · •Data storage –MS Azure Blob Storage, Amazon S3 •Databases –Azure SQL Server, Amazon RDS (MySQL, PostgreSQL etc) •Data warehouse –Amazon](https://reader036.vdocuments.mx/reader036/viewer/2022071106/5fe0d43e2f163924b778d478/html5/thumbnails/74.jpg)
88© 2017 KNIME AG. All Rights Reserved.
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by KNIME.com AG under license from KNIME GmbH, and are registered in the United States.
KNIME® is also registered in Germany.