ebook: machine learning with amazon web services...amazon rekognition deep learning-based image and...

EBOOK: MACHINE LEARNING ON AWS

EBOOK:

Machine Learning with Amazon Web Services


Contents

1

Machine learning and AWS ....................................................................................................2

How AWS looks at machine learning ..................................................................................4

Advantages of AWS for machine learning ........................................................................5

Explore AWS machine learning services ............................................................................6

The AWS Machine Learning Competency .........................................................................7

Anodot case study: Lyft ............................................................................................................8

DataRobot case study: Trupanion ........................................................................................9

Narrative Science case study: Deloitte.............................................................................10

Trifacta case study: Consensus Corporation .................................................................11

Getting started ..........................................................................................................................12


Organizations are adopting cloud-based machine learning solutions at a fast pace. According to Gartner, “By 2019, citizen data scientists will surpass data scientists in the amount of advanced analysis produced. By 2020, more than 40% of data science tasks will be automated, resulting in increased productivity and broader usage by citizen data scientists.”.i Additionally, Gartner states, “by 2020, artificial intelligence will create more jobs than it eliminates. AI will create 2.3 million jobs in 2020, while eliminating 1.8 million.”ii

Organizations looking at ways to incorporate machine learning into their businesses, or to automate and improve existing data science efforts can look to Amazon Web Services (AWS) for solutions that can help. Amazon has leveraged machine learning (ML) for twenty years to power its businesses. AWS is now taking Amazon’s deep expertise in this space to enable any developer to embed ML capabilities into their applications.

Today, customers are using ML services in greater numbers to prepare data for analysis, build and refine ML models, and take advantage of end-user cognitive applications including voice recognition, image and video analysis, providing forecasts and recommendations, and many other intelligent solutions.

iSource: Gartner, 100 Data and Analytics Predictions Through 2021, Douglas Laney, Ankush Jain, 20 June 2017. iiSource: Gartner Press Release, “Gartner Says By 2020, Artificial Intelligence Will Create More Jobs Than It Eliminates.”

https://www.gartner.com/newsroom/id/3837763.

Machine learning and AWS

2

Organizations are using machine learning to prepare data for analysis, build and refine machine learning models, and take advantage of end-user cognitive applications.

https://www.gartner.com/newsroom/id/3837763


The machine learning world today is fast becoming democratized, with solutions available to any company, regardless of data science expertise. Whether you employ a dedicated data science team or are completely new to machine learning and data science, solutions from AWS and AWS Partner Network (APN) Partners can help you benefit from the power of machine learning technology.

In this eBook, you’ll learn how AWS and AWS Machine Learning Competency Partners are helping companies of all sizes implement machine learning faster than ever, taking advantage of the elasticity, scalability, and reliability of the cloud. You’ll learn how the combination of massive compute power, data lakes, security, analytics capabilities, and seamless integration with AWS environments is helping companies to realize the benefits of machine learning for their specific use cases, and how AWS and AWS Partner Network (APN) Partners can help you get more out of machine learning, regardless of your organization’s level of data science expertise.

Machine learning and AWS (cont.)

3


Artificial intelligence

Machine learning

Deep learning

Sense, learn, reason,act, and adapt to the real worldwithout explicit programming

Computational methods that use learning algorithms to build a model

from data (in supervised, unsupervised,semi-supervised,or reinforcement mode)

Algorithms inspired byneural networks with multiple

layers of neurons thatlearn successively

complex representations

Machine learning (ML) refers to the use of learning algorithms that build a model of understanding about the relationships between existing data to make predictions about new data. The term is often used interchangeably with artificial intelligence (AI), but in fact these terms refer to related, but separate, concepts.

AI is the ability to sense, learn, reason, act, and adapt to the real world without explicit programming. AI is the overall concept of building solutions that allow computers to learn and make decisions without explicit human instruction, and ML is the method by which developers create those abilities.

Deep learning (DL) is the third term often used when discussing machine learning. Rather than use explicit mathematical algorithms, DL attempts to model how the brain works and learns with systems called neural networks. DL is an effective tool to use when the solution is hard to articulate. For example, concisely explaining how we identify a dog from all other objects and animals is extraordinarily difficult. However, with DL we can have the neural network teach itself what a dog is through evaluation and feedback, similar to how a child learns to identify dogs.

How AWS looks at machine learning

4


Advantages of AWS for machine learning

5

• Powerful compute resources including the most powerful GPU-based instances in the cloud, to make model training and inference fast and efficient.

• Support for all major frameworks including Apache MXNet and Gluon, TensorFlow, Microsoft Cognitive Toolkit, Caffe, Caffe2, Theano, Torch, PyTorch, Chainer, and Keras.

• Flexibility to deploy ML models in Amazon EC2 instances, Amazon Lambda, on FPGA (on F1 instance types), or even in an edge using AWS Greengrass.

• Deep ML expertise from thousands of Amazon engineers who have developed ML solutions for the Amazon.com retail site, Amazon Alexa, Amazon Go, and many other areas across the company.

• Security, including a broad array of access control, identity management, and encryption capabilities.

• Platform integrations, including integrations for the data lake and database tools you need to run ML workloads.

• Comprehensive analytics services including data warehousing, business intelligence, batch processing, stream processing, and data workflow orchestration to augment ML applications.

AWS provides a flexible, scalable, and cost-effective platform for building, training, and hosting machine learning and artificial intelligence applications.

Advantages of AWS include:


Deploying on AWS allows organizations to adapt and innovate more quickly. AWS offers customers the ability to store and analyze petabyte-scale data sets, using only the resources they need, and paying only for what they consume. AWS services enable organizations to build and deploy cloud-based machine learning solutions that can:

• Prepare data sets for analysis by machine learning models

• Develop, test, deploy, and improve machine learning models

• Deploy end-user machine learning solutions for specific use cases

Explore AWS machine learning services

6

Amazon SageMaker

Build, train, and deploy machine learning models at scale

Amazon Rekognition

Deep learning-based image and video analysis

Amazon Transcribe

Automatic, grammatically-correct transcription of speech

Amazon Lex

Add voice and chat conversational bots to your applications

Amazon Polly

Turn text into lifelike speech across many languages

Amazon DeepLens

A fully programmable video camera, tutorials, code, and pre-trained models designed to expand deep learning skills.

Amazon Comprehend

Discover insights and relationships in text

Amazon Deep Learning AMIs

Pre-configured environments to quickly build deep learning applications

Amazon Translate

Natural and fluent language translation


AWS Machine Learning Competency Partners provide solutions that help organizations solve their data challenges, enable machine learning and data science workflows, or offer SaaS-based capabilities that enhance end applications with machine intelligence.

Admission to the AWS Machine Learning Competency is highly competitive, and prospective Partners are required to go through a rigorous auditing process to be admitted to the program. These Partners have been successful in demonstrating deep expertise in Machine Learning on AWS and the ability to deliver their organization’s solutions seamlessly on AWS.

Featured Partners extend the benefits of AWS by offering technology solutions that can help organizations of all sized get started with cloud-based machine learning.

The AWS Machine Learning Competency

7

Automated machine learning platform

Data wrangling (formatting and preparation) for use as training data for machine learning models

Automated time series analysis for anomaly detection

Natural Language Generation (NLG) for interpreting large data sets in context

Human-in-the-loop automated data annotation


Lyft is a leading ride-sharing organization valued at over US$11 billion. Undetected problems, such as passengers not being matched with rides in a timely manner, can cost the company revenue, customers, and market share. With the massive amount of data being generated by its mobile app traffic, Lyft knew that it needed an automated way to detect meaningful anomalies that could reveal the possibility of larger problems.

Lyft chose Anodot, built on AWS, as its solution. Anodot’s AI-powered analytics uses advanced machine-learning algorithms to overcome the limitations that humans bring to data analysis, identifying potential problems in real time without having to manually inspect multiple dashboards. Anodot automatically learns the data’s normal behavior through time series analysis, then identifies any deviations in real time, grouping and correlating multiple anomalies, and alerting on significant events.

Using Anodot and AWS, Lyft can track large amounts of metrics without having to configure thresholds. The ease of creating real-time alerts and the correlations between anomalies gives the customer the ability to identify issues in real time and to quickly understand the scope and the root cause of the issues.

We had heard a lot from other companies who had used AWS about the ease of use – especially how easy and economical it was to scale out, without any IT effort. It was a no-brainer for us to choose AWS.

“

”Ira CohenCo-founder and Chief Data Scientist, Anodot

8

Case study: Lyft Automated time series analysis for anomaly detection


Founded in 2000, Seattle-based Trupanion provides all-inclusive medical insurance policies for pet owners. In the relatively small world of medical insurance for cats and dogs, only about 1% of pet owners hold insurance policies. Most veterinarians expect pet owners to cover the full bill for their pets’ treatment, making the insurance claims process a manual and time-consuming effort.

Trupanion sought to build an accurate predictive data model that would help reduce the time and manual labor necessary to approve or deny insurance claims. The company identified DataRobot, an ML platform built on AWS, as a solution that could easily ingest data and create trainable algorithms to predict outcomes.

Using DataRobot and AWS, Trupanion was able to build highly accurate predictive models faster that significantly shortened time-to-insight. Trupanion’s claims process has been truly optimized for both claims processors and for policy owners, who now know whether a claim will be approved before leaving the veterinarian’s office – a significant customer service improvement over slower, more manual methods of claims processing.

Case study: Trupanion An automated machine learning platform built on AWS

9

We’re very data-focused. We pull in data from a bunch of different sources and use all of it to create value for our company. We needed a machine learning platform to increase the value we were creating, and DataRobot fit the bill perfectly.

“

”TJ HoukChief Data Officer, Trupanion


Deloitte US is a large professional services firm employing over 84,000 people worldwide. Deloitte often faces data overload: access to multiple, very large data sets without the time and resources to fully analyze that data for meaningful recommendations. Adding to the need for quick data analysis are the needs of Deloitte’s clients, who often approach Deloitte with a desire to innovate their businesses.

Deloitte engaged Narrative Science, an AWS Machine Learning Competency Partner, to help interpret data meaningfully using natural language generation (NLG). Narrative Science’s NLG platform, Quill, built on AWS, helps customers interprets large data sets meaningfully and in context.

Using Quill, Deloitte automates existing internal manual analysis and reporting processes. Quill also helps the company scale and improve customer communication for a variety of clients. Deloitte has brought its own industry expertise and deep knowledge of system and workflow integration to create a full end-to-end solution using NLG for its clients. AWS provides the cloud infrastructure that allows Deloitte to quickly spin up and deploy secure NLG solutions to its clients.

Deloitte’s use of Narrative Science’s NLG platform has already resulted in savings of more than half a million dollars for the firm’s internal operations. As of this writing, Deloitte is planning to deploy three additional NLG solutions internally in the coming months.

10

Quill can generate sophisticated, customized narratives from large data sets, and Narrative Science’s in-house NLG experts are dedicated to creating solutions at the scale and specificity our customers require.

“

”Sheetal ParikhSenior Manager of Innovation, Deloitte

Case study: Deloitte A natural language generation (NLG) platform that runs on AWS


Consensus Corporation, a subsidiary of Target, enables retail stores nationwide to sell bundled packages of technology and services together (such as a smartphone and a data plan) through one unified online platform. However, a major risk for retailers selling expensive devices and services is fraudulent customer activity. To identify potential fraud, Consensus built an advanced data model that leverages huge volumes of disparate data and undergoes routine updates. To constantly refine predictive models, Consensus sought out technologies that would allow it to prepare (“wrangle”) data faster for use in its machine learning models.

Trifacta Wrangler Pro, available in the AWS Marketplace, seamlessly accesses data stored on AWS, including Amazon S3 and Amazon Redshift, and leverages Amazon EMR (Elastic MapReduce) to process the data. Consensus used Trifacta to wrangle large amounts of structured historical data stored in Amazon S3–from many megabytes to gigabytes–much faster than with previous efforts.

Trifacta has helped solve the problem of uploading the most accurate data into Consensus’s fraud detection models quickly, without the cost and potential inaccuracies associated with relying on manual data preparation or traditional languages such as SQL, R, or Python. The result is far faster model development times and increased savings for Consensus’s customers.

11

Trifacta has helped reduce our overall model development time from six weeks to one week. This allows us to more quickly detect and alert retailers of suspicious activity that indicates fraud.

“

”Harrison LynchSenior Director of Product Development, Consensus Corporation

Case study: Consensus CorporationAutomated data wrangling for AWS


For more information on machine learning with AWS, visit:

• AWS featured Machine Learning Partners

• Machine Learning on AWS

• The AWS Machine Learning Blog

• What’s new on AWS

To learn more about AWS and to try AWS for free, visit aws.amazon.com.

Getting Started

12

https://aws.amazon.com/machine-learning/featured-partner-solutions/

https://aws.amazon.com/machine-learning/

https://aws.amazon.com/blogs/machine-learning/

https://aws.amazon.com/new/

http://aws.amazon.com/

ebook: machine learning with amazon web services...amazon rekognition deep learning-based image and...

Documents