amorphic documentation

Amorphic DocumentationRelease 1.0

Amorphic support team

Jun 03, 2020

Contents

1 SAAS Subscription 1

2 Amorphic Data Features 112.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.2 Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.3 Access Request and Group Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.4 Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.5 Role Base Access Control - (RBAC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.6 ML Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.7 Analytics Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.8 Ad-Hoc Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.9 ETL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.10 Schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.11 Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.12 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442.13 SAML Groups Application Roles Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3 Amorphic Data Frequently Asked Questions 513.1 Data Catalog and Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513.2 Security and Governance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.3 Amorphic Data ETL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533.4 Machine Learning and Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4 Amorphic Data Contracts 574.1 Amorphic Data Enterprise Terms of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

5 Amorphic Data RACI matrix 635.1 RACI Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

6 Incident Management Process 656.1 Amorphic Data Incident Management Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656.2 Amorphic Data Enterprise Support Tickets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 676.3 Amorphic Data Enterprise Chat Support Tickets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686.4 Amorphic Data Enterprise Email Tickets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686.5 Amorphic Data Enterprise Phone Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

7 Amorphic Data Tableau Integration 69

i

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697.2 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697.3 Tableau Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

8 Amorphic Data SaaS Billing 71

9 Amorphic Data Platform – SOC Compliance 739.1 1. Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739.2 2. SOC Compliance – Introduction & Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 739.3 3. SOC 1 vs. SOC 2 vs. SOC 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 749.4 4. Type 1 or Type 2 Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 749.5 5. SOC Compliance - SOC 1 Reports (Restricted Use Reports) . . . . . . . . . . . . . . . . . . . . . 759.6 6. SOC Compliance - SOC 2 Reports (Attestation Reports) . . . . . . . . . . . . . . . . . . . . . . . 76

10 Amorphic Data Platform – Security Policy 7910.1 1. Customer Data Access and Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7910.2 2. Encryption and Logical Separation of Customer Data . . . . . . . . . . . . . . . . . . . . . . . . 8010.3 3. Service Infrastructure Access Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8010.4 4. Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8010.5 5. Vulnerability Scanning and Penetration Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . 8110.6 6. Remote Access & Wireless Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8110.7 7. System Event Logging, Monitoring & Alerting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8110.8 8. System Administration and Patch Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 8210.9 9. Amorphic Security Training and Cloudwick Personnel . . . . . . . . . . . . . . . . . . . . . . . . 8210.10 10. Physical Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8210.11 11. Notification of Security Breach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8210.12 12. Disaster Recovery & Business Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8310.13 13. Amorphic Data Platform Security Compliance, Certifications, and Third-party Attestations . . . . 8310.14 14. Customer Responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

11 Amorphic Data Platform – OKTA Integration 8511.1 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8511.2 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8511.3 3. Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8511.4 4. Amorphic - Okta Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8611.5 5. Setting up Okta developer account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9111.6 6. Configuration of federated Identities in AWS Cognito using the Okta SAML assertion . . . . . . . 9611.7 7. Testing the OKTA integration with Cognito user pool . . . . . . . . . . . . . . . . . . . . . . . . 9811.8 8. Navigating to the Amorphic application in OKTA dashboard . . . . . . . . . . . . . . . . . . . . 103

ii

CHAPTER 1

SAAS Subscription

This document contains information related to installation and setup for Amorphic Data Cloud Analytics Platform.Before we get started, you will need your Amazon Web Services account logon information to logon to the AWSMarketplace and Search for “Amorphic Data”.

Step-1: From the product page of the AWS Marketplace, you can learn more about and subscribe to the AmorphicData Cloud Analytics Platform by clicking on “Continue to Subscribe” button.

1

Amorphic Documentation, Release 1.0

In the next page click on “Subscribe” button to get subscribed to this product and will be charged for your accumulatedusage at the end of your next billing cycle.

Step-2: To begin using this software, you will be redirected to the Amorphic Data Lake as a service – professionalwebsite. This is a customer management portal, which will assist in the installation process.

2 Chapter 1. SAAS Subscription


This is how the customer management portal looks like (Note: This will only assist in the installation process. Ap-plication platform will have a separate link). You will need to provide your full name and email address. The emailaddress will be used to send a verification link in order to verify the user. The verification email should contain theusername and temporary password to login to the Amorphic Data customer management portal. Note: You will beasked to change your password.

Step-3: Once you have the final username and password, you can login to the customer management portal. This ishow the customer management portal looks like:

3


You can head over to the Applications section on the left and click on “New Application” to launch the configurationsettings for the installation of new Amorphic Data applications. Note: Currently the platform supports only oneinstallation in a AWS Account.

Step-4: Once you are on the application configuration page, you will need to provide information related to theapplication:

1. Application Name: This will be the name of your application

2. Description: You can provide a short description of your application

3. Product Code: Each AWS subscription of Amorphic Data gives you a product code. This product code isavailable in the “Subscriptions” section of Amorphic Data. You can select the product code from the dropdown.

4. Project Name: Project name for the application

5. Region: You have to select the region in which the region specific services will get deployed.



6. Environment: You can provide an environment name for example, test/ dev/ prod.

7. DataWarehouse: You also have the option to select between Redshift and Aurora as your data warehouse for thedata lake.

8. Tier: You can select among the small, medium or large tiers for the data warehouse clusters.

5


9. Allow DWH Public Access: You have the option to allow public access to the redshift data warehouse. So incase where lets say you would like to use the data warehouse configuration details to connect to a quicksightdashboard in another aws account or a tableau dashboard for dashboard development purposes, you should selectyes so that the cluster has a public IP address that can be accessed from the public internet.

10. Enable WebACL: Select yes for enabling Web ACL in order to enable WAF on the application and allow theweb request for the application based on the IP addresses that the request originates from. If you do not want towhitelist IP’s for the product, then select No.

11. Portal Whitelisted IPS: You can provide your whitelisted IP address here.

12. Portal Allowed Registration Domain: Lastly, you can allow only users from a sepecific company domain as theusers of the platform. In order to do that, you can specify the domain here or select Any Domain in order toallow user registration from any domain.

Once you have provided all the required information, you can go ahead and click on Register button. Upon completionof the registration process, it might take some time to setup the AWS infrastructure for the data lake in the back end.Meanwhile, you can track the status of your deployment.

Step-5: Once the registration processes is complete and the application is fully deployed (Application Status - Ready),you can access application using the application portal URL.



Step-6: User Registration and Login for application:

As the above user was used for registration and application installation purposes in customer management portal, youwill have to register the user again in the application portal before you gain access to the application.

7


This user will have administrative privilages upon login to the portal. After the email verification, you can login as theuser and acess all the portal functionality. This is how Amorphic Data application platform looks like:



9

CHAPTER 2

Amorphic Data Features

Amorphic Data is a new cloud orchestration Saas or Managed Software platform that make it easier for IT,business and data science users to more easily access and manage AWS advanced analytics and machinelearning by stitching together these services side by side with traditional data pipeline.

Amorphic Data is the first cloud orchestration platform that delivers self-service for ingestion, data man-agement and storage for all and any data, orchestration for your compute jobs, enabling users to moreeasily take raw and processed data and apply any machine learning model for prediction and data-drivenanalytics, and in the end connect data to any visualization dashboard, report or tool. This provides a singleplatform for all users in one cloud data, advanced analytics and machine learning pane of glass.

Features:

Amorphic Data is the first orchestration platform to support the ingestion, transformation and analytics ofall structured, unstructured and semi-structured data specific to your use cases.

Once you have your data ingested into the Amorphic Data platform it automates with native machinelearning models to transcribe your PDF or audio files into text files to create a global and secure metadatacatalog that can power your search, sharing, analytics and machine learning capabilities.

Once you have a global catalog that provides for governance and compliance, you can establish rolebased access controls for users, groups and domains. You can also do a search across all the AmorphicData files stored on S3 or Redshift and you can use your original and or transcribed files to power yourbusiness intelligence and perform AI or ML. And this platform is offered as either a SaaS or ManagedOrchestration Platform. Following are the highlight of the features of the platform:

1. It can act as a single source of truth to power search, sharing, analytics and machine learning.

2. It can be used to create unstructured, semi-structured and structured Datasets.

3. It can be used as a development platform to test your ETL scripts using Glue Dev Endpoints orperform ML training on the Dataset created in the platform using Sagemaker Jupyetr Notebooks.

4. Once the training is done, it can be used to import Sagemaker ML models or import model fromAWS Marketplace.

5. It is integrated with AWS Transcribe and AWS Comprehend capabilities so that the user can runAuto ML components on audio and transcribed files in the Datasets.

11


6. User can run standalone ETL transformation jobs or pre-process or post-process ETL jobs for run-ning ML models.

7. You can also ingest data via JDBC or S3 connections on a scheduled based (time-based or on-demand)

8. The platform can be used to provide organization-wide permissions to different users from differentdepartments.

Following are the various features of Amorphic Data in details:

2.1 Dataset

Amorphic Dataset portal helps you create unstructured, semi-structured and structured Datasets. These Datasets canbe used as a single source of truth across the different departments of an organization. Amorphic Datasets helps inproviding complete data lake visibility of data.

Amorphic dataset provides capability to search with a Google-like search index through the Dataset Metadata.

Amorphic Dataset page consists of options to List or Create a new Dataset. Datasets are available to select in theAmorphic Dataset for listing purpose. You can sort through the Dataset using the Domain filters, Create new Datasetsor View Dataset details.

2.1.1 Domain Filters

Each Dataset is registered to a Domain. The Dataset domains need to be created in the Amorphic Administrationoptions.

12 Chapter 2. Amorphic Data Features


The Amorphic dataset has a hierarchical structure such that Files are associated with Datasets which are associated witha domain. Hence to create a Dataset, you need to first create a Domain using Amorphic Administration. Then create aDataset and upload new structured, semi-structured or unstructured files to the Dataset and upload the respective files.

2.1.2 Create New Datasets

You can create new Datasets in Amorphic by using the “New Dataset” functionality of Amorphic application.

2.1. Dataset 13


In order to create a new Dataset, you would require information like Domain, Connection type and File Type etc.Following are the main information required to create a new dataset.

Domain

Each Dataset is registered to a Domain. The Dataset domains need to be created in the Amorphic Administrativeoptions.

Connection Type

Select connection type as JDBC for JDBC connection, S3 for an s3 connection while API for all other types ofconnection.

A JDBC connection type will require you to select a JDBC connection from a list of Amorphic Connections (seeconnection section). You will also need to specify the table name from which the scheduler will run the data ingestionjob.



While for an S3 connection, you will need to specify a S3 connection and the path of the directory on which a schedulewill poll for new datasets on an on-demand or on a time basis. (check schedule section)

2.1. Dataset 15


File Type

This file type should be in sync with the ML Model supported file format option.

Apart from the various supported formats, you can also perform metadata extraction from the unstructured datasetusing auto ML functionalities which are integrated with AWS Transcribe and Comprehend services in the back end.(More details in ML Model Section)

Target Location

This can be s3 or Redshift. Amorphic can ingest the data into a Redshift warehouse. Redshift Data Sets require aschema file upload. You can access the connection details information once the dataset has been created. S3 Datasetsdo not require a schema file upload.

Target Location S3 Dataset



Target Location Redshift Dataset

As you can see from the above figure, there is “DWH Connection Details” for Redshift Datasets. This connectiondetail along with the “DW Credential” information (Group Management Section), can be used to connect to externalbusiness intelligence tools, for example, Quicksight for the purpose of visualization.

Keywords

This keyword information will be required to search through the Dataset listing page for the necessaryDataset. These keywords act as tags of the Dataset for search related purposes.

2.1. Dataset 17


Table Update Method

You can select Append data or Latest Record. Amorphic being a data lake is an append-only immutablemodel. You cannot delete an uploaded file but it can be used to retrieve only the latest file by selecting theLatest Record table update method.

2.1.3 Schema File Upload

This functionality provides automated schema extraction for structured data.

Datasets with Redshift as target location require a sample schema file to be uploaded. The application will automati-cally try to recognize the schema of the Dataset. This schema will be used to create the respective tables in the Redshiftwarehouse. The schema file can be a sample CSV data of a training dataset or any other dataset.

Once the schema file is uploaded, you can validate and edit the schema and publish the new Dataset.



2.1.4 Upload File

Once a dataset is created and schema is validated (if required), you can upload files to a Dataset manually or on ascheduled basis using a scheduler to ingest the data through a JDBC connection or an S3 connection.

You can easily transfer the original Datasets or the result Datasets of each analysis stage of each stage in and out ofplatform as your business requirements dictate. This can be done using the Dataset download functionality integrated

2.1. Dataset 19


into the system.

2.2 Search

This functionality helps to perform fast google like search across all metadata across Data Lake and more. You cansearch for your Dataset as well as request access to a Dataset. For example, some departments or user groups mighthave access to certain Datasets. You can check which Datasets you have access to as well as request access to a Datasetyou do not have access. You can also track your request access request.

2.3 Access Request and Group Management

You can also track the status of your in the Request Listing Page, accessible in the top right corner of the portal.You can use the Request Listing Page to check which dataset requests are pending as well as approve the request ofDatasets that you own.



You can use group management to check logged in users group.

In order to be an admin for a group, you need to be added as an admin to a group by the group administrator. Onceadmin privileges have been assigned to the user, he or she can assign admin privileges to other users, add or removemembers in a group and add or remove the Dataset which the group has access to.

You can also use Group Management to reset your data warehouse (Redshift or Athena) credentials. For example, youwould like to connect your Redshift Datasets to a Quicksight Dashboard. In order to ingest the data in Quicksight, youwill require Data Warehouse credential. These credentials are provided for one time save and use when the ADP useraccount is created. In order to access your credentials again, you can reset the DW credentials here.

2.3. Access Request and Group Management 21


2.4 Connections

Amorphic provides functionality to create connections to import Dataset through a JDBC Connection or S3.

2.4.1 Connection Type

You can perform data ingestion using a JDBC connection or a S3 connection.

2.4.2 Connection Name

Connection name for the connection

2.4.3 Description

Description of the connection



2.4.4 Authorized Users

Select the users who are authorized to perform data ingestion using the connection.

2.4.5 JDBC Connection Credentials

JDBC URL: Provide the JDBC url to ingest the data. Sample JDBC url:

jdbc:protocol://host:port/database-name

Username: Provide the username for connecting to the database

Password: Provide the password for connection to the database

2.4.6 S3 Connection Credential

Bucket: Bucket name

Access Key: Access Key

Secret Access Key: Secret Access key

Once a connection is created, using the connection, you can use schedulers to ingest the data on an on-demand or atime basis.

2.5 Role Base Access Control - (RBAC)

Role Base Access Control in Amorphic helps us to enhance user access management and operational efficiency. Thisfeature helps the system administrators to restrict user access to sensitive information and reduces the risk of databreaches. RBAC provides an efficient way of authorizing users while accessing various services and aligns with usermanagement compliance through out the company.

Amorphic RBAC is designed to continuously adapt any new service deployed to the application and support all userneeds.

Amorphic RBAC provides the following capabilities:

• User can have multiple roles attached and has the ability to switch between them to perform various actionsbased on his/her responsibilities.

• Customize user role permissions to a granular level.

• Flexibility to choose the type of Amorphic view upon login.

The following picture depicts the Role Management Console in Amorphic:

2.5. Role Base Access Control - (RBAC) 23


2.5.1 What is a Role?

Role is defined as a Job function or title which defines the authority level (Source : Wikipedia). Role has the followingproperties:

• A Role can have multiple users attached to it.

• A Role can have many permissions.

In Amorphic we have two types of Roles:

• System Roles : Which are provided by the application by default.

• Custom Roles : Created by users.

System Roles

Amorphic RBAC provides application users with two default roles namely “System Administrators Role”and “System Default Users Role”

• System Administrators Role

The primary goal of this role is to provide a hierarchy and differentiate between regular vsadministrator level users. This role consists full permissions for every service offered inAmorphic and can perform all the activities without any restrictions.

• System Default Users Role System default Users Role is a basic application access role which isprovided to every user. This role consists a list minimal permissions for the user to navigatethrough and understand the application.

Note: This is not to be confused with user’s default role. User’s default role determines whatall services that he/she sees when logged in.

Custom Roles

Amorphic RBAC provides application users flexibility to create customized Roles by selecting permis-sions from a list of fine grained access permissions for each service. Example: User can create customData scientist role and provide access to ML notebooks only. Please check the how to create new sectionfor more details Create Role

Amorphic RBAC Role contains the following information:



Role Metadata Information

Type DescriptionRoleName

Role Name, which uniquely identifies the functionality of the role.

RoleDescription

A brief explanation of the role typically the functionality for what it is used.

PermissionsA permissions is an action defined for a particular service. Each Role consists of a group of per-missions. These permissions determine the level access within a service offered in Amorphic. Apermission can be assigned to multiple roles and vice versa.

UsersAttachedThe list of users to whom the the role is attached to.CreatedByUser who created the role.LastModifiedByUser who has recently updated the role.LastModifiedTimeTimestamp when the role was recently updated.

2.5.2 How are roles associated to an user

As part of Amorphic RBAC, every user is provided with one default Role called User Default-Role which providesbasic application access. Other than the default role, User can be attached to a Administrator created custom role.Each user can have one or more roles based on the level of responsibilities.

User has the ability to switch between roles to perform various activities and can choose his/her default access role forquicker access to Amorphic services.

2.5.3 Role Operations

Amorphic RBAC along with the basic CRUD (Create, Read, Update and Delete) operations for a role, it provides awide range of operations

• Create Role : Create a custom role by choosing from a list of permissions and attach to a User.

• View Role : View existing Role Metadata Information

• Update Role : Update an existing role.

• Delete Role : Delete an existing role.

• Switch Role : This functionality helps user to switch between multiple roles attached.

• Update user default-role : Helps user to customize the landing page view. Example: if User frequently usesMachine learning services, one can pick say a “Data scientist Role” as default login view for quicker access.

Create Role

You can create new Role in Amorphic by using the “New Role” functionality of Amorphic application.

In order to create a new Role, you would require information like Role Permissions and User names who are attachedto the role. Following are the main information required to create a new role.

2.5. Role Base Access Control - (RBAC) 25


View Role

If the user has sufficient permissions to view a role, He/She can view all the existing role information by clicking onthe Role Name under the Roles section inside Management Menu.

Please follow the below animation to view the role information in detail

Update Role

If the user has sufficient permissions to update a role, He/She can view all the existing role information by clicking onthe Role Name under the Roles section inside Management Menu and by clicking on the Edit Role option from theleft side Role Actions drop down menu. This will re-direct you to a different page where you can start editing any ofthe Role metadata.

Please follow the below animation to update the role information in detail

Delete Role

If the user has sufficient permissions to delete a role, He/She can view all the existing role information by clicking onthe Role Name under the Roles section inside Management Menu and by clicking on the Delete Role from the leftside Role Actions drop down menu.

Please follow the below animation to delete the role.

Switch Role

Switch Role functionality is enabled for users with more than one Role attached to them. This functionality can beaccessed by clicking on the User Profile icon and Switch Role item of the the drop down menu. Users will be presentedwith a drop down list of roles that He/She is attached to from where we can pick one for switching.



Please follow the below animation to switch between roles.

Update user default-role

Update Default Role functionality is enabled for users with more than one Role attached to them. This functionalitycan be accessed by clicking on the User Profile icon and Profile & Settings item of the the drop down menu. You willbe taken to the ‘User Profile’ page where you find the ‘default Role’ field. On clicking the ‘Change’ button besideit, Users will be presented with a drop down list of roles that He/She is attached to from where we can pick one forswitching.

Please follow the below animation to update the user’s default role.

More sources: Learn more on RBAC

2.6 ML Model

Amorphic Model portal helps you create/import a SageMaker model that you can apply on a Dataset created in theAmorphic Portal.

2.6.1 How to create an AD model object ?.

Following are the steps or information required to play around with the Amorphic model functionality:

Model Name

This is the model name in the Amorphic portal.

2.6. ML Model 27

https://en.wikipedia.org/wiki/Role-based_access_control


Description

This is the description of the model

Model Resource

There are three ways to integrate Sagemaker model with the Amorphic portal

Existing Model Resource

This is a way to import Sagemaker Marketplace subscribed model in the Amorphic Portal. In order toimport a model from Sagemaker marketplace, raise a request to the Admin. Currently, the platform wouldrequire you to subscribe the model from the AWS console after which the model will be available in theAmorphic Portal.

Artifact Location

This is a way to upload your sagemaker model file directly from any public s3 location. Currently, this functionalityis not available. We will update once the functionality is available.

Select file

This is a way to upload a sagemaker model tar file directly to the Amorphic portal. Upon selecting this option you canupload any model tar or tar.gz file directly into the Amorphic portal.

Output Type

You have two options - Dataset Data or Metadata. Select Dataset data when the requirement is to run the model ona Dataset created using the Amorphic Dataset functionality. Select Metadata, when you would like to run “Analytics



Metadata - Auto ML” on a dataset (explained later). Most of the time you would want to use Dataset data.

Input Schema and Output Schema

Dataset Data would require two additional inputs - Input and Output Schema.

Input Schema

This is the schema that should match the schema for the Dataset on which the pre-processing ETL job or the model isto be run.

Output Schema

This is the schema that should match the schema for the Dataset on which the post-processing job or the model outputwill be saved.

Both the schema should have the same following format matching the respective Datasets:

[{“type”:”Date”,”name”:”CheckoutDate”,”Description”:”a”},

{“type”:”String”,”name”:”MajorProdDesc”,”Description”:”a”}, {“type”:”Double”,”name”:”counts”,”Description”:”a”}]

2.6. ML Model 29


Algorithm Used

The platform currently supports only three models - XgBoost, Seq2Seq, and Deep AR.

Supported file formats

Select the respective file formats as per the Content-Type required in the Sagemaker batch prediction. If you requiremore than the available format, then select the others file type. It will default to no Content-Type for batch predictionpurposes. Note: if a model is selected as “Others” file type, then it can only be run on a “Others” type Dataset.

Preprocess Glue Job

Select the preprocessing ETL jobs created using Amorphic ETL functionality.

Postprocess Glue Job

Select the post process ETL jobs created using Amorphic ETL functionality.

2.6.2 Ml Pipeline

The above figure shows how a typical ML pipeline of Amorphic platform looks like. The pre-processing and post-processing ETL job functionality provides a way to drag and drop ETL workflows for a smooth user access.

2.6.3 Notebook

Amorphic platform also provides a way to host Jupyter/IPython notebook. You can set up or create a new notebookinstance and use your IPython notebook to perform model training. You can call Python Sagemaker SDK to create atraining job. Once a training job is created, you can use the S3 model location information to create a model in theAmorphic portal.



For accessing the Datasets inside the IPython notebooks, you can check the Dataset details for their S3 locationinformation (Check Dataset Section). The notebook has access to “cdap-us-east-1-802489652583-master-dlz” and“cdap-us-east-1-802489652583-master-ml-temp” bucket. All files related to datasets are located in the “cdap-us-east-1-802489652583-master-dlz” bucket while “cdap-us-east-1-802489652583-master-ml-temp/ml-models” can be usedto create a training job and thus a model tar file. This model file location can then be used to create a model using“Artifact Location” of Amorphic model (see model section). You can use the S3 location mentioned here to read thefiles related to training dataset and save the output model tar file.

2.6.4 Run Analytics

Once the model is created, you can run a model on a Dataset in the Amorphic portal by going through the followingsteps:

1. Select a Dataset in the Amorphic portal.

2. Go to the “Files” tab and select the file on which you want to run the model.

3. Click on the Top Right options for the file.

4. Click on “Run Analytics”

5. Select the ml-model from the model dropdown. All models that match the input schema of Datasets will beavailable to select.

6. Select the required instance types. Note: certain aws marketplace subscribed models run on specific instancefamily type.

7. Select the Target Dataset. The Datasets matching the output schema of the model will be available for selection.

8. Click on “Run Analytics”

2.7 Analytics Metadata

This functionality helps in extracting metadata information from unstructured data. This helps in automatically con-verting audio datasets into transcripts and performing entity recognition and sentiment analysis on text documents ie.the transcripts. This is integrated with AWS Transcribe which is an audio translation service and AWS Comprehendwhich automatically performs sentiment analysis on text/pdf documents.

2.7. Analytics Metadata 31


You can use both the transcribe and comprehend service or can choose to use only one of those services as per yourrequirement. For example, you can use transcribe service to convert your audio file to transcripts and then run aSeq-2-Seq model to perform prediction on the text dataset.

Moreover, if the default model that gets automatically configured for Audio to Speech conversion and Text Analysis



does not suit your needs, you can always use the platform to upload your own model. The uploaded model can beused to perform “Run Analytics” on audio/text datasets.

2.8 Ad-Hoc Query

The platform is integrated with AWS Athena. Amazon Athena is a serverless interactive query service that makes iteasy to analyze data in Amazon S3 using standard SQL. You can generate a sample query from your existing Datasetsusing the Filters on the left.

Once the query is run, you can view the results in the platform or download the results and use them outside theplatform.

You can also track the list of queries executed by you in the platform and download the respective results.

2.8. Ad-Hoc Query 33


2.9 ETL

2.9.1 ETL Jobs

Amorphic Data ETL is a platform for pre-processing and post-processing jobs on model input and output respectively.You can also create standalone ETL jobs in a Dataset in the portal.



Standalone ETL

You can create a standalone ETL job on a Dataset created in the portal. Amorphic ETL is based on AWS Glue platformin the back end. So you can call the Dataset via referring to them using the Dataset s3 location found in the AmorphicDataset details.

ML Pipeline

You can use Amorphic ETL pre-processing and post-processing jobs as a part of an ML pipeline. While creating anML model you specify the preprocessing and postprocessing job to use. You also specify the Input and Output schemafor the input dataset on which the preprocessing ETL job will be run and output dataset on which the post-processingjob will dump the dataset.

When running an ETL script, you have only three arguments/job parameters available:

1. originalFileObjectKey: Name of original file object or Dataset file on which the job is run.

2. inputLocation: S3 location of the input Dataset for the job. This can be used to read all the data files from theinput s3 location of the Dataset.

3. outputLocation: S3 location of the output Dataset for the job. This is where all ETL output data will be writtento. This will be the location of the target dataset if the ML job is run on a Dataset.

Right now the platform does not support any other job input parameters.

When you run a model on a Dataset, you by default select the input to the current Dataset and output to the targetdataset specified during the run.

The Amorphic ETL supports python and spark. You can use the boto3 library in ETL jobs.

2.9. ETL 35


2.9.2 How to execute an ETL job?

1. Select the ETL job from Amorphic ETL.

2. Click on “Execute” ETL job.

3. Input the job configuration and click on Submit.

4. The job will start running in the back end.

5. You can track the status of execution of the job in “Execution Status” tab.

6. The output and error logs are available for every job run in the “Execution Status” tab.



2.9.3 Glue Dev Endpoints

Apart from providing a platform to host ML Notebooks and ML Model Objects, the Amorphic platform also providesthe ability to host a glue development endpoint. A glue development endpoint is an environment that you can use todevelop and test your AWS Glue Scripts. Hence you can use a glue dev endpoint to test your ETL scripts and MLNotebooks to train your model. In the end, you can deploy Glue scripts as ETL transformation jobs and ML Modelsas Objects as a part of an ML pipeline.

2.9. ETL 37


In order to create a glue development endpoint in the platform, following are the information required:

Glue Endpoint Name

The unique name that you give the endpoint when you create it.

Description

Brief description of the endpoint.

Capacity

Relative measure of processing power.



Extra Python Libs S3 Path

Comma-separated Amazon Simple Storage Service (Amazon S3) paths to Python libraries that are required by yourscript.

Extra JARS S3 Path

Comma-separated Amazon S3 paths to JAR files that are required by the script.

Public Keys

Current public SSH keys that are associated with the development endpoint (optional). If you provided a public keywhen you created the development endpoint, you should have saved the corresponding SSH private key.

You can generate the key using ssh-keygen -t rsa -C “[email protected]”.

The format of the key generated in the file will be as following:

ssh-rsa <key> <email>

You can use <key> as a public key in the platform to create an endpoint.

Once the endpoint is generated, you can connect to the endpoint and test your ETL scripts by selecting the appropriateendpoint from the dev endpoint list in the “Dev Endpoint” section of Amorphic platform.

2.10 Schedule

Amorphic Schedule provides a self-service data ingestion mechanism which provides a mechanism for batch andstreaming data ingestion on a scheduled basis. You can schedule an ETL job or a Data Ingestion job - time-based oron an on-demand basis.

2.10. Schedule 39

mailto:[email protected]


For the ETL-job you need to provide an ETL script that will ingest the data into the output Datasets S3 location. Note:this does not require a target dataset.

For the data ingestion, you need to provide a target dataset. The target dataset must be created with “Connection Type”



as “JDBC”(Check dataset section).

You can schedule the job to be run on a time-based event or on an on-demand basis. Time-based events require sschedule expression.

On-Demand schedules can be run as per the need. You can disable a schedule to stop running the time-based scheduleas per requirement. Time based schedules require a schedule expression which should be a cron expression like“cron(cron-expression)”.

2.11 Administration

You can use ‘Amorphic Administration’ to create domains for datasets, check system users, change user access toadministrator, check user dataset access details and check usage report of individual AWS services used by yourAmorphic application.

2.11. Administration 41


2.11.1 Users

The user can access the user list from the Administration section. The user can also check Dataset accessdetails (Full Access or Read Only) of each user. But only administrative user will have access to changingthe roles (Administrator or User) of other users.

You can use Amorphic Data Administration to create Domains for Datasets, check system users, change user accessto administrator, check user dataset access details and check usage report of individual AWS services used by yourAmorphic Data application.



2.11.2 Domains

Each Dataset in Amorphic is registered to a domain. Datasets have a hierarchical structure such that files are associatedwith Datasets which are associated with a domain. Hence to create a dataset, you need to first create a domain using‘Administration’. Then create a Dataset and upload new files to the Dataset.

2.11.3 Reports

Reports provides an ability to send out email to users about their access status per dataset.

2.11.4 Inactive Datasets

This is a list of datasets that are marked for deletion by the users.

2.11.5 System Health

A user can check the health and availability of different underlying system components.

2.11.6 Usage

Users can track the usage and cost of various underlying AWS resources.

2.11. Administration 43


2.12 Settings

2.12.1 Profile

Settings consist of all the information and setting related to the logged in user.

The user can subscribe or unsubscribe from alert preferences. Users can also create groups that can cater to differentdepartmental functions of an organization.

2.12.2 Groups

The user can select all the groups available in the amorphic platform from the Groups section. This group list isreadable for every user.While only admin users (refer to Administration section) have access to changing the additionalprivileges of the group.



The user can select the group type - Full Access or Read Only. Full Access group type has full access to all the datasetsin the group while Read-Only group type has access to read-only privileges of the Datasets in the group.

2.12. Settings 45


Once the group is created, the user can edit Datasets in the group to add/remove any datasets in the group, add orremove group admins, add or remove members to the group or delete the group.



2.12.3 Reset DWH Credentials

The user can use settings to reset your data warehouse (Redshift or Athena) credentials. For example, you wouldlike to connect your Redshift Datasets to a Quicksight Dashboard. In order to ingest the data in Quicksight, you willrequire Data Warehouse credential. These credentials are provided for one time save and use when the ADP useraccount is created. In order to access your credentials again, you can reset the DW credentials here.

2.12. Settings 47


2.12.4 Change Password

The user can use settings to change the login password.

2.13 SAML Groups Application Roles Mapping

SAML (Security Assertion Markup Language) is an XML-based standard for exchanging authentication and autho-rization data between an identity provider (IdP) such as Okta, and a service provider (SP) such as Amorphic DataPlatform In Amorphic Data Platform, Users/API calls get authenticated using Cognito. The authentication token re-cieved has the groups embedded in it. These groups will have role assigned to it Amorphic application. User will begranted access to the role based on the groups assigned to him in identity provider(IdP) such as Okta.

Amorphic provides SAML Groups with access to application resources through its roles. SAML Groups are thegroups in an Idp which contains a list of users. To delegate the users in the group with certain set of permissions in theAmorphic application, map the group with the application role. To know more about the roles in Amorphic applicationroles click on the link Amorphic application ROLE BASE ACCESS CONTROL

2.13.1 What is a SAML Mapping?

A SAML mapping is a way of assigining a SAML Group with a role in Amorphic application. An administrator in theAmorphic application will have permissions to perform this operation.


https://amorphicdata.readthedocs.io/en/latest/base/amorphic-data-features/amorphic_rbac.html


SAML Mapping Metadata Information

Type DescriptionSamlGroupId SAML Group name which the administrator has to enter manually.RoleId Id of the role which will be used by the users of the group. Administrator selects the name of the

role from the drop down.CreationTimeTimestamp when the mapping was created.CreatedBy Administrator who created the mapping.

2.13.2 SAML Mapping Operations

Administrator of the Amorphic Application can add a mapping, edit or delete an exisiting mapping.

• Add New Mapping : Add a new mapping by entering a SAML group name and choosing a role name from thedrop down.

• Edit Mapping : Edit an existing mapping

• Delete Mapping : Delete an existing mapping

Add New Mapping

You can add a new mapping in the Amorphic application by using the “Add New Mapping” functionality.

In order to add a new mapping, you need to be an administrator in the application. Below is the image that shows howto add a new mapping.

Edit Mapping

You can edit an existing mapping. You can change the role associated with the group but not the other way. To changethe group name delete the existing mapping and add a new one.

Below is the image that will show how to edit a mapping.

Delete Mapping

You can delete an existing mapping.

Below is the image that will show how to delete a mapping.

Note

Below are some the important points that the Amorphic administrator needs to keep in mind when a mapping is added or deleted.

• If a user is a part of a SAML group and there is a mapping of the SAML group to an application role whichthe user already has access to will loose the access to role when the mapping is deleted or when the user isremoved from the SAML group.

2.13. SAML Groups Application Roles Mapping 49


• This only doesn’t apply to default-role i.e., if there is a mapping between a SAML group and a default-roleand when a user has been removed from the SAML group or the mapping in the Amorphic application isdeleted. The user won’t loose access to the default role. Had it been some other role he would have lostaccess to it.


CHAPTER 3

Amorphic Data Frequently Asked Questions

The following section describes the frequently asked questions

3.1 Data Catalog and Datasets

1. What is a ‘Dataset’ in Amorphic Data? A. Dataset is logical name given by a Amorphic user to a group of one moresame kind of files. The supported file types are:

a. CSV files

b. Excel documents

c. Image files

d. Video files

e. Audio Files

f. PDF documents

g. TXT documents

2. What is the difference between ‘Public Description’ vs ‘Internal Description’? A. Public Description is visible toall the users in the platform whereas Internal Description is visible to owners and viewers of that data set.

3. Can I limit the data set search results based on my choice? A. Yes, You should generally choose ‘public’ unless youhave a very specific reason to hide information. Amorphic Data promotes the reuse of data and does this by makingdata visible to a wide range of users. Definitions:

Public: All fields are visible in search results. Private: Only the public name and description of the datasetare visible in search results (all other fields are private to the owners and viewers of the dataset). None:Hides all fields from search results.

4. What is a ‘Domain’ in Amorphic Data? A. ‘Domain’ is analogous to a ‘Business Unit’ in traditional system. It canbe simply as a group of data sets for a specific need.

5. What are the ways to import data to Amorphic Data? A. API based: You can upload a file directly in browser

51


JDBC: You can connect to any of the JDBC sources S3: You can connect an existing AWS Account’s S3

6. Can I tag my data set? A. Yes, you can tag the data set while registering your data set. You can provide a list of tagsin a comma separated format under ‘Keyword’ section

7. Can I append data to my existing data set? A. Yes, Amorphic Data, being a data lake, is an append-only model bydefault. This means that you can add data to your data set but never remove. Data in the data lake is immutable. Often,you only care about the most recent records for a given key and not the whole history (such as with transactionaldata from an ERP system). Amorphic can automatically create a view of your data which will show you only themost recent records and not the entire history. Select ‘Latest Record’ to have Amorphic create such a view for you.You must also select that your dataset is for analytic purposes. Later, after defining the schema, you will be asked toindicate which fields form the keys for your records and which indicate the latest record

8. Can I tag my data set to indicate that the data set contains confidential information? A. Yes, At the time of Dataset registration, if your data contains Personally Identifiable Information (PII) or Credit Card data (PCI), please flagit under ‘PROTECTED DATA’. Please take extra care with these kinds of data and consider carefully if they shouldbe uploaded to Amorphic Data. Check with your local Information Security team if you have any questions. You areresponsible for the datasets that you create and the data that you upload to Amorphic Data.

9. Can I get notifications for my data sets? A. Yes, there are 2 ways you can set your notification preferences:

All: You will receive emails for both success and failures uploading data to Amorphic Data. Error: Youwill receive emails only when an uploaded file fails to load to Amorphic Data. By default, you will getnotifications about Access Request, Access Grant etc.

10. Can I create ‘New Domain’ in Amorphic Data? A. The create ‘New Domain’ activity is limited to the administratorof the platform. Please contact your ‘admin’ for adding a new domain to the platform.

11. Can I modify the data dictionary after I create a ‘data set’? A. Yes, You have the ability to modify some fields inthe data dictionary such as:

Public Description, Internal Description, Protected data tags and Search Visibility.

12. How many files can be ingested to a data set? A. There are no limits on how many files can be added to a data set.

13. Can I delete a data set? A. Yes. You can tag a data set for deletion if you have ‘OWNER’ access the data set. Thephysical deletion is carried out by an administrator.

14. Is there any limit on how many data sets can be created? A. No. There is no limit on the no of data sets.

15. How can I connect my Business Intelligence tool to analyze and visualize my data? A. ‘Connection Details’ underthe data set details provides the detailed instructions to connect to that data set.

Connection details will have the following information Connection-string Host Port Database Table Name

3.2 Security and Governance

1. What kind encryption is available on Amorphic Data?

By default, ‘Encryption at Rest’ and ‘Encryption in Motion’ is enabled in the platform. Amorphic Data uses AWSKey Management Store to enable encryption at rest.

2. What are the access control mechanisms for Amorphic Data users?

Amorphic data provides ‘Role Based Access Control’ mechanism for the users. You can define access controls atvarious levels to achieve fine grained control over the data.

Security controls available at

1. Data Set Level

2. Domain Level

52 Chapter 3. Amorphic Data Frequently Asked Questions


3. User Level

4. Group Level

3. Can I restrict Amorphic Data access to a set of ‘IP’ addresses?

Yes. You can provide a list of IP addresses to be whitelisted at the time of Amorphic Data deployment.

4. Is Amorphic Data uses a multi-tenant environment?

No. Amorphic data will be installed to a dedicated AWS account for each customer.

5. What is the ‘Password’ strength for Amorphic Data?

Passwords must be at least 8 characters but fewer than 99 characters, Require numbers, Require a special characterfrom this set:^ $ * . [ ] { } ( ) ? - ! @ # % & / \ , > < ‘ : ; | _ ~ ‘ Require uppercase letters, Require lowercase letters.

6. Can I change the role of user in Amorphic Data?

Yes. Administrators can change the role a user in Amorphic Data.

7. Can I remove a user from ‘Amorphic Data’?

Yes. Administrators can remove users from ‘Amorphic Data’.

Is Amorphic Data running on a web server?

No, Amorphic Data is a ‘Single Page Application’ deployed as a serverless model hosted in ‘Simple Storage Service(S3)’ on your dedicated SAAS AWS account.

8. How can I see my current usage and billing estimate?

Administrators can view the current usage under Administration->usage.

3.3 Amorphic Data ETL

1. What is Amorphic Data ETL?

It’s an extract, transform, and load (ETL) service that automates the time-consuming steps of data preparation foranalytics.

2. What can I do with Amorphic Data ETL service?

You can generate ETL code to transform your source data into target schemas, and run the ETL jobs on a fullymanaged, Apache Spark environment to load your data into its destination. It uses AWS Glue on the backend and thusit automates the setting up and management of the clusters for an Apache Spark environment.

3. How do I get started with Amorphic Data ETL capabilities?

To get started with CDAP ETL, login to Amorphic Data and you can create a new job by providing some AWS Gluerelated infrastructure information. The job will let you edit your ETL script on the Amorphic Data.

4. What if I have my own ETL script?

Upon creation of a job, you can bring your own AWS Glue ETL script in the platform using the Upload script func-tionality. Keep in mind that AWS Glue supports python and spark environment.

5. How is the ETL component integrated with the rest of Amorphic Data services?

The ETL jobs created in the Amorphic Data can allow you to perform ETL on a dataset created in the CDAP Datasetplatform on a standalone basis or can help you create preprocessing and postprocessing ETL jobs for purpose ofrunning a ML model.

6. How can you run ETL job in a Dataset in Amorphic Data?

3.3. Amorphic Data ETL 53


Every Dataset in Amorphic Data provides you the connection information (ie. s3 location) which you can use insidethe ETL script to run ETL job in standalone Datasets.

7. How can you use ETL jobs when running ML model in Amorphic Data?

In model creation process, you specify the preprocessing and postprocessing ETL jobs in Amorphic Data. The pre-processing and postprocessing ETL jobs will run on the respective Dataset matching the input and output schema asspecified during the model creation process.

8. How do I get started with creating ETL scripts in Amorphic Data?

Any ETL job by default comes loaded with a set of libraries and spark context that you can use to develop your ETLscript. You can add on more code in the script as per your ETL job requirement.

9. How do I access input dataset and save in output dataset in Amorphic Data ETL service?

By default, every script accepts three arguments:

1. originalFileObjectKey: This is useful when running post processing job with an ML model and you would liketo access the name of original file on which the model was run.

2. inputLocation: This is useful when you would like to access or set the input location of the dataset on which theETL job is run.

3. outputLocation: This is useful when you would like to access or set the output dataset location in which theETL jobs will save the results.

For the purpose of ML model, the above parameters can be useful in identifying or setting the input and output locationof data in preprocess and post process state respectively.

10. Can I call the input datasets from AWS Glue Catalog?

Since the Datasets are also registered as AWS Glue Catalog, you can create Spark Dataframes from the AWS GlueCatalogs inside the ETL scripts.

11. What are additional functionalities provided by the Amorphic Data ETL service?

Apart from a sample code, the ETL service also provides a sample code generation feature which generates a codesnippet corresponding to the transformation selected.

12. How do I provide ETL job run related information?

Upon clicking on the ETL job, you can click on Job Actions > Edit Job Detail to enter AWS Glue execution relatedinformation like Allocated Capacity etc. Since the service runs on AWS Glue, you can further look into AWS Gluedocumentation for information relating to job tuning.

13. How do I execute an ETL job?

In the ETL job page, there is an execution functionality (left to “Job Actions”). Upon clicking, you provide the job-related information and click submit. You will be redirected to the Execution page which will track the status of yourETL application.

14. Why is my Execution status for ETL job run not refreshing?

You can click refresh on individual job run to know the current status of your job run.

15. What happens after an ETL job is run successfully?

On successful ETL job run, the result data is ingested to the respective Dataset in the Amorphic Data Dataset cor-responding to the script and the output location. The Dataset Redshift tables gets appended with new data in thebackend.



3.4 Machine Learning and Neural Networks

1. What is Amorphic Data Analytics?

Amorphic Data Analytics is a fully managed service provided by the platform that enables scientist and developers toquickly and easily build, train and deploy machine learning models.

2. What can I do with Amorphic Data Analytics?

You can create new machine learning model or create a new notebook to perform modelling on a dataset created in theDatasets tab.

3. How do I get started with Amorphic Data Analytics?

To get started with Amorphic Data Analytics, login to Amorphic Data and you can follow two approaches - ML Modelsand Notebooks. ML models provide you a way to bring your existing models into the Amorphic Data platform and runthem on a dataset created in the Amorphic Data. In Notebooks - you can launch a notebook instance with an examplenotebook, modify it to connect to your data sources and then build, train, validate, and deploy the resulting model intoproduction with just a few inputs.

4. What if I have my own training environment?

Amorphic Data provides a full end-to-end workflow in which you can perform ETL as well as train and host an MLmodel, but you can continue to use your existing tools. You can easily transfer the results of each stage in and out ofAmorphic Data as your business requirements dictate.

5. What is ML models component of Amorphic Data?

ML models provide you a way to bring your existing models into the Amorphic Data and run them on a dataset createdin the Amorphic Data.

6. How do I get started with the ML models component of Amorphic Data?

You can get started by adding a new model from the ML Model page of Amorphic Data.

7. What are the ways in which you can create a model with the ML models component of Amorphic Data?

There are three ways you can create a model from the ML Model page of Amorphic Data - you can use AWS Market-place Subscribed model or any model artifact s3 location or upload an AWS SageMaker model tar file to create yourmodel.

8. How can I use an AWS Marketplace model in the Amorphic Data?

Upon subscribing the ml model from AWS Marketplace using the AWS credentials provided by the Amorphic Data,the model resource is available in the “Existing Model Resource” dropdown in “Register New Model” page. You canuse the model resource to create a new model after providing other information regarding the model.

9. How can I bring my own AWS Sagemaker model into the Amorphic Data?

In the “Register New Model” page of the Amorphic Data, you can import your existing AWS Sagemaker model tarfiles to create the model.

10. What other information do I need to create a model?

You need to know whether the output of the run analytics model on the dataset will be just the metadata or a Datasetdata. Most of the time the output will be a Dataset data. You also need to provide information related to algorithm.Currently, the Amorphic Data supports only three type of algorithms - AWS XGBoost, Seq2Seq and DeepAR algo-rithm. You need to provide the supported file format of datasets in which the model will be run. You also need toprovide the preprocess and post process ETL job created in the Amorphic Data ETL.

11. How is Amorphic Data ETL connected to the Amorphic Data ML?

3.4. Machine Learning and Neural Networks 55


The Amorphic Data ETL provides you the capability to run preprocessing and post processing ETL jobs in the MLpipeline. With preprocessing, you can perform ETL required to convert the original dataset into the format requiredto run the ML model. With post processing you can perform ETL operation required to convert the prediction resultsfrom the model into an output dataset format.

12. What does output type Dataset data mean in the “Register New Model” page of the Amorphic Data?

Dataset data provides you the capability of specifying the input and output schema for the preprocessing and postpro-cessing ETL jobs. This means that Datasets from the Dataset listing page of Amorphic Data will be used to performML as well as ingest the ML output data.

13. What is the format of input and output schema in the “Register New Model” page of Amorphic Data?.

The format of the input and output schema is an array of JSON objects (key value pairs) containing the column type(String, Integer, Double, Date), name of column and the description.

Following is the sample format:

[{“type”:”String”,”name”: “Vin”,”Description”:”a”}, {“type”:”String”,”name”: “Cus-tomerId”,”Description”:”a”}, {“type”:”String”,”name”: “CheckoutLocation”,”Description”:”a”}]

14. How does the input and output schema of output type dataset relate to the preprocess and post process ETL jobin the Amorphic Data ETL?.

The input schema should match the schema of the dataset (from the Amorphic Data Dataset) on which the preprocess-ing ETL job will be run. Output schema should match the schema of the dataset (from the Amorphic Data Dataset)which will ingest the post process results from the post processing ETL job run on the ML model output. You createthe preprocessing and post processing using the Amorphic Data ETL.

15. How do I run the model created in the Amorphic Data ML?

You can run the model created in the Amorphic Data ML by selecting the dataset from the dataset listing page of theAmorphic Data Dataset. Upon selecting the dataset, go to the respective files in the dataset and from the top rightcorner of the file click on “Run Analytics”.

“Run analytics” will provide you the option to select the ML model you want to run on the dataset. Note that onlycertain ML models will be available to run on the dataset. The available models will be the one which match theschema of the dataset to the preprocessing input schema as specified during the model creation.

Apart from that you will also be given the option to specify the target dataset. The target dataset is the dataset that willingest the model output after the post processing ETL job. Only those target datasets will be available for selectionwhich match the output schema as specified during the model creation.


CHAPTER 4

Amorphic Data Contracts

4.1 Amorphic Data Enterprise Terms of Service

PLEASE READ THESE ENTERPRISE TERMS (“TERMS”) CAREFULLY BEFORE USING THE SERVICESOFFERED BY CLOUDWICK TECHNOLOGIES, INC. (“CLOUDWICK”). BY MUTUALLY EXECUTING ONEOR MORE ORDER FORMS WITH CLOUDWICK WHICH REFERENCE THESE TERMS (EACH, AN “ORDERFORM”), YOU (“CUSTOMER”) AGREE TO BE BOUND BY THESE TERMS (TOGETHER WITH ALL ORDERFORMS, THE “AGREEMENT”) TO THE EXCLUSION OF ALL OTHER TERMS. IN ADDITION, ANY ONLINEORDER FORM WHICH YOU SUBMIT VIA CLOUDWICK’S STANDARD ONLINE PROCESS AND WHICHIS ACCEPTED BY CLOUDWICK SHALL BE DEEMED TO BE MUTUALLY EXECUTED. IF THE TERMSOF THIS AGREEMENT ARE CONSIDERED AN OFFER, ACCEPTANCE IS EXPRESSLY LIMITED TO SUCHTERMS.

1. Order Forms; Access to the Service. Upon mutual execution, each Order Form shall be incorporated intoand form a part of the Agreement. For each Order Form, subject to Customer’s compliance with the termsand conditions of this Agreement (including any limitations and restrictions set forth on the applicable OrderForm) Cloudwick grants Customer a nonexclusive, limited, personal, nonsublicensable, nontransferable rightand license to internally access and use the Cloudwick product(s) and/or service(s) specified in such OrderForm (collectively, the “Service,” or “Services”) during the applicable Order Form Term (as defined below) forthe internal business purposes of Customer, only as provided herein and only in accordance with Cloudwick’sapplicable official user documentation for such Service (the “Documentation”).

2. Service Provision and Access; Client Software. Cloudwick will make the Service available to Customer forthe Subscription Term solely for use by Customer and its Users in accordance with the terms and conditions ofthis Agreement, the Documentation, and the Order Form. Customer may permit its Contractors and Affiliates toserve as Users provided that any use of the Service by each such Contractor or Affiliate is solely for the benefitof Customer or such Affiliate. Customer shall be responsible for each User’s compliance with this Agreement.To the extent use of a Service requires Customer to install Client Software, Cloudwick grants to Customera limited, non-transferable, non-sublicensable, non-exclusive license during the Subscription Term to use theobject code form of the Client Software internally in connection with Customer’s and its Affiliates use of theService, subject to the terms and conditions of this Agreement and the Documentation.

3. Implementation. Upon payment of any applicable fees set forth in each Order Form, Cloudwick agrees to usereasonable commercial efforts to provide standard implementation assistance for the Service only if and to the

57


extent such assistance is set forth on such Order Form (“Implementation Assistance”). If Cloudwick providesImplementation Assistance in excess of any agreed-upon hours estimate, or if Cloudwick otherwise providesadditional services beyond those agreed in an Order Form, Customer will pay Cloudwick at its then-currenthourly rates for consultation.

4. Customer Obligations. User ID and Password Protection. Customer will require that all permitted Users keepuser ID and password information strictly confidential and not share such information with any unauthorizedperson. Amorphic will have no liability for actions taken using Customer’s user IDs and passwords, includingany unauthorized use or access caused by misuse or misappropriation of such user IDs and passwords. Customerwill be responsible for restricting access by any User who is no longer authorized to access the Service.

5. Support; Service Levels. Subject to Customer’s payment of all applicable fees, Cloudwick will provide sup-port, maintenance service, and uptime for each Service in accordance with (i) the support package selectedby Customer on the applicable Order Form (if any) and (ii) Cloudwick’s then-current standard Support andAvailability Policy (the current version of which is set forth at www.amorphicdata.com/legal).

6. Service Updates. From time to time, Cloudwick may provide upgrades, patches, enhancements, or fixes forthe Services to its customers generally without additional charge (“Updates”), and such Updates will becomepart of the Services and subject to this Agreement; provided that Cloudwick shall have no obligation underthis Agreement or otherwise to provide any such Updates. Customer understands that Cloudwick may ceasesupporting old versions or releases of the Services at any time in its sole discretion; provided that Cloudwickshall use commercially reasonable efforts to give Customer reasonable prior notice of any major changes.

7. Ownership; Feedback. As between the parties, Cloudwick retains all right, title, and interest in and to theServices, and all software, products, works, and other intellectual property and moral rights related thereto orcreated, used, or provided by Cloudwick for the purposes of this Agreement, including any copies and deriva-tive works of the foregoing. Any software which is distributed or otherwise provided to Customer hereunder(including without limitation any software identified on an Order Form) shall be deemed a part of the “Services”and subject to all of the terms and conditions of this Agreement. No rights or licenses are granted except asexpressly and unambiguously set forth in this Agreement. Customer may (but is not obligated to) provide sug-gestions, comments or other feedback to Cloudwick with respect to the Service (“Feedback”). Feedback, evenif designated as confidential by Customer, shall not create any confidentiality obligation for Cloudwick notwith-standing anything else. Cloudwick acknowledges and agrees that all Feedback is provided “AS IS” and withoutwarranty of any kind. Customer shall, and hereby does, grant to Cloudwick a nonexclusive, worldwide, perpet-ual, irrevocable, transferable, sublicensable, royalty-free, fully paid up license to use and exploit the Feedbackfor any purpose. Nothing in this Agreement will impair Cloudwick’s right to develop, acquire, license, market,promote or distribute products, software or technologies that perform the same or similar functions as, or oth-erwise compete with any products, software or technologies that Customer may develop, produce, market, ordistribute.

8. Fees; Payment. Customer will be billed by Amazon Marketplace for all Platform base services and any ap-plicable tiered monthly consumption overage fees as set forth in each Order Form (“Fees”). Amazon Inc.Marketplace terms and conditions apply for all Amorphic Data Platform subscriptions purchased by Customerusing the Amazon Marketplace. For Cloudwick Professional Services purchased by Customer all invoices willbe issued by Cloudwick under this Agreement and are payable in U.S. dollars within (30) days from date ofinvoice. Past due Cloudwick invoices are subject to interest on any outstanding balance of the lesser of 1.5%per month or the maximum amount permitted by law. Customer shall be responsible for all taxes associatedwith Service (excluding taxes based on Cloudwick’s net income). All Fees paid are non-refundable and are notsubject to set-off. If Customer exceeds any user or usage limitations set forth on an Order Form, then (i) AmazonMarketplace shall invoice Customer for such additional users or usage at the overage rates set forth on the OrderForm (or if no overage rates are set forth on the Order Form, at Cloudwick’s then-current standard overage ratesfor such usage), in each case on a pro-rata basis from the first date of such excess usage through the end of theOrder Form Initial Term or then-current Order Form Renewal Term (as applicable), and (ii) if such Order FormTerm renews (in accordance with the section entitled “Term; Termination”, below, such renewal shall includethe additional fees for such excess users and usage.

9. Restrictions. Except as expressly set forth in this Agreement, Customer shall not (and shall not permit any third

58 Chapter 4. Amorphic Data Contracts


party to), directly or indirectly: (i) reverse engineer, decompile, disassemble, or otherwise attempt to discoverthe source code, object code, or underlying structure, ideas, or algorithms of the Service (except to the extentapplicable laws specifically prohibit such restriction); (ii) modify, translate, or create derivative works basedon the Service; (iii) copy, rent, lease, distribute, pledge, assign, or otherwise transfer or encumber rights tothe Service; (iv) use the Service for the benefit of a third party; (v) remove or otherwise alter any proprietarynotices or labels from the Service or any portion thereof; (vi) use the Service to build an application or productthat is competitive with any Cloudwick product or service; (vii) interfere or attempt to interfere with the properworking of the Service or any activities conducted on the Service; or (viii) bypass any measures Cloudwick mayuse to prevent or restrict access to the Service (or other accounts, computer systems or networks connected to theService). Customer is responsible for all of Customer’s activity in connection with the Service, including but notlimited to uploading Customer Data (as defined below) onto the Service. Customer (a) shall use the Service incompliance with all applicable local, state, national and foreign laws, treaties and regulations in connection withCustomer’s use of the Service (including those related to data privacy, international communications, exportlaws and the transmission of technical or personal data laws), and (b) shall not use the Service in a manner thatviolates any third party intellectual property, contractual or other proprietary rights.

10. Customer Data. For purposes of this Agreement, “Customer Data” shall mean any data, information or othermaterial provided, uploaded, or submitted by Customer to the Service in the course of using the Service. Cus-tomer shall retain all right, title and interest in and to the Customer Data, including all intellectual property rightstherein. Customer, not Cloudwick, shall have sole responsibility for the accuracy, quality, integrity, legality, re-liability, appropriateness, and intellectual property ownership or right to use of all Customer Data. Cloudwickshall use commercially reasonable efforts to maintain the security and integrity of the Service and the CustomerData. Cloudwick is not responsible to Customer for unauthorized access to Customer Data or the unauthorizeduse of the Service unless such access is due to Cloudwick’s gross negligence or willful misconduct. Customeris responsible for the use of the Service by any person to whom Customer has given access to the Service,even if Customer did not authorize such use. Customer agrees and acknowledges that Customer Data may beirretrievably deleted if Customer’s account is ninety (90) days or more delinquent. Notwithstanding anythingto the contrary, Customer acknowledges and agrees that Cloudwick may (i) internally use and modify (but notdisclose) Customer Data for the purposes of (A) providing the Service to Customer and (B) generating Aggre-gated Anonymous Data (as defined below), and (ii) freely use and make available Aggregated Anonymous Datafor Cloudwick’s business purposes (including without limitation, for purposes of improving, testing, operating,promoting and marketing Cloudwick’s products and services). “Aggregated Anonymous Data” means data sub-mitted to, collected by, or generated by Cloudwick in connection with Customer’s use of the Service, but onlyin aggregate, anonymized form which can in no way be linked specifically to Customer.

11. Third Party Services. Customer acknowledges and agrees that the Service may operate on, with or usingapplication programming interfaces (APIs) and/or other services operated or provided by third parties (“ThirdParty Services”), including without limitation through integrations or connectors to such Third Party Servicesthat are provided by Cloudwick. Cloudwick is not responsible for the operation of any Third Party Servicesnor the availability or operation of the Service to the extent such availability and operation is dependent uponThird Party Services. Customer is solely responsible for procuring any and all rights necessary for it to accessThird Party Services (including any Customer Data or other information relating thereto) and for complyingwith any applicable terms or conditions thereof. Cloudwick does not make any representations or warrantieswith respect to Third Party Services or any third party providers. Any exchange of data or other interactionbetween Customer and a third party provider is solely between Customer and such third party provider and isgoverned by such third party’s terms and conditions.

12. Term; Termination. This Agreement shall commence upon the date of the first Order Form, and, unless earlierterminated in accordance herewith, shall last until the expiration of all Order Form Terms. For each Order Form,unless otherwise specified therein, the “Order Form Term” shall begin as of the effective date set forth on suchOrder Form, and unless earlier terminated as set forth herein, (x) shall continue for the initial term specified onsuch Order Form (the “Order Form Initial Term”), and (y) following the Order Form Initial Term, [shall automat-ically renew for additional successive periods of equal duration to the Order Form Initial Term (each, a “OrderForm Renewal Term”) unless either party notifies the other party of such party’s intention not to renew no laterthan thirty (30) days prior to the expiration of the Order Form Initial Term or then-current Order Form Renewal

4.1. Amorphic Data Enterprise Terms of Service 59


Term, as applicable]. In the event of a material breach of this Agreement by either party, the non-breaching partymay terminate this Agreement by providing written notice to the breaching party, provided that the breachingparty does not materially cure such breach within thirty (30) days of receipt of such notice. Without limiting theforegoing, Cloudwick may suspend or limit Customer’s access to or use of the Service if (i) Customer’s accountis more than sixty (60) days past due, or (ii) Customer’s use of the Service results in (or is reasonably likely toresult in) damage to or material degradation of the Service which interferes with Cloudwick’s ability to provideaccess to the Service to other customers; provided that in the case of subsection (ii): (a) Cloudwick shall usereasonable good faith efforts to work with Customer to resolve or mitigate the damage or degradation in orderto resolve the issue without resorting to suspension or limitation; (b) prior to any such suspension or limitation,Cloudwick shall use commercially reasonable efforts to provide notice to Customer describing the nature ofthe damage or degradation; and (c) Cloudwick shall reinstate Customer’s use of or access to the Service, asapplicable, if Customer remediates the issue within thirty (30) days of receipt of such notice. All provisions ofthis Agreement which by their nature should survive termination shall survive termination, including, withoutlimitation, accrued payment obligations, ownership provisions, warranty disclaimers, indemnity and limitationsof liability. In the case of expiration or termination of this Agreement, upon request by Customer made before,or within [thirty (30)] days after, the effective date of expiration or termination, Cloudwick shall make availableto Customer a complete transfer of all Customer Data to a Customer’s Amazon S3 bucket in a file or databaseformat in Cloudwick’s discretion. For clarity, any services provided by Cloudwick to Customer, including thedata export set out above, and any assistance in exporting the Customer Data, shall be billable at $250 per hour.

13. **Indemnification. ** Each party (“Indemnitor”) shall defend, indemnify, and hold harmless the other party,its affiliates and each of its and its affiliates’ employees, contractors, directors, suppliers and representatives(collectively, the “Indemnitee”) from all liabilities, claims, and expenses paid or payable to an unaffiliatedthird party (including reasonable attorneys’ fees) (“Losses”), that arise from or relate to any claim that (i) theCustomer Data or Customer’s use of the Service (in the case of Customer as Indemnitor), or (ii) the Service(in the case of Cloudwick as Indemnitor), infringes, violates, or misappropriates any third party intellectualproperty or proprietary right. Each Indemnitor’s indemnification obligations hereunder shall be conditionedupon the Indemnitee providing the Indemnitor with: (i) prompt written notice of any claim (provided that afailure to provide such notice shall only relieve the Indemnitor of its indemnity obligations if the Indemnitor ismaterially prejudiced by such failure); (ii) the option to assume sole control over the defense and settlement ofany claim (provided that the Indemnitee may participate in such defense and settlement at its own expense); and(iii) reasonable information and assistance in connection with such defense and settlement (at the Indemnitor’sexpense). The foregoing obligations of Cloudwick do not apply with respect to the Service or any information,technology, materials or data (or any portions or components of the foregoing) to the extent (i) not createdor provided by Cloudwick (including without limitation any Customer Data), (ii) made in whole or in part inaccordance to Customer specifications, (iii) modified after delivery by Cloudwick, (iv) combined with otherproducts, processes or materials not provided by Cloudwick (where the alleged Losses arise from or relate tosuch combination), (v) where Customer continues allegedly infringing activity after being notified thereof orafter being informed of modifications that would have avoided the alleged infringement, or (vi) Customer’s useof the Service is not strictly in accordance herewith.

14. **Disclaimer. ** EXCEPT AS EXPRESSLY SET FORTH HEREIN, THE SERVICE IS PROVIDED “AS IS”AND “AS AVAILABLE” AND ARE WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, IN-CLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF TITLE, NON-INFRINGEMENT,MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, AND ANY WARRANTIES IM-PLIED BY ANY COURSE OF PERFORMANCE, USAGE OF TRADE, OR COURSE OF DEALING, ALLOF WHICH ARE EXPRESSLY DISCLAIMED.

15. Limitation of Liability. EXCEPT FOR THE PARTIES’ INDEMNIFICATION OBLIGATIONS AND FORCUSTOMER’S BREACH OF SECTION 7, IN NO EVENT SHALL EITHER PARTY, NOR ITS DIREC-TORS, EMPLOYEES, AGENTS, PARTNERS, SUPPLIERS OR CONTENT PROVIDERS, BE LIABLE UN-DER CONTRACT, TORT, STRICT LIABILITY, NEGLIGENCE OR ANY OTHER LEGAL OR EQUITABLETHEORY WITH RESPECT TO THE SUBJECT MATTER OF THIS AGREEMENT (I) FOR ANY LOSTPROFITS, DATA LOSS, COST OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, OR SPE-CIAL, INDIRECT, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES OF ANY KIND WHAT-



SOEVER, SUBSTITUTE GOODS OR SERVICES (HOWEVER ARISING), (II) FOR ANY BUGS, VIRUSES,TROJAN HORSES, OR THE LIKE (REGARDLESS OF THE SOURCE OF ORIGINATION), OR (III) FORANY DIRECT DAMAGES IN EXCESS OF (IN THE AGGREGATE) THE FEES PAID (OR PAYABLE) BYCUSTOMER TO Cloudwick HEREUNDER IN THE TWELVE (12) MONTHS PRIOR TO THE EVENT GIV-ING RISE TO A CLAIM HEREUNDER.

16. Miscellaneous. This Agreement represents the entire agreement between Customer and Cloudwick with re-spect to the subject matter hereof, and supersedes all prior or contemporaneous communications and proposals(whether oral, written or electronic) between Customer and Cloudwick with respect thereto. The Agreementshall be governed by and construed in accordance with the laws of the State of California, excluding its conflictsof law rules, and the parties consent to exclusive jurisdiction and venue in the state and federal courts located inSan Francisco, California. All notices under this Agreement shall be in writing and shall be deemed to have beenduly given when received, if personally delivered or sent by certified or registered mail, return receipt requested;when receipt is electronically confirmed, if transmitted by facsimile or e-mail; or the day after it is sent, if sentfor next day delivery by recognized overnight delivery service. Notices must be sent to the contacts for eachparty set forth on the Order Form. Either party may update its address set forth above by giving notice in accor-dance with this section. Except as otherwise provided herein, no modification or amendment of any provisionof this Agreement shall be effective unless agreed by both parties in writing, and no waiver of any provisionof this Agreement shall be effective unless in writing and signed by the waiving party. Except for paymentobligations, neither party shall be liable for any failure to perform its obligations hereunder where such failureresults from any cause beyond such party’s reasonable control, including, without limitation, the elements; fire;flood; severe weather; earthquake; vandalism; accidents; sabotage; power failure; denial of service attacks orsimilar attacks; Internet failure; acts of God and the public enemy; acts of war; acts of terrorism; riots; civil orpublic disturbances; strikes lock-outs or labor disruptions; any laws, orders, rules, regulations, acts or restraintsof any government or governmental body or authority, civil or military, including the orders and judgments ofcourts. Neither party may assign any of its rights or obligations hereunder without the other party’s consent;provided that (i) either party may assign all of its rights and obligations hereunder without such consent to asuccessor-in-interest in connection with a sale of substantially all of such party’s business relating to this Agree-ment, and (ii) Cloudwick may utilize subcontractors in the performance of its obligations hereunder. No agency,partnership, joint venture, or employment relationship is created as a result of this Agreement and neither partyhas any authority of any kind to bind the other in any respect. In any action or proceeding to enforce rights underthis Agreement, the prevailing party shall be entitled to recover costs and attorneys’ fees. If any provision ofthis Agreement is held to be unenforceable for any reason, such provision shall be reformed only to the extentnecessary to make it enforceable. The failure of either party to act with respect to a breach of this Agreementby the other party shall not constitute a waiver and shall not limit such party’s rights with respect to such breachor any subsequent breaches.

17. Supporting Agreements

a. “Acceptable Use Policy”, attached or made available at https://www.amorphicdata.com/support-services-and-legal/

b. “Security Policy”, attached or made available at https://www.amorphicdata.com/support-services-and-legal/

c. “DPA or Data Processing Addendum” attached or made available at https://www.amorphicdata.com/support-services-and-legal/ on the Effective Date of this Agreement.

d. “Support Policy” attached or made available at https://www.amorphicdata.com/support-services-and-legal/

4.1. Amorphic Data Enterprise Terms of Service 61

https://www.amorphicdata.com/support-services-and-legal/






CHAPTER 5

Amorphic Data RACI matrix

A Responsible, Accountable, Consulted, and Informed (RACI) diagram or RACI matrix is used to describe the rolesand responsibilities of cloudwick and customer when customer choose either SaaS deployment or Managed Service.The RACI matrix for AMorphic Service Level Management is shown in the table below.

• RACI represents: R - Responsibility, A - Accountable, C - Consulted, and I - Informed

Responsibility = person or role responsible for ensuring that the item is completed

Accountable = person or role responsible for actually doing or completing the item

Consulted = person or role whose subject matter expertise is required in order to complete the item

Informed = person or role that needs to be kept informed of the status of item completion

63


5.1 RACI Matrix

Process ID Activity Cloudwick CustomerAD 1 Amorphic Data patch

managementR/A I

AD 2 Amorphic Data upgradeand release

R/A I

AD 3 Amorphic Data Supportticket creation

I R

AD 4 Amorphic Data Supportticket intake & assignment

R/A I

AD 5 Amorphic Data Supportticket status update

R/A I

AD 6 Amorphic Data Supportticket resolution

R/A I

AD 7 Review/Revise servicelevel definitions

R I

AD 8 Review/Revise Amorphicdata terms of service

R/A I

AD 9 Amorphic Data supportedAWS health issues

R/A I

AD 10 Amorphic Data servicestatus update

R/A I

AD 11 Amorphic Data new fea-ture request & develop-ment

R/A I/C

AD 12 Amorphic Data data setentitlement reports

• R

AD 13 Review/Revise AmorphicData’s security policy

R/A I

AD 14 Amorphic Data AWS In-fra management & moni-toring

R/A I

AD 15 Amorhic Data AWS sup-port ticket management

R/A I

AD 16 Amorphic Data platformperformance tuning

R/A I

AD 17 Amorphic Data user &data governance

• R

AD 18 Customer compliance forAmorphic Data Platform

I R

AD 19 3rd party tool support • R

AD 20 AWS Infrastructure costand payment to AWS

I R

AD 21 Amorphic Data cost andpayment to AWS

I R

AD 22 Amorphic Data beta fea-ture/service eligibility

R R

AD 23 AWS beta feature/serviceeligibility

R R

64 Chapter 5. Amorphic Data RACI matrix

CHAPTER 6

Incident Management Process

This section describes how Amorphic Data Incident Management implements the best practice guidelines for theIncident Management processes.

Amorphic Data uses ‘Zendesk’ as the sustomer service software and support ticketing system

Topics in this section include:

6.1 Amorphic Data Incident Management Process

With AMorphic Data Incident Management, the appropriate people can escalate and reassign incidents. IncidentManagement can also escalate an incident to properly meet the agreed-upon terms of the service contract. For example,if a priority 1 incident occures, an agent or manager can escalate the incident to a higher priority to ensure that theincident is fixed quickly.

Incident Management restores normal service operation as quickly as possible and minimizes the adverse im-pact on business operations, thus ensuring that the best possible levels of service quality and availabilityare maintained. It includes events that are communicated directly by Users, through amorphic support page(www.amorphicdata.com/support), email, chat or phone.

Incident Management defines normal service operation as service performance to meet Service Level Agreement(SLA).

Incidents can be reported and logged by support staff, who may notify the Service Desk if they notice an issue. Not allevents are logged as incidents. Many classes of events are not related to disruptions at all, but are indicators of normaloperation or are simply informational

6.1.1 Incident Management Workflow

The process includes all necessary steps to log and resolve an incident, including any necessary escalations or reas-signments.

The Incident Management process consists of the following processes, which are included in this section:

65


Incident Logging and Categorization

Incidents are initiated and logged as described in the ‘incident management’ section.

• If the incident is logged by a Amorphic Data Agent, most incident details are already provided by the interactionrecord. The Service Desk Agent verifies the Assignment Group to make sure the selected group is the mostsuitable group to solve the incident.

• If an incident is logged by a customer/user, usually by support website,email, or by chat, the incident must bebased on the applicable incident model.

Customer/user and Service Desk Agents can perform the following Incident Logging tasks:

• Create new incident from Amorphic Data support website (www.amorphicdata.com/support), email ([email protected])(user)

• Create new incident from user interaction (Service Desk Agent)

• Review and update incident information (Service Desk Agent)

You can see the details of this process in the following figure and table.

The Incident Logging and Categorization workflow is illustrated in the following figure:

Incident Logging and Categorization process

66 Chapter 6. Incident Management Process




Incident Assignment

Internal to Amorphic Data support team

Incidents are logged from an interaction by a Service Desk Agent or from an a user/customer. The Incident Coordinatormonitors the incident queue and reviews open status incidents. The Incident Coordinator verifies whether an incidentis a major incident using predefined criteria. If it is, the Incident Manager is informed about the incident arrival;otherwise, it is assigned to an Incident Analyst for further investigation and diagnosis.

The Incident Analyst receives an assigned incident and determines whether the incident can be resolved with thetools and knowledge available. If the incident cannot be resolved, the Incident Analyst reassigns it to the IncidentCoordinator.

You can see the details of this process in the following figure and table.

The Incident Assignment workflow is illustrated in the following figure:

Incident Assignment process

Incident Investigation and Diagnosis

Incident Resolution and Recovery

Incident Review and Closure

Incident Escalation

SLA Monitoring

6.2 Amorphic Data Enterprise Support Tickets

Users of Amorphic Data can create a new ticket by going to Amorphic support website (https://www.amorphicdata.com/support)

6.2. Amorphic Data Enterprise Support Tickets 67

https://www.amorphicdata.com/support



6.3 Amorphic Data Enterprise Chat Support Tickets

Please navigate to https://www.amorphicdata.com/support and use the available chat window to create support ticketsautomatically.

6.4 Amorphic Data Enterprise Email Tickets

Emails sent to AmorphicData support address: [email protected] automatically become tickets.

6.5 Amorphic Data Enterprise Phone Support

Coming Soon.

68 Chapter 6. Incident Management Process


CHAPTER 7

Amorphic Data Tableau Integration

The following section describes the frequently asked questions

7.1 Introduction

This document contains information related to connecting Amorphic Datasets to Tableau Dashboard and importingthe dashboard into the Amorphic Data platform for access and sharing related purposes.

7.2 Audience

This document is intended to provide Amorphic Data Platform’s Tableau connectivity userguide for the followingusers

Solution architecture team

Security team

DevOps team

Program/Technical management

This is a formal document subject to revision and change control. The list of authors is given in the section at thebeginning of the document.Following are the steps relating to visualization with Amorphic Data.

7.3 Tableau Connectivity

7.3.1 Data Warehouse Configuration

The user can search for the respective dataset in the dataset listing page. Once the user has finalized the dataset, theuser can go to the Data Warehouse Configuration Details section to connect to the dataset to Tableau for visualizationpurpose.

69


70 Chapter 7. Amorphic Data Tableau Integration

CHAPTER 8

Amorphic Data SaaS Billing

Please find the pricing information at https://www.amorphicdata.com/amorphic-pricing/

71

https://www.amorphicdata.com/amorphic-pricing/


72 Chapter 8. Amorphic Data SaaS Billing

CHAPTER 9

Amorphic Data Platform – SOC Compliance

9.1 1. Abstract

Amorphic Data Platform (ADP) is a SaaS product offering from Cloudwick. Amorphic Data is the first self-serviceAWS analytics and machine learning orchestration platform that simplifies cloud analytics and machine learning forall business, data science and engineering users. Information security is a reason for concern for all organizations,including those that outsource key business operation to third-party vendors (e.g., SaaS, cloud-computing providers).Rightfully so, since mishandled data—especially by application and network security providers—can leave enter-prises vulnerable to attacks, such as data theft, extortion and malware installation. Amorphic Data Platform (ADP)is deployed on Amazon Web Services cloud (AWS) provider and is deployed for individual clients with their chosencustomizations. As Cloudwick provides production environments of ADP to various clients, there is a need to dis-play the security and compliance measures that are embedded in ADP to safeguard the client’s data. This documentexplores the current state of security and compliance in ADP from the viewpoint of SOC compliance requirements –specifically, SOC 1 and SOC 2.

9.2 2. SOC Compliance – Introduction & Background

SOC is a system of service organization controls. The SOC controls are a series of standards designed to help mea-sure how well a given service organization conducts and regulates its information. The purpose of SOC standards isto provide confidence and peace of mind for organizations when they engage third-party vendors. A SOC-certifiedorganization has been audited by an independent certified public accountant who determined the firm has the appro-priate SOC safeguards and procedures in place. The American Institute of Certified Public Accountants (AICPA)provides the guidance for SOC examinations. Within the SOC guides, the AICPA provides some guidance on whatmethods of testing are acceptable.

73


Statement on Standards for Attestation Engagements (SSAE) No. 18 (Clarification and Recodification) is the standardgoverning SOC engagements. SSAE 18, effectively establishes requirements for performing and reporting on theexamination, review, and agreed-upon procedures engagements that enable practitioners to report on subject matterother than historical financial statements. SSAE 18 was put forth to address concerns over the clarity, length, andcomplexity of various standards developed by the Auditing Standards Board (ASB) of the American Institute ofCertified Public Accountants (AICPA).

Furthermore, SSAE 18 is effective for practitioners’ reports dated on or after May 1, 2017.

9.3 3. SOC 1 vs. SOC 2 vs. SOC 3

SOC concerns the internal controls in place at the third-party service organization such as - Cloudwick. For a companyto receive SOC certification, it must have sufficient policies and strategies that satisfactorily protect clients’ data. SOC1, SOC 2, and SOC 3 certifications all require a service organization to display controls regulating their interactionwith clients and client data. Note that SOC levels indicate differences both in the purview of the certification and inthe intended audience for the reports.

SOC 1 reports on the service organization’s controls related to its clients’ financial reporting.

SOC 2 reports build on the financial reporting basis of SOC 1 and also require standard operating procedures fororganizational oversight, vendor management, risk management, and regulatory oversight. A SOC 2-certified ser-vice organization is appropriate for businesses whose regulators, auditors, compliance officers, business partners, andexecutives require documented standards. SOC 2 auditing procedure ensures that the service provider (Cloudwick)securely manages the client’s data in a way that protects the interests of the client’s organization and its privacy. Forsecurity-conscious businesses, SOC 2 compliance is a minimal requirement when considering a SaaS provider. WhereSOC 1 focuses on the internal controls over financial reporting, SOC 2 concentrates on the protection and privacy ofdata.

SOC 3 reports are a simplified version of SOC 2 reports, requiring less formalized documentation. SOC 3 reporting isappropriate for businesses with less regulatory oversight concerns.

9.4 4. Type 1 or Type 2 Reports

SOC 1 and SOC 2 reports can be of two types - Type 1 or Type 2 reports.

base/amorphic-soc-compliance/../../images/amorphic-soc-compliance/ADP2.jpg

74 Chapter 9. Amorphic Data Platform – SOC Compliance


Type 1 reports are as of a particular date (sometimes referred to as point-in-time reports) that include a description of aservice organization’s system as well as tests to help determine whether a service organization’s controls are designedappropriately. Type 1 reports test the design of a service organization’s controls, but not the operating effectiveness.

Type 2 reports cover a period of time (usually 12 months), include a description of the service organization’s system,and test the design and operating effectiveness of key internal controls over a period of time.

9.5 5. SOC Compliance - SOC 1 Reports (Restricted Use Reports)

9.5.1 5.1. Need for a SOC 1 Report

The American Institute of Certified Public Accountants (AICPA) clarifies that SOC 1 report is for service organizationsthat do directly impact or may impact their clients’ financial reporting and is relevant to the clients’ internal controlover financial reporting.

9.5.2 5.2. Practical questions to determine the need for a SOC 1 report


Caution: SOC 1 reports are considered as Restricted Use Reports

Because SOC 1 reports may contain sensitive information about service organizations (Cloudwick), they are con-sidered restricted use reports and should only be shared with management of the service organization (the companywho has the SOC 1 performed), user entities of the service organization (Cloudwick’s clients) and the user entities’financial auditors (Cloudwick client’s auditors). The report can assist the customers’ financial auditors with laws andregulations like the Sarbanes–Oxley Act.

9.5.3 5.3. Does Amorphic Data Platform (ADP) require SOC 1 compliance report?

Cloudwick’s Amorphic Data Platform (ADP) being a Software-as-a-Service (SaaS) product, requires SOC 1 com-pliance report as it may impact the financials of customers – especially those customers/stakeholders that store theirfinancial data using ADP.

SOC 1 – Type 1 compliance report will suffice for the Cloudwick Amorphic Data Platform.

9.5.4 5.4. SOC 1 Report Structure


9.5. 5. SOC Compliance - SOC 1 Reports (Restricted Use Reports) 75


9.6 6. SOC Compliance - SOC 2 Reports (Attestation Reports)

9.6.1 6.1. Need for a SOC 2 Report

SOC 2 is an auditing procedure that ensures your service providers securely manage your data to protect the interestsof your organization and the privacy of its clients. For security-conscious businesses, SOC 2 compliance is a minimalrequirement when considering a SaaS provider.

The SOC 2 compliance is designed for more advanced I.T. service providers. These can include Managed I.T. ServiceProviders (MSPs), cloud computing vendors, data centers, and SaaS (software-as-a-service) companies.

For SOC 2 compliance, the American Institute of Certified Public Accountants (AICPA) provides certain Trust ServiceCriteria (TSCs) that can be selected by a service organization (Cloudwick) to demonstrate they have controls in placeto mitigate risks to the service (ADP) they provide.

9.6.2 6.2. SOC 2 compliance – Trust Service Criteria (TSCs)

SOC 2 defines criteria for managing customer data based on five “Trust Service Criteria”:

1. Security

2. Availability

3. Processing Integrity

4. Confidentiality and

5. Privacy.

Trust Service Criteria are explained as follows:



1. Security:

The security principle refers to protection of system resources against unauthorized access. Access controls helpprevent potential system abuse, theft or unauthorized removal of data, misuse of software, and improper alterationor disclosure of information. IT security tools such as network and web application firewalls (WAFs), two factorauthentication and intrusion detection are useful in preventing security breaches that can lead to unauthorized accessof systems and data.

2. Availability:

The availability principle refers to the accessibility of the system, products or services as stipulated by a contractor service level agreement (SLA). As such, the minimum acceptable performance level for system availability is setby both parties. This principle does not address system functionality and usability, but does involve security-relatedcriteria that may affect availability. Monitoring network performance and availability, site failover and security incidenthandling are critical in this context.

3. Processing Integrity:

The processing integrity principle addresses whether or not a system achieves its purpose (i.e., delivers the right data atthe right price at the right time). Accordingly, data processing must be complete, valid, accurate, timely and authorized.However, processing integrity does not necessarily imply data integrity. If data contains errors prior to being input intothe system, detecting them is not usually the responsibility of the processing entity. Monitoring of data processing,coupled with quality assurance procedures, can help ensure processing integrity.

4. Confidentiality:

Data is considered confidential if its access and disclosure is restricted to a specified set of persons or organizations.Examples may include data intended only for company personnel, as well as business plans, intellectual property,internal price lists and other types of sensitive information. Encryption is an important control for protecting confi-dentiality during transmission. Network and application firewalls, together with rigorous access controls, can be usedto safeguard information being processed or stored on computer systems.

5. Privacy

The privacy principle addresses the system’s collection, use, retention, disclosure and disposal of personal informationin conformity with an organization’s privacy notice, as well as with criteria set forth in the AICPA’s Generally AcceptedPrivacy Principles (GAPP). Personal identifiable information (PII) refers to details that can distinguish an individual(e.g., name, address, Social Security number). Some personal data related to health, race, sexuality and religion is alsoconsidered sensitive and generally requires an extra level of protection. Controls must be put in place to protect all PIIfrom unauthorized access.

9.6.3 6.3. SOC 2 compliance report types

SOC 2 provides two options for auditing service organizations, which are Type 1 and Type 2.

With a Type 1 audit, the auditor reviews and reports on the service organization’s system and the design of its controls,relating to one or all of the five Trust Services Criteria (TSC) at a specific moment in time.

A Type 2 audit includes all the same information as Type 1, but it also features the auditor’s assessment that a serviceorganization’s controls have been tested for operational effectiveness over a period of time (minimum of six months)

9.6. 6. SOC Compliance - SOC 2 Reports (Attestation Reports) 77


9.6.4 6.4. What type of SOC 2 compliance report does Amorphic Data Platform(ADP) require?

SOC 2 - Type 2 reports are the most comprehensive reports within the Systems and Organization Controls (SOC). Ingeneral, businesses/customers seeking a vendor such as an I.T. services provider will find SOC 2 - Type 2 is the mostuseful certification/report when considering a possible service provider’s credentials.

A company that has achieved SOC 2 - Type 2 certification has proven that its system is designed to keep its clients’sensitive data secure. When it comes to working with the cloud and related I.T. services, such performance andreliability is absolutely essential and increasingly required by regulators, examiners, and auditors.

Cloudwick’s Amorphic Data Platform (ADP) being a Software-as-a-Service (SaaS) product that is deployed on theAmazon Web Services (AWS) public cloud, requires SOC 2 - Type 2 compliance report.

9.6.5 6.5. SOC 2 compliance procedure

Note: SOC 2 reports are considered attestation reports

For a SOC 2 attestation, management of a service organization (Cloudwick) asserts that certain controls are in placeto meet some or all of the AICPA’s SOC 2 Trust Services Criteria (TSC).

Management (Cloudwick Management) also selects which of the five TSCs best address the risk of the services (ADP)provided by the service organization. When a service organization (Cloudwick) completes a SOC 2 report, the reportcontains an opinion from a CPA firm that states whether the CPA firm agrees with management’s assertion. Theopinion states that the appropriate controls are in place to address the selected TSCs and the controls are designed(Type 1 report) or designed and operating effectively (Type 2 report).

The SOC 2 compliance report structure is same as the SOC 1 compliance report structure and consists of the followingsections:

1. The Opinion Letter (SOC 1 Qualified Opinion vs. Unqualified)

2. Management’s Assertion

3. Description of the System

4. Description of Tests of Controls and Results of Testing

5. Other Information

Caution:

A SOC 2 report includes a detailed description of the service auditor’s tests of controls and results. The use of thisreport is generally restricted.


CHAPTER 10

Amorphic Data Platform – Security Policy

This Amorphic Data Platform (ADP) Security Policy outlines the technical and procedural measures that AmorphicData Platform and thereby, Cloudwick - undertakes to protect Customer Data from unauthorized access or disclosure.Cloudwick maintains these security measures in a manner consistent with SOC 2 (Type II) and addresses the mitigationstrategies for the OWASP top 10 vulnerabilities.

Amorphic Data Platform (ADP) has a written information security plan to implement the terms of this Security Policythat is reviewed and approved annually by its senior management team. As used in this Security Policy: “CloudProvider” means the third-party cloud provider, specifically - Amazon Web Services, Inc. (“AWS”) and “Cloud PrivateNetwork” means the VPC from which the Service is provided and “Cloudwick Personnel” or “Amorphic ProductTeam” means Cloudwick employees and individual subcontractors. This Security Policy is referenced in and made apart of your customer agreement with Cloudwick (the “Agreement”) and any capitalized terms used but not definedherein shall have the meaning set forth in the Agreement or Documentation, as applicable. In the event of any conflictbetween the terms of the Agreement and this Security Policy, this Security Policy shall govern. This Security Policymay be updated from time to time upon reasonable notice to Customer (which may be provided through the Serviceor the AWS Marketplace) to reflect process improvements or changing practices, but any such modifications will notmaterially diminish either party’s obligations as compared to those reflected below.

10.1 1. Customer Data Access and Management

1.1. Customer controls access to its Account(s) in the Amorphic Data Platform (ADP) via User IDs and passwords.

1.2. Cloudwick Personnel do not have access to unencrypted Customer Data unless Customer provides access to itsAmorphic account to such Cloudwick Personnel. If such access is granted, Cloudwick Personnel are prohibited fromstoring Customer Data on local desktops, laptops, mobile devices, shared drives, removable media such as USB drives,or on public facing systems that do not fall under the administrative control or compliance monitoring processes ofAmorphic Data Platform (ADP) and thereby – the Customer Amorphic Data Platform associated AWS account that ismanaged by Cloudwick Personnel.

1.3. Cloudwick uses Customer Data only as necessary to provide services to Customer, as provided in the Agreement.

1.4. Customer Data is stored only in the Customer’s Amorphic Data Platform production environment in the CloudPrivate Network.

79


1.5. Customer Data is stored in the available ADP Region for the account requested by Customer.

1.6. Cloudwick shall create and maintain flow diagram(s) indicating how Customer Data flows through the Service(“Flow Diagrams”) and shall provide Flow Diagrams upon Customer’s reasonable request. Flow Diagrams are Cloud-wick Confidential Information.

10.2 2. Encryption and Logical Separation of Customer Data

2.1. The Amorphic Data Platform service in the production storage environment always encrypts Customer Datawhile at rest using the AWS Key Management Service (KMS) symmetric key encryption – managed keys which useAES-256-bit encryption. These AWS KMS managed encryption keys are automatically rotated by AWS.

2.2. The Amorphic Data Platform service encrypts traffic with Transport Layer Security (“TLS”) 1.2 when communi-cating across untrusted networks such as the public internet.

2.3. The encryption keys are logically separated from Customer Data. Specifically, Amorphic Data Platform employsAWS KMS managed keys to safeguard the encryption keys.

10.3 3. Service Infrastructure Access Management

3.1. Access to the systems and infrastructure that support the Amorphic Data Platform service is restricted to Cloud-wick Personnel who require such access as part of their job responsibilities.

3.2. Unique User IDs are assigned to Cloudwick Personnel requiring access to the Amorphic Data Platform in orderto support the Service.

3.3. Amorphic Data Platform service password policy in the production environment adheres to the PCI-DSS passwordrequirements and is defined by Cloudwick in the Customer’s Amorphic Data Platform corresponding AWS account’s- IAM service.

3.4. Access privileges of separated Cloudwick Personnel are disabled promptly. Access privileges of persons transfer-ring to jobs requiring reduced privileges are adjusted accordingly.

3.5. User access to the systems and infrastructure that support the Service is reviewed quarterly.

3.6. Access attempts to the systems and infrastructure that support the Service are logged, monitored, and alerted forsuspicious activities.

3.7. Cloud Provider network security groups have deny-all default policies and only enable business required networkprotocols for egress and ingress network traffic. The Service only allows TLS 1.2 protocol from the public internet.

10.4 4. Risk Management

4.1. Cloudwick’s Amorphic Data Platform - Risk Management process is modeled on SOC 2 Type II.

4.2. Cloudwick conducts risk assessments of various kinds throughout the year, including self- and third-party assess-ments, audits, tests, automated scans, and manual reviews.

4.3. Results of assessments, including formal reports as relevant, are reported to the VP of Security. A Security Com-mittee meets weekly to review reports, identify control deficiencies and material changes in the threat environment,and make recommendations for new or improved controls and threat mitigation strategies to senior management.

4.4. Changes to controls and threat mitigation strategies are evaluated and prioritized for implementation on a riskadjusted basis.

80 Chapter 10. Amorphic Data Platform – Security Policy


4.5. Threats are monitored through various means, including threat intelligence services, vendor notifications, andtrusted public sources.

10.5 5. Vulnerability Scanning and Penetration Testing

5.1. Vulnerability scans are automatically performed weekly on systems required to operate and manage the AmorphicData Platform. The vulnerability database is updated regularly.

5.2. Scans that detect vulnerabilities meeting Cloudwick-defined risk criteria automatically trigger notifications tosecurity personnel.

5.3. Potential impact of vulnerabilities that trigger alerts are evaluated by staff.

5.4. Vulnerabilities that trigger alerts and have published exploits are reported to the Security Committee, whichdetermines and supervises appropriate remediation action.

5.5. Vulnerabilities are prioritized based on potential impact to the Service, with “critical” and “high” vulnerabilitiestypically being addressed within 30 days of discovery and “medium” vulnerabilities being addressed within 90 daysof discovery.

5.6. Security management monitors or subscribes to trusted sources of vulnerability reports and threat intelligence.

5.7. Penetration tests by an independent third-party expert are conducted at least annually.

5.8. Penetration tests performed by Cloudwick Security are performed regularly throughout the year.

10.6 6. Remote Access & Wireless Network

6.1. All access by Cloudwick Personnel to the Cloud Private Network requires successful authentication through asecure connection via approved methods such as VPNs and enforced with mutual certificate authentication and multi-factor authentication (“MFA”).

6.2. VPN access is further enforced by mutual TLS authentication.

6.3. Cloudwick corporate offices, including LAN and Wi-Fi networks in those offices, are considered to be untrustednetworks.

10.7 7. System Event Logging, Monitoring & Alerting

7.1. Monitoring tools and services are used to monitor systems including network, service events, and Cloud ProviderAPI security events, availability events, and resource utilization.

7.2. Amorphic Data Platform’s infrastructure Security event Logs are collected in a central system and protected fromtampering. Logs are stored for a minimum of 12 months.

7.3. All Amorphic Data Platform provided user endpoints have Endpoint Detection & Response (“EDR”) tools tomonitor and alert for suspicious activities and potential malware.

7.4. All Cloud Private Networks leverage advanced threat detection tools to monitor and alert for suspicious activitiesand potential malware.

10.5. 5. Vulnerability Scanning and Penetration Testing 81


10.8 8. System Administration and Patch Management

8.1. Cloudwick shall create, implement and maintain system administration procedures for systems that access Cus-tomer Data that meet or exceed industry standards, including without limitation, system hardening, system and devicepatching (operating system and applications) and proper installation of threat detection software as well as daily sig-nature updates of same.

8.2. Cloudwick Security reviews US-Cert new vulnerabilities announcements weekly and assess their impact to Amor-phic Data Platform based on Cloudwick-defined risk criteria, including applicability and severity.

8.3. Applicable US-Cert security updates rated as “high” or “critical” are addressed within 30 days of the patch releaseand those rated as “medium” are addressed within 90 days of the patch release.

10.9 9. Amorphic Security Training and Cloudwick Personnel

9.1. Cloudwick maintains a security awareness program for Cloudwick Personnel, which provides initial education,ongoing awareness and individual Cloudwick Personnel acknowledgment of intent to comply with Amorphic DataPlatform’s corporate security policies. New hires complete initial training on security, SOC, HIPAA, OWASP andPCI, sign a proprietary information agreement, and digitally sign the information security policy that covers keyaspects of the Cloudwick’s Amorphic Data Platform Information Security Policy.

9.2. All Cloudwick Personnel acknowledge they are responsible for reporting actual or suspected security incidents orconcerns, thefts, breaches, losses, and unauthorized disclosures of or access to Customer Data.

9.3. All Cloudwick Personnel are required to satisfactorily complete quarterly security training.

9.4. Cloudwick performs criminal background screening as part of the Cloudwick hiring process, to the extent legallypermissible.

9.5. Cloudwick will ensure that its subcontractors, vendors, and other third parties (if any) that have direct access tothe Customer Data in connection with the services adhere to data security standards consistent with the compliancestandards.

10.10 10. Physical Security

10.1. The Amorphic Data Platform service is hosted with Amazon Web Services (AWS) Cloud Provider and all phys-ical security controls are managed by AWS. Cloudwick reviews the Cloud Provider’s SOC 2 Type 2 report annually toensure appropriate physical security controls, including:

10.1.1. Visitor management including tracking and monitoring physical access.

10.1.2. Physical access point to server locations are managed by electronic access control devices.

10.1.3. Monitor and alarm response procedures.

10.1.4. Use of CCTV cameras at facilities.

10.1.5. Video capturing devices in data centers with 90 days of image retention.

10.11 11. Notification of Security Breach

11.1. A “Security Breach” is (a) the unauthorized access to or disclosure of Customer Data, or (b) the unauthorizedaccess to the systems/services within the Amorphic Data Platform that transmit or analyze Customer Data.



11.2. Cloudwick will notify Customer in writing within seventy-two (72) hours of a confirmed Security Breach.

11.3. Such notification will describe the Security Breach and the status of Cloudwick’s investigation.

11.4. Cloudwick will take appropriate actions to contain, investigate, and mitigate the Security Breach.

10.12 12. Disaster Recovery & Business Continuity

12.1. Cloudwick maintains a Disaster Recovery Plan (“DRP”) for the Amorphic Data Platform. The DRP is testedannually. However, changes to the DRP based on the Customer’s required Recovery Time Objective (RTO) andRecovery Point Objective (RPO) will be made by Cloudwick upon mutual agreement.

12.2. For the AWS Cloud Provider, the Amorphic Data Platform is managed in different AWS Regions as standalonedeployments, which can be employed as part of Customer’s DRP strategy. To effectively use the AWS cross-regionalavailability of the Service for disaster recovery purposes, Cloudwick is responsible for the following:

12.2.1. Requesting the Customer information pertaining to the deployment of additional Amorphic DataPlatform accounts as needed in different AWS regions to support the Cloudwick-Customer defined DRPprogram. Cloudwick will work with the Customer to ensure the DRP includes detailed Service LevelAgreements for Disaster Recovery and defines the Recovery Point Objective (RPO) and Recovery TimeObjective (RTO) for the respective Amorphic components.

12.2.2. Managing the Customer’s Amorphic Data Platform data replication across applicable regions.

12.2.3. Configuring and managing the Customer’s Amorphic Data Platform accounts in the identifiedAWS regions.

12.2.4. Managing the Customer’s Amorphic Data Platform backup and restoration strategies.

12.3. Cloudwick will define a Business Continuity Plan (“BCP”) with the Customer and maintain the defined BusinessContinuity Plan (“BCP”) for the Customer’s Amorphic Data Platform. The BCP is assessed annually.

10.13 13. Amorphic Data Platform Security Compliance, Certifica-tions, and Third-party Attestations

Cloudwick hires accredited third parties to perform SOC audit and to attest to compliance and certify annually toproduce a SOC 2 Type 2 Attestation Report.

10.14 14. Customer Responsibilities

14.1. Customer acknowledges that Cloudwick does not assess the contents of Customer Data and that Cloudwick isresponsible for enabling appropriate security controls in the Customer’s deployment of Amorphic Data Platform toensure a level of security appropriate to the particular nature of Customer Data, managing and protecting its accounts,roles and credentials. The Customer agrees to take appropriate steps to pseudonymize Customer Data where appropri-ate, and to update their Client Software (if any) as required whenever Cloudwick announces an update to AmorphicData Platform.

14.2. Customer will promptly notify Cloudwick if a user credential has been compromised or if Customer suspectspossible suspicious activities that could negatively impact security of the Amorphic Data Platform or Customer’saccount.

14.3. Customer may not perform any security penetration tests or security assessment activities without the expressadvance written consent of Cloudwick.

10.12. 12. Disaster Recovery & Business Continuity 83


14.4. Customers whose Customer Data includes PCI, PHI, PII or other sensitive data must agree to the implementationof Cloudwick provided IP whitelisting and MFA in the Amorphic Data Platform and, to the extent Customer Data issubject to PCI-DSS, HIPAA, or FEDRAMP, Customer may only upload such data to the Amorphic Data Platform.


CHAPTER 11

Amorphic Data Platform – OKTA Integration

11.1 1. Introduction

This document provides detailed steps on Amorphic Data (Amorphic) Platform - OKTA integration. Amorphic sim-plifies analytics for all users and teams by orchestrating and automating analytic pipelines & workflows across AWSservices, infrastructure and analytic tools and platforms.

Okta is an enterprise grade identity management service. With Okta IT can manage access across any application,person or device. Whether the people are employees, partners or customers or the applications are in the cloud, onpremises or on a mobile device. Okta “integrates” applications into its service for us, and we simply deploy thesepre-integrated applications to our users as necessary. We can authenticate these users against our own user store(e.g. AWS Cognito User Pools, Active Directory or LDAP) or we can use Okta as the user store. In Amorphic DataPlatform, Users/API calls get authenticated using Cognito. Amazon Cognito User Pools allow sign-in through a thirdparty (federation), including through a SAML IdP such as Okta. Here we use Okta as a Security Assertion MarkupLanguage 2.0 (SAML 2.0) identity provider (IdP) with an Amazon Cognito user pool.

11.2 2. Scope

This document covers the Amorphic Data Platform’s OKTA integration mechanism with AWS Cognito User Pools.

11.3 3. Audience

This document is intended to provide Amorphic Data Platform’s OKTA integration procedure and related answers forcustomer’s: Solution architecture team Security team DevOps team Program/Technical management

85


11.4 4. Amorphic - Okta Integration

11.4.1 4.1 Pre-requisites for Okta integration

Before we proceed with the process of Okta integration, we need the following pre-requisites:

1. Amazon AWS account.

2. OKTA developer account—https://developer.okta.com/signup/

11.4.2 4.2 Setup AWS Cognito User Pool

• Login to AWS management console and go to Cognito service dashboard.

• In the Cognito AWS dashboard, click on “Manage User Pools”

• Next, click on “Create a user pool”, which will bring us to the following page:

• In the above “Create a user pool” page, please enter pool name then click on “Step through settings” andsetup Attribute as the following:

86 Chapter 11. Amorphic Data Platform – OKTA Integration

https://developer.okta.com/signup/


Please note that Amorphic Data Platform requires two attributes: email and name. In the above snapshot of “At-tributes”, we define name attribute for the Cognito user pool. Upon saving the user pool settings, we will proceedto add a custom attribute as illustrated in the section: 4.5 - Adding a custom attribute to the Cognito user pool for“Email”

• Next, click on “Next Step” and setup “Policies” as following, you can customize based on your requirements:

• After setting up the policies, click “Next Step” and setup “MFA and verification” as following:

11.4. 4. Amorphic - Okta Integration 87


• After setting up MFA, click “Next Step” and setup “Message and customization” as following:



• After setting up the message customizations, click “Next Step” and it will ask us to setup “Tags” (optional).Save the changes and proceed to further steps.

• Now, it will ask you to setup “Devices” as following:

11.4. 4. Amorphic - Okta Integration 89


11.4.3 4.3 Adding an AWS Cognito App Client:

Click on “App Clients” in the Cognito dashboard as following and click on “Add an app client”:

• After populating the fields as shown above for the App Client, click on: “Create app client” to create the appclient in our Cognito User Pool.

• Leave the populated defaults for the “Triggers” section and proceed to Review the Cognito User Pool configu-ration.

• Once the user pool is created, you will see following details. Save the user pool id somewhere as we need thislater when we setup OKTA account.

11.4.4 4.4 Configure the Cognito User Pool Domain Name

After the Cognito User Pool is successfully created, click on “Domain name” under the section: “App Integra-tion”—add a domain name and click on “Save Changes” as shown below:



11.4.5 4.5 Adding a custom attribute to the Cognito user pool for “Email”:

• Proceed to the Cognito user pool’s “Attributes” section and click on “Add custom attributes” and create thefollowing custom attribute for Email as shown below:

• After creating the “Email” custom attribute and marking the attribute as “Mutable”, please proceed to the “AppClients” section to set the custom attribute’s read and write permissions as shown below:

11.5 5. Setting up Okta developer account

5.1. Login into a newly created OKTA developer account or an existing account.

5.2. After login, please click on “Admin”, will bring you to the console page:

11.5. 5. Setting up Okta developer account 91


5.3. From the console page, click on “Applications” and switch layout to “Classic UI”.

5.4. Click on “Add Applications” shortcut and it will bring you to following page:

5.5. Click on “Create New App” button and it will open a dialog as shown below:



5.6. Please enter the “App name” and click ‘Next’ as shown below:

5.7. Now, we proceed to configure the “SAML settings” of the OKTA application:

5.7.1. Enter the “Single Sign on URL”.

The single sign on URL consists of the: Cognito Domain Name + saml2/idpresponse

Example: https://your_domain_name.auth.us-east-1.amazoncognito.com/saml2/idresponse


https://your_domain_name.auth.us-east-1.amazoncognito.com/saml2/idresponse


5.7.2. Please add “Audience URI (SP Entity ID)” as:

“urn:amazon:cognito:sp:” + cognito user pool id (which we saved from the earlier steps)

Example: urn:amazon:cognito:sp:us-east-1_YQ3n16VOT

Please refer to the below screenshot for the SAML settings described in the above 5.7.1 and 5.7.2 points.

5.7.3. After filling in the SAML settings as detailed above, please click on “Preview the SAML Assertion” button andthis will open xml docs into separate tab, COPY that content as save as xml file. For example, save as: “metadata.xml”

5.7.4. Click ‘Next’ and configure ‘Feedback’. Click the “Finish” button and click on “Assignment” tab and assignto users (if you do not have user created, please create one) as shown below:


urn:amazon:cognito:sp:us-east-1_YQ3n16VOT


Next

We have successfully configured OKTA developer account.



11.6 6. Configuration of federated Identities in AWS Cognito usingthe Okta SAML assertion

6.1. Now that we have successfully configured OKTA, let us configure federated identities using the previouslydownloaded xml content/file (SAML assertion file). Please browse to the AWS Cognito setup page and click on“Identity Providers” under “Federation” and choose “SAML” as shown below:

6.2. In order to create a Cognito identity provider, we need the Identity Provider metadata from our application inOKTA. To acquire the Identity Provider metadata, please navigate to the OKTA admin dashboard and go to “Applica-tions” as shown below:

6.3. Click on the existing active application that we earlier created and go to the “Sign On” tab of the application andclick on the link that says: “Identity Provider metadata” as shown below:



6.4. This will open up the Identity Provider Metadata on a separate URL. Please copy the URL so that it can be used toadd a SAML 2.0 based Identity provider in Cognito dashboard. Saving the XML information in the Identity ProviderMetadata URL as a file (example: metadata.xml) also works.

6.5. Navigate to the Cognito dashboard and proceed to: “Federation” and “Identity Providers” and upload theSAML xml file (if saved as detailed in step 6.4) or enter the URL of the OKTA Identity Provider Metadata and clickon “Create Provider” as shown below:

11.6. 6. Configuration of federated Identities in AWS Cognito using the Okta SAML assertion 97


6.6. After successfully creation of the Identity provider in Cognito Federation, click on “App Client Settings” under“App Integration” and configure as shown below. Click “Save Changes” after the configuration of App clientsettings:

Please select the Cognito user pool’s OKTA identity provider that we earlier created (as highlighted in above snapshot).The callback URL(s) of the Cognito user pool point to “localhost” in the above screenshot as this is a demo of OKTAintegration. The callback URL(s) will be populated with the Client’s Amorphic URL in production.

11.7 7. Testing the OKTA integration with Cognito user pool

Note:

The following testing procedure is subject to change as the Amorphic web UI evolves enabling us a more UI friendlytesting mechanism.

11.7.1 7.1. Testing procedure flow

When we enable IDP based sign-in to Amorphic Data Platform and in this scenario, specifically using OKTA as theidentity provider, the basic test flow is as follows:

• Customer starts the OKTA based sign-in from the Amorphic login page.



• Amorphic login page contains the external link to the customer’s OKTA login.

• The end user clicks the IDP (OKTA) login link and authenticates with the customer’s OKTA.

• The end user’s OKTA login has Amorphic as an application (assuming the end user was already granted accessto Amorphic application as defined in the earlier sections)

• The end user clicks on the Amorphic Data Platform application available in the list of OKTA applications andgoes to the landing page of Amorphic Data Platform.

• Cognito - behind the hood authenticates with OKTA and assigns a JWT token to the user so that upon OKTAauthentication, upon clicking on the Amorphic app in the OKTA apps, the end user is directly taken to his/herAmorphic landing page)

11.7.2 7.2. Test of OKTA integration for demo purposes

Note:

This test is for demo purposes only and is bound to change as the Amorphic web UI evolves.

At the time of this demo, the Amorphic web UI is not yet changed to reflect the UI redirection from successful OKTAlogin to the Amorphic landing page. So, to demonstrate the test, we adopt a slightly different approach and theAmorphic web app is launched on the localhost (local test machine). The callback URL(s) of the Cognito user poolhas https://localhost:3000 as the callback URL.

7.2.1. Launch the Amorphic web app on the localhost using the following npm command. The appropriate Amorphiccode branch was cloned prior to the execution of the following npm command:

base/amorphic-okta-integration/../../images/amorphic-okta-integration/ADP27.jpg

7.2.2. Alternatively, if the npm pakcages are already built, we can issue the following command to start the Amorphicdemo web UI:

base/amorphic-okta-integration/../../images/amorphic-okta-integration/ADP28.jpg

7.2.3. Go to the URL: https://localhost:3000 on your local machine. The Amorphic web UI will come up as shownbelow: (click on “show advanced” in the browser window if it says – “Not Secure”)

11.7. 7. Testing the OKTA integration with Cognito user pool 99

https://localhost:3000

https://localhost:3000


7.2.4. Please click on “IDP Login” link in the login web page and this will take you to the OKTA login page – whereinyou can provide your OKTA credentials as shown below:

11.7. 7. Testing the OKTA integration with Cognito user pool 101


7.2.5. Upon entering the OKTA username and corresponding password, ideally, we will be redirected to the Amorphic landing page. At the time of documenting this demo, the redirection to landing page isn’t implemented. So, we take a slightly different approach to test the successful integration.Technically, upon successful authentication with the OKTA identity provider, Cognito will generate a JWTtoken and allocate the token to the end user. So, in this demo, after the end user successfully authenticates usinghis/her OKTA credentials, the JWT token/SAML assertion object will be indicated when we navigate to thebrowser’s => ”Inspect element” and go to => ”Console” as shown below:



11.8 8. Navigating to the Amorphic application in OKTA dashboard

Login to your OKTA account and browse to “My Applications” as shown below:

This will take us to the OKTA dashboard wherein our previously created Amorphic application is shown to us:

11.8. 8. Navigating to the Amorphic application in OKTA dashboard 103


For signing into the Amorphic Data Platform, click on the corresponding application and it will take you to theAmorphic Data Platform landing page.


amorphic documentation

Documents