Connecting your .NET Applications to NoSQL Databases

- MongoDB and Cassandra

30th June 2016

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.2

Product Owner ODBC, JDBC & ADO.NET Drivers, Progress DataDirecthttps://www.progress.com/documentation/datadirect-connectors

Avadhoot.Kulkarni@progress.com

Avadhoot Kulkarni

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.3

Agenda What is NoSQL?

• Understanding Schema Design• Benefits• Implementations

Different ways of Connecting to NoSQL Data• Client APIs • REST/SOAP APIs • SQL Based Connectivity, Progress DataDirect Drivers

Understanding Schema Design

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.5

What is NoSQL?

Sample JSON Document (MongoDB):{ name: “sue”, age: 26, status: “A”, groups: [“news”, “sports”]}

Relational database design focuses on data storage

NoSQL database design focuses on data use

Key Value Store (Cassandra):

NoSQL RDBMS

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.6

Schema Design Comparison

Relational Design NoSQL Document Design{ user: {

first: “Brody,

last: “Messmer”, ...

}

purchases: [

{ symbol: “PRGS”, date: “2013-02-13”, price: 23.50, qty: 100, ...},

{ symbol: “PRGS”, date: “2012-06-12”, price: 20.57, qty: 100, ...},

...

]

}

...

user_id first last …

123456 Brody Messmer …

…

user_id symbol date price qty …

123456 PRGS 2013-02-13 23.50 100 …

123456 PRGS 2012-06-12 20.57 100 …

…

Table: users

Table: purchases

A little Humor…..

Benefits of NoSQL

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.9

Benefits of NoSQL

High Performance• Data can easily be partitioned across multiple nodes• Low default isolation levels for both read and write operations• Object models (Denormalized schema design) reduce need for expensive joins• Typical index support, even on fields within embedded documents and arrays

High Availability & Fault Tolerance• Replica sets / nodes provide automatic failover and data redundancy

Easily Scale Up or Scale Out• Capable of running on commodity hardware

Cost

NoSQL Implementations

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.11

NoSQL Implementations

MongoDB

Type: JSON Document Store

Query Language: API / PSQL

Typical Use Case:• Web Applications (especially when

built with JavaScript)

Additional Benefits:• Node.js / Web friendly -- JSON• Dynamic schema

Apache Cassandra

Type: Key Value Store

Query Language: CQL

Typical Use Case:• Real-time analytic workloads• Heavy Writes, with desire for

reporting

Additional Benefits:• Especially High Availability

with CAP focus on Availability and Partition Tolerance

MarkLogic

Type: Multi-Model

Query Language: API

Typical Use Case:• Search• Recommendation Engine

Additional Benefits:• Handles any types of data

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.12

Ways to Access NoSQL Data

Client SDK

REST/SOAP APIs

SQL Connectivity

Connectivity using Client API

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.14

Client API Based Connectivity

Almost every NoSQL database has development SDKs to allow applications to integrate with its data. Pros:

• Easiest and quickest way to develop application Cons:

• Functionality may be limited, • Creates dependency on vendor SDK• Learning the SDK is vendor specific learning

When to Use:• Time critical deliveries, • Vendor specific Integration is acceptable

Example: Client SDK Connectivity to MongoDB

Components Used for the Demo SampleMongoDB C# Driver

• Nuget Package Link https://www.nuget.org/packages/mongocsharpdriver/

• Getting Started https://docs.mongodb.com/getting-started/csharp/client/

Example: Client SDK Connectivity to MongoDB

What does the sample do?• Insert object of type MyClass – Square• Insert object of type MyClass – Circle• Insert object of type SubClass – SubClass• Count All Objects which Has Corners > 1• Count All Objects which Has Ratio > 1

Example: Client SDK Connectivity to MongoDB

Example: Client SDK Connectivity to Cassandra

Components Used for the Demo SampleCassandra C# Driver

Nuget Package Link https://www.nuget.org/packages/CassandraCSharpDriver/

Getting Started https://academy.datastax.com/resources/getting-started-apache-cassandra-and-c-net

Example: Client SDK Connectivity to Apache Cassandra

What does the sample do?• CREATE the USERS Table• Insert User – Salman• Insert Another User - Shahrukh• Select a User using Where Clause Filter Condition

Example: Client SDK Connectivity to Cassandra

Example: Client SDK Connectivity to Cassandra

REST/SOAP API Based Connectivity

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.24

REST/SOAP API Based Connectivity

Some NoSQL vendors expose their low level APIs over REST and/or SOAP. Few provide this connectivity out-of-the box i.e. CouchDB; While few need extra setup, e.g. Cassandra & MongoDB Pros:

• Usually more flexible and performant than SDK Access• Small vendor specific learning compared to Client SDK

Cons:• More complex to integrate compared to SDK

When to Use:• Performance is Key• Looking integration which is less vendor dependent

REST API Connectivity to MongoDBThere are Multiple REST Interfaces for MongoDB

• https://docs.mongodb.com/ecosystem/tools/http-interfaces/

Building First Mongo DB Application using REST • https://www.mongodb.com/blog/post/building-your-first-appl

ication-mongodb-creating-rest-api-using-mean-stack-part-1

REST APIs for MongoDB CRUD Operations• http://www.codeproject.com/Articles/1071823/RESTful-WEB-

API-for-CRUD-operations-in-MongoDB

REST API Connectivity to CassandraApache-Meos REST API Reference

• https://mesosphere.github.io/cassandra-mesos/docs/rest-api.html

Writing your first client• https://docs.datastax.com/en/developer/nodejs-driver/1.0/n

odejs-driver/quick_start/qsQuickstart.html

RESTful APIs using spring Boot and Cassandra• http://myjourneyonjava.blogspot.in/2015/08/restful-api-usin

g-spring-boot-and.html

SQL Based Connectivity

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.28

SQL Based Connectivity

There are few third party vendors like Progress DataDirect, who provides SQL standard based drivers for NoSQL Data sources. Pros:

• No new learning, we all know SQL and ODBC/JDBC/ADO.NET • Easy to integrate with BI/Analytics tool which expect xDBC/ADO.NET

Integration Cons:

• Not a natural fit for the Data Mapping.• Some operations could become very slow because of limitations of

underlying APIs/Protocols When to Use:

• Third party BI/Analytical tool integration• Designing Applications which are vendor agnostic• Data Sync/Integration applications with RDBMS DataSources

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.29

SQL Connectivity to NoSQL Databases Is Hard

Cassandra MongoDB Challenges

Non-Standard Query Language

Lack of Common RDBMS Functionality

• No Support for Joins

• Limited support for filters, aggregates, etc

• Sorting is not ANSI SQL Compliant

• No ACID Transactions

• Unique Authentication

Non-relational Schema

• Heavy use of complex types (denormalized data model)

• Self-describing schema – Can only discover columns by selecting data

• Primary / Foreign Keys maintained by apps, not the database

Frequent Release Cadence

Example: SQL Connectivity to MongoDB

Progress DataDirect ODBC Driver for MongoDB

Download Page : https://www.progress.com/connectors/mongodb

Documentation:https://documentation.progress.com/output/DataDirect/odbcmongohelp/index.html

Example: SQL Connectivity to MongoDB

What does the sample do?• Uses ADO.NET ODBC Bridge to Progress DataDirect

MongoDB ODBC Driver• Application clears all rows in the MyClass Collection

(Table).• Insert object of type MyClass – Square• Insert object of type MyClass – Circle• Insert object of type SubClass – SubClass• Count All Objects which Has Corners > 1• Count All Objects which Has Ratio > 1

Example: SQL Connectivity to MongoDB

Example: SQL Connectivity to MongoDB

Example: SQL Connectivity to Cassandra

Progress DataDirect ODBC Driver for Apache Cassandra <Preview>

Download Page : https://www.progress.com/connectors/apache-cassandra

Documentation:https://documentation.progress.com/output/DataDirect/jdbccassandrahelp/

Example: SQL Connectivity to Apache Cassandra

What does the sample do?• Uses .NET ODBC Bridge to Progress DataDirect Apache

Cassandra ODBC Driver• Delete all the rows from USERS Table• Insert User – Salman• Insert Another User - Shahrukh• Select a User using Where Clause Filter Condition

Example: SQL Connectivity to Cassandra

Example: SQL Connectivity to Cassandra

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.38

Collection Name: stock

{ symbol: “PRGS”,

purchases:[

{date: ISODate(“2013-02-13T16:58:36Z”), price: 23.50, qty: 100},

{date: ISODate(“2012-06-12T08:00:01Z”), price: 20.57, qty: 100,

sellDate: ISODate(“2013-08-16T12:34:58Z”)}, sellPrice: 24.60}

]

}

“Normalizing” the NoSQL Data Model

Table Name: stock

_id symbol

1 PRGS

stock_id Date Price qty sellDate sellPrice

1 2013-02-13 16:58:36

23.50 100 NULL NULL

1 2012-06-12 08:00:01

20.57 100 2013-08-16 12:34:58

24.60

Table Name: stock_purchases

© 2016 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.39

“Normalizing” the NoSQL Data Model

The Benefits: Re-use of existing skills (SQL, Joins, etc.)

• Exposing complex types using concepts familiar to those savvy with RDBMS

As arrays/lists/maps/sets grow, table definitions remain constant Simplified / Narrower table definitions Joins across parent/child tables result in a single query to the database.

In other words, there’s no performance penalty. Data in arrays can be sorted and/or aggregated via SQL

NoSQL is non-standard, non-relational way of storing information.Client SDK API based access is the easiest way to reach NoSQL data

from applications, though it might be limited, creates dependency on the SDK libraries and has lot of data source specific learning.

REST APIs based access is a bit more complex, but provides more flexibility in terms of its usage. The API knowledge is data-source specific; If written good code, should perform better than client SDK.

SQL based connectivity, no new data source specific learnings, easy integration with third party BI/Analytics tools which expects SQL connectivity. As it’s not the natural way of connectivity to NoSQL, Data Normalization is required. Some SQL connectivity solutions like Progress DataDirect takes care of this transparent to the application.

Normalisation techniques should be the key criteria for choosing the right SQL Connectivity solution for NoSQL DataSources.

Recap

Questions?