full stack development with java and nosql - meetupfiles.meetup.com/6295162/full stack development...
TRANSCRIPT
©2015 Couchbase Inc. 2
What is Couchbase?
Couchbase is a distributed operational database that enables you to develop with agility and operate at any scale.
Managed Cache Key-Value Store Document Database Embedded Database Sync Management
©2015 Couchbase Inc. 3
Where is NoSQL a Good Fit?
Low-latency Critical
High Throughput or Large Numbers of Users
Unknown Demand with Sudden growth
Predominantly Direct Document Access
Read / Mixed / Write-heavy Workloads
Traditional Business Applications
Transaction-heavy Apps
Legacy Hardware
Full ACID support
Web / Mobile / IoT Legacy Business Apps
©2015 Couchbase Inc. 4
Develop with Agility
Easier, Faster Development Flexible Data Modeling Powerful Querying
SQL Integration & Migration Big Data Integration Mobile / IoT
©2015 Couchbase Inc. 5
Operate at Any Scale
Elastic Scalability Consistent High Performance Always-on Availability
Multi-Data Center Deployment Simple, Powerful Administration Enterprise Grade Security
©2015 Couchbase Inc. 8
Couchbase Server – Single Node Architecture
Data Service – builds and maintains local view indexes
Indexing Engine – builds and maintains Global Secondary Indexes
Query Engine – plans, coordinates, and executes queries against either Global or Local view indexes
Cluster Manager – configuration, heartbeat, statistics, RESTful Management interface
©2015 Couchbase Inc. 9
Simplified Administration
• Online upgrades and operations
• Built-in enterprise class Admin Console
• RESTful APIs
©2015 Couchbase Inc. 10
The Power of the Flexible JSON Schema
• Ability to store data in multiple ways
• De-normalized single document, as opposed to normalizing data across multiple table
• Dynamic Schema to add new values when needed
©2015 Couchbase Inc. 11
Accessing Data From Couchbase
Key access using Document ID
• Operations are extremely fast
with consistent low latency
• Reads and writes are evenly
distributed across Data Service
nodes
• Data is cached in built-in
Managed Caching layer and
stored in persistent storage layer
Queries using N1QL
• SQL-like : SELECT * FROM
WHERE, LIKE, GROUP,
etc.,
• JOINs
• Powerful Extensions (nest,
unnest) for JSON to support
nested and hierarchical data
structures.
• Multiple access paths – Views
and global secondary indexes
• ODBC/JDBC drivers available
Views using static queries
• Pre-computed complex Map-
Reduce queries
• Incrementally updated to
power analytics, reporting
and dashboards
• Strong for complex custom
aggregations
©2015 Couchbase Inc. 12
The Couchbase Java SDK
Synchronous and asynchronous interfaces
Compatible with various Java frameworks
Minimal coding
– No database maintenance via code
– No parsing queries via application code
©2015 Couchbase Inc. 13
Building Applications with Java SDK – Synchronous API
//connecting to the cluster via known node(s)
Cluster cluster = CouchbaseCluster.create("192.168.1.101");
//opening a bucket, establishing resources
Bucket bucket = cluster.openBucket("customBucket", "password");
//creating JSON and a Document
JsonObject json = JsonObject.create().put("name", "John");
JsonDocument doc = JsonDocument.create("key1", json);
//storing the Document
Document inDb = bucket.insert(doc);
©2015 Couchbase Inc. 14
Building Applications with Java SDK
public List<Map<String, Object>> function(Bucket bucket) {
String query = "SELECT * FROM `" + bucket.name() + “`”;
N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
if (!result.finalSuccess()) {
throw new DataRetrievalFailureException("Query error: " + result.errors());
}
List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
for(N1qlQueryRow row : result) {
content.add(row.value().toMap());
}
return content;
}
©2015 Couchbase Inc. 15
Building Applications with Java SDK
public List<Map<String, Object>> function(Bucket bucket) {
String query = "SELECT * FROM `" + bucket.name() + “`”;
N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
if (!result.finalSuccess()) {
throw new DataRetrievalFailureException("Query error: " + result.errors());
}
List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
for(N1qlQueryRow row : result) {
content.add(row.value().toMap());
}
return content;
}
©2015 Couchbase Inc. 16
Building Applications with Java SDK
public List<Map<String, Object>> function(Bucket bucket) {
String query = "SELECT * FROM `" + bucket.name() + “`”;
N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
if (!result.finalSuccess()) {
throw new DataRetrievalFailureException("Query error: " + result.errors());
}
List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
for(N1qlQueryRow row : result) {
content.add(row.value().toMap());
}
return content;
}
©2015 Couchbase Inc. 17
Building Applications with Java SDK
public List<Map<String, Object>> function(Bucket bucket) {
String query = "SELECT * FROM `" + bucket.name() + “`”;
N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
if (!result.finalSuccess()) {
throw new DataRetrievalFailureException("Query error: " + result.errors());
}
List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
for(N1qlQueryRow row : result) {
content.add(row.value().toMap());
}
return content;
}
©2015 Couchbase Inc. 18
Building Applications with Java SDK
public List<Map<String, Object>> function(Bucket bucket) {
String query = "SELECT * FROM `" + bucket.name() + “`”;
N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
if (!result.finalSuccess()) {
throw new DataRetrievalFailureException("Query error: " + result.errors());
}
List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
for(N1qlQueryRow row : result) {
content.add(row.value().toMap());
}
return content;
}
©2015 Couchbase Inc. 19
Building Applications with Java SDK
public List<Map<String, Object>> function(Bucket bucket) {
String query = "SELECT * FROM `" + bucket.name() + “`”;
N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
if (!result.finalSuccess()) {
throw new DataRetrievalFailureException("Query error: " + result.errors());
}
List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
for(N1qlQueryRow row : result) {
content.add(row.value().toMap());
}
return content;
}
©2015 Couchbase Inc. 20
Building Applications with Java SDK
public List<Map<String, Object>> function(Bucket bucket) {
String query = "SELECT * FROM `" + bucket.name() + “`”;
N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
if (!result.finalSuccess()) {
throw new DataRetrievalFailureException("Query error: " + result.errors());
}
List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
for(N1qlQueryRow row : result) {
content.add(row.value().toMap());
}
return content;
}
©2015 Couchbase Inc. 21
Complex N1QL Query
public static List<Map<String, Object>> getAll(final Bucket bucket, String from, String to) {
String queryStr = "SELECT faa AS fromAirport, geo " +
"FROM `" + bucket.name() + "` r " +
"WHERE airportname = $1 " +
"UNION SELECT faa AS toAirport, geo " +
"FROM `" + bucket.name() + "` r " +
"WHERE airportname = $2";
ParameterizedN1qlQuery query = ParameterizedN1qlQuery.parameterized(queryStr,
JsonArray.create().add(from).add(to));
N1qlQueryResult queryResult = bucket.query(query);
return extractResultOrThrow(queryResult);
}
©2015 Couchbase Inc. 24
Java Separation of Frontend and Backend
API driven
Less client and server coupling
The backend can evolve without affecting the frontend
Frontend can be extended to web as well as mobile
©2015 Couchbase Inc. 27
Java Configuration
// application.properties
hostname=127.0.0.1
bucket=restful-sample
password=
©2015 Couchbase Inc. 28
Java Create or Update Endpoint
// Application.java
@RequestMapping(value="/save", method=RequestMethod.POST)
public Object save(@RequestBody String json) {
JsonObject jsonData = JsonObject.fromJson(json);
if(jsonData.getString("firstname") == null || jsonData.getString("firstname").equals("")) {
return new ResponseEntity<String>(JsonObject.create().put("message", "A firstname is
required").toString(), HttpStatus.BAD_REQUEST);
}
// Other parameter validation here…
return Database.save(bucket, jsonData);
}
©2015 Couchbase Inc. 29
Java Get Document Endpoint
// Application.java continued…
@RequestMapping(value="/get", method= RequestMethod.GET)
public Object getByDocumentId(@RequestParam String document_id) {
if(document_id.equals("")) {
return new ResponseEntity<String>(JsonObject.create().put("message", "A document id is
required").toString(), HttpStatus.BAD_REQUEST);
}
return Database.getByDocumentId(bucket, document_id);
}
©2015 Couchbase Inc. 30
Java Delete Endpoint
// Application.java continued…
@RequestMapping(value="/delete", method=RequestMethod.POST)
public Object delete(@RequestBody String json) {
JsonObject jsonData = JsonObject.fromJson(json);
if(jsonData.getString("document_id") == null || jsonData.getString("document_id").equals("")) {
return new ResponseEntity<String>(JsonObject.create().put("message", "A document id is
required").toString(), HttpStatus.BAD_REQUEST);
}
return Database.delete(bucket, jsonData.getString("document_id"));
}
©2015 Couchbase Inc. 31
Java Extract Result
// Database.java
private static List<Map<String, Object>> extractResultOrThrow(N1qlQueryResult result) {
if (!result.finalSuccess()) {
throw new DataRetrievalFailureException("Query error: " + result.errors());
}
List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
for (N1qlQueryRow row : result) {
content.add(row.value().toMap());
}
return content;
}
©2015 Couchbase Inc. 32
Java Upsert Document Function
// Database.java
public static List<Map<String, Object>> save(final Bucket bucket, JsonObject data) {
String documentId = !data.getString("document_id").equals("") ? data.getString("document_id") :
UUID.randomUUID().toString();
String queryStr = "UPSERT INTO `" + bucket.name() + "` (KEY, VALUE) VALUES " +
"($1, {'firstname': $2, 'lastname': $3, 'email': $4})";
JsonArray parameters = JsonArray.create()
.add(documentId)
.add(data.getString("firstname"))
.add(data.getString("lastname"))
.add(data.getString("email"));
ParameterizedN1qlQuery query = ParameterizedN1qlQuery.parameterized(queryStr, parameters);
N1qlQueryResult queryResult = bucket.query(query);
return extractResultOrThrow(queryResult);
}
©2015 Couchbase Inc. 34
Java Get Document with N1QL Function
// Database.java continued…
public static List<Map<String, Object>> getByDocumentId(final Bucket bucket, String documentId) {
String queryStr = "SELECT firstname, lastname, email " +
"FROM `" + bucket.name() + "` AS users " +
"WHERE META(users).id = $1";
ParameterizedN1qlQuery query = ParameterizedN1qlQuery.parameterized(queryStr,
JsonArray.create().add(documentId));
N1qlQueryResult queryResult = bucket.query(query);
return extractResultOrThrow(queryResult);
}
©2015 Couchbase Inc. 35
Java Delete Document Function
// Database.java continued…
public static List<Map<String, Object>> delete(final Bucket bucket, String documentId) {
String queryStr = "DELETE " +
"FROM `" + bucket.name() + "` AS users " +
"WHERE META(users).id = $1";
ParameterizedN1qlQuery query = ParameterizedN1qlQuery.parameterized(queryStr,
JsonArray.create().add(documentId));
N1qlQueryResult queryResult = bucket.query(query);
return extractResultOrThrow(queryResult);
}
©2015 Couchbase Inc. 37
Get all documents
// AngularJS app.js
$scope.fetchAll = function() {
$http(
{
method: "GET",
url: "/api/getAll"
}
)
.success(function(result) {
for(var i = 0; i < result.length; i++) {
$scope.items[result[i].id] = result[i];
}
});
}
©2015 Couchbase Inc. 38
Save a document
// AngularJS app.s
$scope.save = function(firstname, lastname, email) {
$http(
{
method: "POST",
url: "/api/create",
data: {
firstname: firstname,
lastname: lastname,
email: email,
document_id: $stateParams.documentId
}
}
)
}
©2015 Couchbase Inc. 39
Building Applications with Java SDK - Asynchronous API
• The Cluster and Bucket both have async versions, obtained by calling async() method.
• Asynchronous API exposes RxJava Observables.
• Very rich and expressive API in terms of combinations and transformations.
©2015 Couchbase Inc. 40
Building Applications with Java SDK - Asynchronous API
//retrieving a document and extracting data for output
bucket.async()
.get("key1")
.map(doc -> doc.content().getString("name"))
.subscribe(name -> System.out.println("Hello " + name))
©2015 Couchbase Inc. 41
Building Applications with Java SDK - Asynchronous API
Async API, exposing an Observable<JsonDocument>
Observable is a stream, can be connected to an Observer
//retrieving a document and extracting data for output
bucket.async()
.get("key1")
.map(doc -> doc.content().getString("name"))
.subscribe(name -> System.out.println("Hello " + name))
©2015 Couchbase Inc. 42
Building Applications with Java SDK - Asynchronous API
Simple transformation operator from RxJava, T->R
Gets a JsonDocument
Extract String name value
Gives Observable<String>
//retrieving a document and extracting data for output
bucket.async()
.get("key1")
.map(doc -> doc.content().getString("name"))
.subscribe(name -> System.out.println("Hello " + name))
©2015 Couchbase Inc. 44
Couchbase Lite Embedded NoSQL Database
Sync Gateway Secure Synchronization
Couchbase
Server Cloud NoSQL Database
Couchbase Mobile
©2015 Couchbase Inc. 46
Sync Gateway
Secure
Synchronization
Authentication
Data Read Access
Data Write Access
©2015 Couchbase Inc. 49
AndroidContext context = new AndroidContext(this); manager = new Manager(context, Manager.DEFAULT_OPTIONS); database = manager.getDatabase("kitchen-sync"); Map<String,Object> properties = new HashMap<>(); properties.put("foo", "bar"); Document document = database.createDocument(); document.putProperties(properties);
©2015 Couchbase Inc. 50
Map Reduce Indexes
• Building indexes in your native language
• Results are persisted for fast querying
• Just set breakpoints to debug
©2015 Couchbase Inc. 51
// Create an index view = database.getView("itemsByDate"); view.setMap(new Mapper() { @Override public void map(Map<String, Object> document, Emitter emitter) { Object createdAt = document.get("created_at"); if (createdAt != null) { emitter.emit(createdAt.toString(), null); } } }, "1.0"); // Now query it Query query = view.createQuery(); query.setDescending(true); query.run();
©2015 Couchbase Inc. 52
Change Notifications
Change notifications
• Listen for changes
• Cuts down on a ton of cruft code
• Data, queries, replications – even documents
©2015 Couchbase Inc. 53
doc.addChangeListener(new Document.ChangeListener() { @Override public void changed(Document.ChangeEvent event) { DocumentChange change = event.getChange(); if (change.isConflict()) { // use business logic to resolve } } });
©2015 Couchbase Inc. 54
Sync
Sync
• Full multi-master replication
• Ability to minimize battery drain
• Change notifications and conflict detection
©2015 Couchbase Inc. 55
Replication pullReplication = database.createPullReplication(syncUrl); Replication pushReplication = database.createPushReplication(syncUrl); pullReplication.setContinuous(true); pushReplication.setContinuous(true); Authenticator authenticator = AuthenticatorFactory.createBasicAuthenticator(username, password); pullReplication.setAuthenticator(authenticator); pushReplication.setAuthenticator(authenticator); pullReplication.addChangeListener(this); pushReplication.addChangeListener(this); pullReplication.start(); pushReplication.start();
©2015 Couchbase Inc. 59
How Databases (Including Relational) compare
Flexible:
Flexible Data Model (JSON) NO YES NO YES
“Queryable”:
Complete SQL Capability YES LIMITED LIMITED YES
Fast:
Sub-Millisecond Latency NO NO YES YES
Scalable:
Elastic Scaling in Clusters NO LIMITED YES YES
Available:
HA / DR LIMITED LIMITED YES YES
Multi-Purpose:
Cache / Key-Value / Query LIMITED LIMITED LIMITED YES
Data Locality:
Geo-Replication / XDCR LIMITED LIMITED LIMITED YES
Mobility:
Local / Offline / Sync NO NO NO YES
©2015 Couchbase Inc. 60
Java and Android Sample Applications
https://github.com/couchbaselabs/restful-angularjs-java
https://github.com/couchbaselabs/try-cb-java
https://github.com/couchbaselabs/mini-hacks