imdg sdk development guide

Distributed Cache Service

IMDG SDK Development Guide

Issue 02

Date 2018-10-18

Contents

1 Overview......................................................................................................................................... 11.1 What Can DCS IMDG SDK Do?................................................................................................................................... 11.2 Content Navigation.........................................................................................................................................................2

2 Related Resources..........................................................................................................................3

3 Getting Started with DCS IMDG SDK.....................................................................................43.1 Development Process..................................................................................................................................................... 43.2 Setting Up the DCS Service Environment..................................................................................................................... 53.3 Setting up the SDK Development Environment.............................................................................................................63.4 Configuring the Authentication Parameters................................................................................................................... 83.5 Code Implementation..................................................................................................................................................... 9

4 Development in Typical Scenarios.......................................................................................... 104.1 Reading and Writing Cached Data............................................................................................................................... 104.2 Using SQL to Operate a Table......................................................................................................................................12

5 FAQs...............................................................................................................................................155.1 How to Develop an Application?................................................................................................................................. 155.2 What Can I Do in Case of Connection Timeout?......................................................................................................... 16

A Change History........................................................................................................................... 17

Distributed Cache ServiceIMDG SDK Development Guide Contents

Issue 02 (2018-10-18) ii

1 Overview

1.1 What Can DCS IMDG SDK Do?

1.2 Content Navigation

1.1 What Can DCS IMDG SDK Do?NOTE

IMDG is supported only in some regions. The DCS console prevails. This document is applicable onlyto regions that support this feature.

DCS IMDG SDKDistributed Cache Service In-memory Data Grids (DCS IMDG) SDK provides GridGain-based interfaces for third-party applications to read, write, and delete cached data. For detailsabout the interfaces, see the DCS IMDG API Reference.

Key Featuresl Support for ACID

Data stored in IMDG instances supports atomicity, consistency, isolation, and durability(ACID), so that transactions can be maintained on multiple servers on the entire network.

l Support for complete SQL, DDL, and DMLIMDG supports complete SQL, DDL, and DML. SQL can be used instead of code tointeract with IMDG for creating tables and indexes, and inserting, updating, andquerying data. With complete SQL support, IMDG can be used as a distributed SQLdatabase.

l JCache (JSR107) basedJCache (JSR107)-based IMDG IgniteCache provides data loading, query, asynchronousmodels, and other functions in addition to the standard JCache functions.

Distributed Cache ServiceIMDG SDK Development Guide 1 Overview

Issue 02 (2018-10-18) 1

http://static.huaweicloud.com/upload/files/dcs/DCS-IMDG-API.zip

1.2 Content Navigation

Table 1-1 Content navigation

Chapter Content

Overview Briefly introduces the features and functions of the DCS IMDGSDK.

Related Resources Describes the resources involved in secondary development byusing the DCS IMDG SDK, including SDK download addressesand development tools.

Getting Started withDCS IMDG SDK

Describes the entire process of sample project development byusing the DCS IMDG SDK. To quickly learn how to use the DCSIMDG SDK for development, read this chapter first.

Development inTypical Scenarios

Describes the development in typical DCS function scenarios,including the sample code, code description, and precautions.

FAQs Provides answers to common problems that may occur during theuse of the DCS IMDG SDK.

Distributed Cache ServiceIMDG SDK Development Guide 1 Overview

Issue 02 (2018-10-18) 2

2 Related Resources

Helpful Linksl Download DCS IMDG SDKl Download DCS IMDG API Documentsl Download Sample DCS IMDG Project

The sample DCS IMDG project is developed based on the DCS IMDG SDK and containssample code for basic functions of the DCS IMDG SDK. The following chapters use thissample project to describe the process of development by using the DCS IMDG SDK anddevelopment in typical scenarios.

Development Toolsl Eclipse: Eclipse 3.6.0 or later. It is available at the Eclipse official website.l JDK: Java Development Kit 1.8.111 or later. It is available at the Oracle official

website.l Maven: Apache Maven 3.0.3 or later

Distributed Cache ServiceIMDG SDK Development Guide 2 Related Resources

Issue 02 (2018-10-18) 3

http://static.huaweicloud.com/upload/files/dcs/DCS-IMDG-SDK.zip

http://static.huaweicloud.com/upload/files/dcs/DCS-IMDG-API.zip

https://dcs-demo.obs.myhwclouds.com/dcs-imdg-demo.zip

http://www.eclipse.org/downloads/

http://www.oracle.com/technetwork/java/javase/downloads/index.html


3 Getting Started with DCS IMDG SDK

This chapter describes how to use the DCS IMDG sample project to quickly get started withthe DCS IMDG APIs.

3.1 Development Process

3.2 Setting Up the DCS Service Environment

3.3 Setting up the SDK Development Environment

3.4 Configuring the Authentication Parameters

3.5 Code Implementation

3.1 Development ProcessThe Java language is used as an example to describe how to perform secondary developmentby using the DCS IMDG SDK.

The development process is as follows:

Distributed Cache ServiceIMDG SDK Development Guide 3 Getting Started with DCS IMDG SDK

Issue 02 (2018-10-18) 4

3.2 Setting Up the DCS Service EnvironmentWhen setting up the DCS service environment, record the environment information listed inTable 3-1 for future use.

Table 3-1 Environment information

Type Item Example Value

ElasticCloudServer(ECS)(runningtheWindowsOS)

Elastic IP address (EIP) -

Private IP address 192.168.0.2

hostname Host name of the ECS

Username administrator

Password password

DCS Address for connectingto a DCS instance

Single-node: 192.168.0.11Cluster: 192.168.0.11, 192.168.0.12

Username username

Password password

Registering a Cloud Service Account

Skip this section if you have registered a HUAWEI CLOUD service account.

Step 1 Access https://www.huaweicloud.com/en-us/ and click Register on the upper right corner ofthe page to register an account.

Step 2 Fill in user information as instructed.

Step 3 Activate the account by email.

Step 4 Log in to the public cloud management console.

NOTE

To use paid services, click Fees in the upper right corner of the console and then click Top Up. After thetop-up is successful, you can access the paid services.

----End

Applying for an ECS

Skip this section if you already have an ECS.

Log in to HUAWEI CLOUD, buy a Windows ECS on the ECS console, and bind an EIP tothe ECS. Record the EIP, username, and password for subsequent access to the IMDG.


Issue 02 (2018-10-18) 5

https://www.huaweicloud.com/en-us/

https://console.huaweicloud.com/ecm/

NOTE

Ensure that the same Virtual Private Cloud (VPC) and security group are configured for the ECS youpurchase and DCS instance to be accessed, so that the ECS can communicate with the DCS instance.If the ECS and DCS instance are not in the same VPC, you can establish a VPC peering connection toachieve network connectivity between the ECS and DCS instance. For details, see Do DCS InstancesSupport Cross-VPC Access?If different security groups have been configured for the ECS and DCS instance, you can set securitygroup rules to achieve network connectivity between the ECS and DCS instance. For details, see How toSelect and Configure a Security Group?

Creating a DCS IMDG InstanceNOTE

Ensure that the DCS IMDG instance you create belongs to the same VPC and subnet as the ECS, so thatthe ECS can communicate with the DCS IMDG instance.

Log in to HUAWEI CLOUD, and create a DCS IMDG instance on the DCS console. Recordthe username and password that you entered when creating the DCS IMDG instance, and theconnection address displayed on the console after the instance is created.

3.3 Setting up the SDK Development Environment

All operations described in this section are performed on the Windows ECS purchased inApplying for an ECS to facilitate joint commissioning.

Configuring the Java EnvironmentSkip this section if you have already downloaded the JDK and configured the environment.

JDK 1.8.111 or later is required. The following uses JDK 8 (Windows 7 x64) as an example.

Step 1 Download the JDK at the Oracle official website.

Step 2 Install the JDK as prompted. For example, install the JDK to the C:\Program Files\Java\jdk1.8.0_172 directory on the local PC.

Step 3 Configure Java environment variables.

1. Choose Control Panel > System and Security > System > Advanced system settings.The System Properties page is displayed.

2. On the Advanced tab page, click Environment Variables. The Environment Variablespage is displayed.

3. Create system variable JAVA_HOME whose value is the JDK installation path.4. Add %JAVA_HOME%\bin;%JAVA_HOME%\jre\bin to Path. Separate multiple

values with semicolons (;).5. Create system variable CLASSPATH whose value is %JAVA_HOME%\lib\dt.jar;

%JAVA_HOME%\lib\tools.jar.

Step 4 Open the command line interface (CLI) and run java -version. If the information shown inthe following figure is displayed, the configuration is successful.


Issue 02 (2018-10-18) 6

https://support.huaweicloud.com/en-us/dcs_faq/en-us_topic_0100698850.html




https://console.huaweicloud.com/dcs/


----End

Downloading the Sample ProjectDownload and decompress the DCS IMDG sample project code package to a local PC.

For more examples, visit https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples.

Installing Eclipse and Importing the Sample ProjectEclipse 3.6.0 or later is required. The following uses Eclipse as an example to describe how toimport the sample project. Operations for other IDE tools are similar to those for Eclipse.

Step 1 Download Eclipse, for example, eclipse-jee-mars-R-win32-x86_64.zip, from the Eclipseofficial website.

Step 2 Extract and open Eclipse.

Step 3 Configure the correct JRE path in Windows > Preferences > Java > Installed JREs.

Step 4 Right-click the project on the left and choose Build path > Configure Build Path. On thepage that is displayed, choose Java Build Path in the left pane and click the Libraries tab.Then, click Add External JARs, and select the local path where the DCS IMDG sampleproject is located.

NOTE

To import the DCS IMDG SDK package, create a project first, and then import the package by followingthe procedure described in this section.


Issue 02 (2018-10-18) 7


https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples

https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples



Step 5 Click OK. Then open the sample project after it is imported.

----End

3.4 Configuring the Authentication ParametersThis section describes how to configure information such as the username, password, andaddress for connecting to the DCS instance in the sample project. Such informationmentioned has been recorded in Creating a DCS IMDG Instance.

Step 1 Open the example-cache.xml file in src/main/resources on the IDE, and set the username,password, and multicast group as follows:l Replace username_for_replace with the username of the instance.l Replace password_for_replace with the password of the instance.l Replace 192.168.0.11 and 192.168.0.12 with the connection addresses of the instance.

You can add multiple lines of <value>XXX</value> to add more IP addresses, whereXXX indicates the connection address of the instance.

<bean id="allowAll.cred" class="org.apache.ignite.plugin.security.SecurityCredentials"> <constructor-arg value="username_for_replace"/> <constructor-arg value="password_for_replace"/> </bean> ..... <property name="discoverySpi"> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> <property name="ipFinder"> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"> <property name="addresses"> <list>


Issue 02 (2018-10-18) 8

<value>192.168.0.11</value> <value>192.168.0.12</value> </list></property></bean></property></bean></property>

----End

3.5 Code ImplementationFor details about code implementation in typical scenarios, see Development in TypicalScenarios.


Issue 02 (2018-10-18) 9

4 Development in Typical Scenarios

4.1 Reading and Writing Cached Data

4.2 Using SQL to Operate a Table

4.1 Reading and Writing Cached DataWrite data to a DCS IMDG instance, and then read and delete the data.

Sample Codeimport org.apache.ignite.Ignite;import org.apache.ignite.IgniteCache;import org.apache.ignite.IgniteException;import org.apache.ignite.Ignition;import org.apache.ignite.configuration.CacheConfiguration; import java.util.HashMap;import java.util.Map;import java.util.UUID; /** * This example demonstrates very basic operations on cache, such as 'put' and 'get'. * <p> * Remote nodes should always be started */public class CachePutGetExample{ /** Cache name. */ private static final String CACHE_NAME = "CachePutGetExample"; /** * Executes example. * * @param args Command line arguments, none required. * @throws IgniteException If example execution failed. */ public static void main(String[] args) throws IgniteException { try (Ignite ignite = Ignition.start("config/example-cache.xml")) { CacheConfiguration<Integer, String> cacheCfg = new CacheConfiguration<>(CACHE_NAME); //if you want see the cache metric in ces, set the statisticsEnabled

Distributed Cache ServiceIMDG SDK Development Guide 4 Development in Typical Scenarios

Issue 02 (2018-10-18) 10

as true cacheCfg.setStatisticsEnabled(true); //if your imdg instance is cluster and want to backup your cache, you can set the backups param. default is 0 cacheCfg.setBackups(0); try (IgniteCache<Integer, String> cache = ignite.getOrCreateCache(cacheCfg)) { // Individual puts and gets. putGet(cache); // Bulk puts and gets. putAllGetAll(cache); } finally { //ignite.destroyCache(CACHE_NAME); } } } /** * Execute individual puts and gets. * * @throws IgniteException If failed. */ private static void putGet(IgniteCache<Integer, String> cache) throws IgniteException { System.out.println(">>> Cache put-get example started."); final int keyCnt = 5; // Store keys in cache. for (int i = 0; i < keyCnt; i++) { cache.put(i, UUID.randomUUID().toString()); } System.out.println(">>> Stored values in cache. key size is " + keyCnt); for (int i = 0; i < keyCnt; i++) { cache.get(i); cache.remove(i); System.out.println(">>> get and remove value in cache. key is " + i); } } /** * Execute bulk {@code putAll(...)} and {@code getAll(...)} operations. * * @throws IgniteException If failed. */ private static void putAllGetAll(IgniteCache<Integer, String> cache) throws IgniteException { System.out.println(">>> Starting putAll-getAll example."); final int keyCnt = 5; // Create batch. Map<Integer, String> batch = new HashMap<>(); for (int i = 0; i < keyCnt; i++) batch.put(i, "bulk-" + Integer.toString(i)); // Bulk-store entries in cache.


Issue 02 (2018-10-18) 11

cache.putAll(batch); System.out.println(">>> Bulk-stored values in cache."); // Bulk-get values from cache. Map<Integer, String> vals = cache.getAll(batch.keySet()); for (Map.Entry<Integer, String> e : vals.entrySet()) System.out.println("Got entry [key=" + e.getKey() + ", val=" + e.getValue() + ']'); }}

Code Descriptionl Read the configuration information of a DCS IMDG instance from the example-

cache.xml configuration file and connect to the instance.l Create a cache named CachePutGetExample and set the number of backups to 0.l Write five data records (keys: 0–4; values: five random values) to the cache, and then

read and delete the data records from the cache.l Write data to and read the data from the cache in batches.

4.2 Using SQL to Operate a TableUse SQL statements to create and delete table structures, and write data to and query datafrom created tables.

Sample Codeimport org.apache.ignite.Ignite;import org.apache.ignite.IgniteCache;import org.apache.ignite.Ignition;import org.apache.ignite.cache.query.SqlFieldsQuery;import org.apache.ignite.configuration.CacheConfiguration; import java.util.List; /** * Example to showcase DDL capabilities of Ignite's SQL engine. * <p> * Remote nodes could be started * <p> */public class CacheQueryDdlExample{ /** Dummy cache name. */ private static final String DUMMY_CACHE_NAME = "CacheQueryDdlExample"; /** * Executes example. * * @param args Command line arguments, none required. * @throws Exception If example execution failed. */ @SuppressWarnings({"unused", "ThrowFromFinallyBlock"}) public static void main(String[] args) throws Exception { try (Ignite ignite = Ignition.start("config/example-cache.xml")) { print("Cache query DDL example started."); // Create dummy cache to act as an entry point for SQL queries (new


Issue 02 (2018-10-18) 12

SQL API which do not require this // will appear in future versions, JDBC and ODBC drivers do not require it already). CacheConfiguration<?, ?> cacheCfg = new CacheConfiguration<>(DUMMY_CACHE_NAME).setSqlSchema("PUBLIC"); try (IgniteCache<?, ?> cache = ignite.getOrCreateCache(cacheCfg)) { // Create reference City table based on REPLICATED template. cache.query(new SqlFieldsQuery( "CREATE TABLE city (id LONG PRIMARY KEY, name VARCHAR) WITH \"template=replicated\"")).getAll(); // Create table based on PARTITIONED template with one backup. cache.query(new SqlFieldsQuery( "CREATE TABLE person (id LONG, name VARCHAR, city_id LONG, PRIMARY KEY (id, city_id)) " + "WITH \"backups=1, affinityKey=city_id\"")).getAll(); // Create an index. cache.query(new SqlFieldsQuery("CREATE INDEX on Person (city_id)")).getAll(); print("Created database objects."); SqlFieldsQuery qry = new SqlFieldsQuery("INSERT INTO city (id, name) VALUES (?, ?)"); cache.query(qry.setArgs(1L, "Forest Hill")).getAll(); cache.query(qry.setArgs(2L, "Denver")).getAll(); cache.query(qry.setArgs(3L, "St. Petersburg")).getAll(); qry = new SqlFieldsQuery("INSERT INTO person (id, name, city_id) values (?, ?, ?)"); cache.query(qry.setArgs(1L, "John Doe", 3L)).getAll(); cache.query(qry.setArgs(2L, "Jane Roe", 2L)).getAll(); cache.query(qry.setArgs(3L, "Mary Major", 1L)).getAll(); cache.query(qry.setArgs(4L, "Richard Miles", 2L)).getAll(); print("Populated data."); List<List<?>> res = cache.query(new SqlFieldsQuery( "SELECT p.name, c.name FROM Person p INNER JOIN City c on c.id = p.city_id")).getAll(); print("Query results:"); for (Object next : res) System.out.println(">>> " + next); cache.query(new SqlFieldsQuery("drop table Person")).getAll(); cache.query(new SqlFieldsQuery("drop table City")).getAll(); print("Dropped database objects."); } finally { // Distributed cache can be removed from cluster only by #destroyCache() call. ignite.destroyCache(DUMMY_CACHE_NAME); } print("Cache query DDL example finished."); } } /**


Issue 02 (2018-10-18) 13

* Prints message. * * @param msg Message to print before all objects are printed. */ private static void print(String msg) { System.out.println(); System.out.println(">>> " + msg); }}

Code Descriptionl Read the configuration information of a DCS IMDG instance from the example-

cache.xml configuration file and connect to the instance.l Use SQL statements to create two tables respectively named city and person.l Insert three data records to the city table and four data records to the person table.l Perform a union query on the city and person tables.l Delete the city and person tables.


Issue 02 (2018-10-18) 14

5 FAQs

5.1 How to Develop an Application?

5.2 What Can I Do in Case of Connection Timeout?

5.1 How to Develop an Application?The following describes how to use the DCS IMDG demo package to develop an application,including downloading the instance code, modifying the code for typical scenarios,configuring the authentication parameters, and running and compiling the sample project.

Procedure

Step 1 Download and decompress the DCS IMDG sample project code package to a local PC.

Step 2 Open Eclipse. Choose File > Import. On the displayed Import dialog box, choose Maven >Existing Maven Projects, click Next. Select the directory where the sample project code isdecompressed to import the sample project code.

Step 3 Write the sample code for typical scenarios into src/main/java/com/imdg/examples/datagrid/MyFirstExample by referring to Development in Typical Scenarios.

Step 4 Configure information such as the username, password, and address for connecting to a DCSinstance by referring to Configuring the Authentication Parameters.

Step 5 On Eclipse, right-click the dcs.imdg.example project, choose Run As > Maven build fromthe displayed shortcut menu, and specify clean package as the execution target to executecompilation and building.

NOTE

l The GridGain Enterprises library is used in compiling the Maven project. External Repository ofGridGain is added to the Maven configuration file in the instance code library. Ensure that thenetwork is reachable before the compilation.

l If the network is unreachable, download the IMDG SDK package and decompress it to the localMaven repository. If the local Maven repository directory is D:\Repositories\Maven\repo, copy thefolders in the IMDG SDK directory where the IMDG SDK package is decompressed to D:\Repositories\Maven\repo.

Step 6 Copy dcs.imdg.example-1.0-SNAPSHOT.zip to any directory on a Linux ECS.

Distributed Cache ServiceIMDG SDK Development Guide 5 FAQs

Issue 02 (2018-10-18) 15


http://static.huaweicloud.com/upload/files/dcs/DCS-IMDG-SDK.zip

NOTE

Ensure that the same VPC and security group are configured for the ECS and DCS instance to beaccessed, so that the ECS can communicate with the DCS instance.

Step 7 Decompress the dcs.imdg.example-1.0-SNAPSHOT.zip package to the demo directory.unzip dcs.imdg.example-1.0-SNAPSHOT.zip -d demo

Step 8 Run the example script.cd demo bash bin/start_my_first_example.sh

----End

5.2 What Can I Do in Case of Connection Timeout?The possible causes for connection timeout are as follows:

l The instance connection address is incorrect. Check the instance connection address.l The network connection is abnormal. Check the network conditions.l The ECS and DCS instance are not in the same VPC. Establish a VPC peering

connection to achieve network connectivity between the ECS and DCS instance. Fordetails, see Do DCS Instances Support Cross-VPC Access?

l The ECS and DCS instance are not in the same security group. Set security group rulesto achieve network connectivity between the ECS and DCS instance. For details, seeHow to Select and Configure a Security Group?

Distributed Cache ServiceIMDG SDK Development Guide 5 FAQs

Issue 02 (2018-10-18) 16



A Change History

Table A-1 Change history

Released On What's New

2018-10-18 This issue is the second official release and incorporates thefollowing change:Adjusted the document structure and optimized the content.

2018-02-02 This is the first official release.

Distributed Cache ServiceIMDG SDK Development Guide A Change History

Issue 02 (2018-10-18) 17