manipulating data

MANIPULATING DATAhttp://www.flickr.com/photos/maggiew/6145245962/

To manipulate data, you need a reference to the data… Welcome to Object keys

• Full Key = app engine application ID + separator + entity kind + separator + entity group + separator + entity ID key– Actually, each part is hashed before concatenation

• Typical ways to specify the entity ID key– You set it yourself as a String (or Long)– You let GAE automatically assign a Long– You let GAE automatically set an encoded String– You set it yourself as a custom Key object

Setting your own String as a key

package edu.oregonstate.mobilecloud.lectures.clouddatastore;import javax.jdo.annotations.PersistenceCapable;import javax.jdo.annotations.Persistent;import javax.jdo.annotations.PrimaryKey;

@PersistenceCapablepublic class Test2Simple {

@PrimaryKey@Persistentprivate String title;

public String getTitle() {return title != null ? title : "";

}public void setTitle(String title) {

this.title = title;}

}

1. Declare @PrimaryKey2. Before trying to

makePersistent, be sure to set the value

3. Any existing entity of that type with that key is overwritten

Letting GAE automatically assign a Long key

package edu.oregonstate.mobilecloud.lectures.clouddatastore;

import javax.jdo.annotations.IdGeneratorStrategy;import javax.jdo.annotations.PersistenceCapable;import javax.jdo.annotations.Persistent;import javax.jdo.annotations.PrimaryKey;

@PersistenceCapablepublic class Test3Members {

@PrimaryKey@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)private Long id;

public Long getID() {return id;

}

1. Declare @PrimaryKey2. Specify IDENTITY

generator type3. Do not try to assign

the ID when instantiating object

Listing all entities of a certain kind

<%@ page import="javax.jdo.Query" %><%@ page import="javax.jdo.PersistenceManager" %><%@ page import="edu.oregonstate.mobilecloud.PMF" %><%@ page import="edu.oregonstate.mobilecloud.lectures.clouddatastore.*" %><%@ page import="java.util.*" %>

<%@ page import="edu.oregonstate.mobilecloud.PMF" %><% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class); List<Test3Members> allTest3 = (List<Test3Members>)query.execute(); for (Test3Members obj : allTest3) { out.write("<B>"+obj.getDepartment()+obj.getCoursenum() +": difficulty "+obj.getDifficulty()+"</B>"); out.write("<div style='height:100; “ +”overflow-y:auto'>"+obj.getDescription()+"</div>"); out.write("<BR><BR><BR>"); } } finally {pm.close();}%>

1. Create Query2. Execute3. Cast to List of objects

test5.jsp

Retrieving a subset of entities based on a value


<%@ page import="edu.oregonstate.mobilecloud.PMF" %><% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class, "department == :dd && coursenum >= :cn"); List<Test3Members> allTest3 = (List<Test3Members>)query.execute("CS", 400); for (Test3Members obj : allTest3) { out.write("<B>"+obj.getDepartment()+obj.getCoursenum()+": difficulty "+obj.getDifficulty()+"</B>"); out.write("<div style='height:100; overflow-y:auto'>"+obj.getDescription()+"</div>"); out.write("<BR><BR><BR>"); } query.closeAll(); } finally {pm.close();}%>

1. Create Query with parameters

2. Bind parameter values on execute

3. Cast to List of objects4. Close query when done

test5a.jsp

Retrieving an object by key


<%@ page import="edu.oregonstate.mobilecloud.PMF" %><% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class); query.setOrdering("id desc"); List<Test3Members> allTest3 = (List<Test3Members>)query.execute(); long maxId = allTest3.get(0).getID(); Test3Members highest = pm.getObjectById(Test3Members.class, maxId); out.write(highest+"..."+highest.getID()); query.closeAll(); } finally {pm.close();}%>

1. Use getObjectById to retrieve2. If object might not exist, catch

JDOObjectNotFoundException or JDOException

test5b.jsp

Deleting entities

test5c.jsp


<%@ page import="edu.oregonstate.mobilecloud.PMF" %><% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class, "coursenum >= 600"); List<Test3Members> toDelete = (List<Test3Members>)(query.execute()); pm.deletePersistentAll(toDelete); query.closeAll();} finally {pm.close();}%>

1. Retrieve list of objects to delete (which seems inefficient?!?!)

2. Invoke pm.deletePersistentAll

By the way…

• Query has a closeAll() method. – close the query when you’re done with results

• Query has a deletePersistentAll() instance method. It is painfully slow in some SDKs.– If you use this method, be sure to assess how fast

it is on very large sets of objects before you rely on it.

Now what if two people want to manipulate an object at the same time?

…@PersistenceCapablepublic class Test1Object {

@PrimaryKey@Persistentprivate String title;

…@Persistentprivate TreeSet<Integer> categories =

new TreeSet<Integer>();

public void addCategory(Integer category) {

categories.add(category);}

}

<%String title = request.getParameter("title");if (title == null) out.write("provide a title"); else {

PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Test1Object obj = null; try { obj = pm.getObjectById(Test1Object.class, title); } catch (JDOObjectNotFoundException doesNotExist) { obj = new Test1Object(); obj.setTitle(title); }obj.addCategory(Math.round(Math.rand()*100)); pm.makePersistent(obj); } finally {pm.close(); }}%>

You need a transaction when your servlet needs to…

• Update an entity if it already exists, or create a new one otherwise

• Compute a new member variable value based on an existing value

• Modify multiple entities in a consistent way

• … Do anything where your servlet could potentially be confused if gets two simultaneous requests/hits from users

Setting up a simple transaction

Transaction trans = pm.currentTransaction(); trans.begin();Test1Object obj = null;try { obj = (Test1Object)pm.getObjectById(Test1Object.class, title);} catch (JDOObjectNotFoundException doesNotExist) { obj = new Test1Object(); obj.setTitle(title);}pm.makePersistent(obj);trans.commit();

1. Tell the current transaction to begin tracking operations

2. Do your operations3. Commit your changes4. (On uncaught exception,

changes are rolled back automatically)

test2.jsp

Transactions on GAE

• A transaction places an optimistic lock all of the entities in a certain group

• If any other transaction tries to modify entities in that group, then – The first transaction to commit is successful– The second (and later) transactions fail on commit

• In practice, contention can cause commits to fail… so just retry

• Very very similar to DBMS optimistic locking, which we all know and love, except for this concept of entity groups

Entity groups

• Groups partition the set of all entities in GAE• Every entity is in exactly one group• Every group has at least one entity• Every group is very similar to tree– It has a root (which is used to identify the group)– Every entity has a parent (which for roots is the

entity itself – this is where the tree analogy breaks)

• Overall, therefore, the datastore is a forest

In other words…

• A transaction can lock one, and only one, tree in the forest

Oregon State

CS ECE

CS361CS496

Univ Oregon

Info SysArt History

IS434IS210

Univ Calif Berkeley

CS ECE

CS364CS351 CS364

Entities in a group are stored in the same place(s) in distributed file system

• Remember: Your data values are copied (replicated) to many different servers.

• Storing entities that are locked together (a group) helps to improve performance

• Downside: an entity’s group cannot be changed after the entity is first saved

• Odd side-effect: entities are only affected by a transaction if they are retrieved by implicit specification of a file system location, e.g., retrieval by key or via a query that references an ancestor filter (e.g., parent key)

How to set up entity groups

• Option 1: Use JDO Relationships to specify that the child is dependent on the parent

• Option 2: Use custom JDO Key object that identifies the parent entity for a child entity

Option 1: Use JDO Relationships

• This is different than a compound object (an object that has multiple parts)

• It is when you have two entities, one of which is a child of another

• The parent “owns” the child in JDO-speak

Example code snippet

…@PersistenceCapablepublic class Test3JDOEmployer {

@PrimaryKey@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)@Extension(vendorName = "datanucleus", key = "gae.encoded-pk", value = "true")private String id;

@Element(dependent="true")private HashSet<Test3JDOEmployee> employees = new HashSet<Test3JDOEmployee>();

…

@PersistenceCapable(identityType = IdentityType.APPLICATION)public class Test3JDOEmployee {

@PrimaryKey@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)@Extension(vendorName = "datanucleus", key = "gae.encoded-pk", value = "true")private String id;

Example servlet code

Test3JDOEmployer boss = new Test3JDOEmployer();

boss.setName("Employer "+empname);for (int empid = 1; empid <= 3; empid++) {

Test3JDOEmployee worker = new Test3JDOEmployee();

worker.setName("Employee "+empname);boss.addEmployee(worker);

}pm.makePersistent(boss);

In the servlet…

• In the servlet– Just initialize and store the parent (no explicit child

save)– On load, just load the parent and access child via

the parent as needed• FYI: Child objects are not loaded until they are read or

written, so loading the parent can be faster.

Option 2: Use explicit JDO Keys

• Which I’m not going to show you in detail because it is much more hassle to set up, and it doesn’t really add any more benefits.

• But here’s how you can do it:– Only list the keys of the child in the parent– When instantiating the child, pass the parent’s key– In the child’s constructor, use KeyFactory to

construct a child key, based on the parent’s key– See GAE Book 324-325 for details (note typos)

Summary of important points

• Use a transaction if two simultaneous hits to the server could lead to inconsistency

• Each transaction can only operate on a single entity group– To put multiple entities in the same group, use a

JDO parent-child relationship• If you need a transaction and you need to

modify multiple entity groups, use cursors and transactional tasks

manipulating data

Documents