Apache Curator™

Past, Present, Future

Jordan Zimmermanjordan@jordanzimmerman.com

@randgalthttp://www.linkedin.com/in/jordanzimmerman

http://aisallc.com

Introduction

About Me• Netflix Platform Team April 2011 – July 2013

• Started the Netflix OSS initiative

• Created the Netflix OSS Web Portal

• Currently Independent Contractor

• Lives in Panamá City, Panamá (!)

• Current contract: Blue Jeans NetworkPanamá City!

Curator’s History• Initially developed by me at Netflix

• Open-sourced by Netflix on GitHub in July 2011

• Accepted into the Apache Incubator in March 2013

• Curator graduated to an Apache Top Level Project in September, 2013

State of the Project

Since Last Presentation

• Feb 2012?

• Apache

• Lots of bug fixes, new features, etc.

• Lots of adoption

• More than 10 releases

Move to Apache• Apache verb |əˈpaSH, äˈpäSH|

Native American word meaning “slow down”

Move to Apache• Apache verb |əˈpaSH, äˈpäSH|

Native American word meaning “slow down”

• Huge Success

• Adoption Skyrocketed

• Thank you to Patrick Hunt for championing

😄

Lots of Users

Lots of Posts

Curator Gets an Entire Chapter in the ZK Book!

Widely Adopted - Apache

• Apache Storm

• Apache Drill

• Apache Oozie

• Apache Spark

• Apache Flume

• Apache Hive

• ???

Widely Adopted - Others

• Netflix (of course)

• Maginatics (now EMC)

• Bazaarvoice

• eBay

• General Electric

• Wix

• Spring

• Nirmata

Widely Adopted - Others

• Netflix (of course)

• Maginatics (now EMC)

• Bazaarvoice

• eBay

• General Electric

• Wix

• Spring

• Nirmata

I KNOW THERE ARE LOTS MORE

LET US KNOW WHO YOU ARE!

What’s New

New Committers

• Scott Blum

• Cameron McKenzie

Cameron McKenzieI'm a software engineer / architect and live in Hobart, Australia with my partner and our 5 1/2 year old daughter. I graduated from the Royal Melbourne Institute of Technology in 1999 with a Bachelors of Applied Science (Computer Science) and have been working for Unico, a Melbourne based IT company, ever since. I've primarily worked in the Telco space, dealing with call handling and provisioning systems with high throughput requirements and 5 9's reliability.

I became involved with Curator and ZooKeeper when implementing a system for coordinating application functions across geographically separated application instances.

In my spare time I like to climb rocks and run up hills.

Scott Blum

I'm a software engineer at Square, Inc since 2012, working on highly available system and infrastructure.  I got involved with Curator and Zookeeper while working on shared systems for cluster-level leases, locks, and other coordination data that was previously kept in databases.  Prior to Square, I was at Google for 7 years, where my main claim to fame is Google Web Toolkit (GWT) where I was the main developer on the Java-to-Javascript compiler, debugging environment, and JRE emulation libraries.

The Entire Team

Committer/PMC PMC

• Jordan Zimmerman

• Jay Zarfoss

• Eric Tschetter

• Ioannis Canellos

• Cameron McKenzie

• Scott Blum

• Patrick Hunt

• Mahadev Konar

• Luciano Resende

• Enis Söztutar

Notable New(ish) Recipes

• Tree Cache

• Persistent Ephemeral Node

• Node Cache

Service Discovery

• One of the most widely used Curator recipes!

Don’t Forget - It’s a Framework for Writing

Recipes

• Connection management

• Tons of utilities

• Simplifies most use-cases

Apache Curator 2.7.0

Apache Curator 2.7.0

Our Biggest Release Yet!

Curator RPC Proxy• Gives access to Curator to non JVM

languages/environments

• Organizations can unify their ZooKeeper usage across languages/environments (i.e. use Curator's Service Discovery recipe)

• The quality of ZooKeeper clients for some non-JVM languages is lacking

• There are Thrift implementations for a large number of languagesand environments

Nirmata Workflow• Nirmata Workflow is a Java Apache ZooKeeper and Apache Curator based

library that enables distributed task workflows.

• Can model simple to complex task relationships

• Manages task relationships and distributed scheduling

• Idempotent and non-idempotent tasks supported

• Custom task-types can be defined and targeted to specific servers

• Simple API

• Supports runtime cluster changes

• No single point of failure

• http://nirmataoss.github.io/workflow/

The Future

ZooKeeper 3.5.x Support

• Curator will fully support dynamic reconfig: CURATOR-160

• Curator will fully support watcher removal: CURATOR-161

• Investigating if Curator needs to do anything for local sessions: CURATOR-162

• We want to hear from the community if there’s anything else we need to do

Other Stuff

• Java 8?

• Rework some of the APIs (get rid of KeeperException, e.g.)

• More recipes

• More committers (could use 1 or 2 more)

Thank you!

Q&A

Jordan Zimmermanjordan@jordanzimmerman.com@randgalthttp://www.linkedin.com/in/jordanzimmermanhttp://aisallc.com