[rakuten techconf2014] [a-4] rakuten ichiba

52
1 Rakuten Ichiba. Vol.01 Oct/25/2014 Takao Shiono ISDOD(Ichiba Service Development and Operation Department), Rakuten Inc.

Upload: rakuten-inc

Post on 03-Jul-2015

525 views

Category:

Technology


7 download

DESCRIPTION

Rakuten Technology Conference 2014 "Rakuten Ichiba" Takao Shiono (Rakuten)

TRANSCRIPT

Page 1: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

1

Rakuten Ichiba.

Vol.01 Oct/25/2014

Takao Shiono

ISDOD(Ichiba Service Development and Operation Department), Rakuten Inc.

Page 2: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

2

The purpose of this presentation is to introduce our

company ,Rakuten and share the issues for our futu

re development.

Objective

Page 3: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

3

Agenda

1.About Rakuten1-1. Japan Business1-2. Global Business1-3. Business KPI

2.System Situation2-1. Network2-2. Database2-3. Application2-4. Summary

3.Organization

Page 4: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

4

0. Introduce myself.

Page 5: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

5

0. About meAbout my Career.2014/04/01 Rakuten Ichiba Development and Operation Department manager.

2013/06/27 Board of director of STYLIFE.

2012/03/30 Board of director of NETS PARTNERS.(- 1st Dec 2012.)

2011/11/17 Executive officer of RAKUTEN.Inc.

2011/10/22 Development Managing Officer.

2011/01/01 Japan Ichiba Section Manager & Senior Service Producer of Japan Rakuten Ichiba.

2009/05/01 Shopping & Auction Section Vice manager & Senior Service Producer of Auction.

2008/07/01 Portal Service Section Manager & Senior Service Producer of Infoseek.

2008/02/01 New Service Division Vice manager.

2007/04/01 Portal Produce Department manager.

2006/04/01 Portal media company Produce department.

2005/12/01 Portal media company Infoseek CWD/MKT Department manager.

2005/01/01 Corporate Development planning department & Quality Assurance team Supervisor.

2004/04/01 Rakuten Ichiba ID produce department.

2004/02/01 Development Head Office.

Page 6: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

6

1. Rakuten

Page 7: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

7

Rakuten,Inc.

Founded: February 7, 1997

IPO: April 19, 2000 (JASDAQ Stock Exchange)

Office: Rakuten Tower (Tokyo, Japan)

Employees: 9,311 (as of Dec. 2012)

Market Cap: JPY \2,332 Billion (as of Jan 17, 2014)

Page 8: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

8

Most E-commerce

Uniqueness

Efficiency

Controllable Middleman.

Direct Sales

Page 9: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

9

RakutenE-commerce

Uniqueness

Bazaar

Entertainment

Pla

tform

Encounter Platform

Page 10: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

10

Rakuten Ecosystem

Page 11: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

11

Expanding Business Portfolio

Taiwan

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

Page 12: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

12

1-1. Japan Business

Page 13: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

13

E-Commerce Personal Finance

Travel / Booking Pro SportsCommunication

Digital Contents

Service for Japan Customer

Page 14: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

14

About Rakuten Ichiba in Japan

http://global.rakuten.com/corp/about/strength/data.html

Page 15: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

15

About Rakuten Ichiba in Japan

http://global.rakuten.com/corp/about/strength/data.html

Page 16: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

16

About Rakuten Ichiba in Japan

Page 17: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

17

1-2. Global Business

Page 18: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

18

Rapidly Expanding Worldwide from 2010

English-nization

/Globalization.

Global Expansion

Page 19: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

19

Global Expansion

E-Commerce

eBook

Travel

Other services & businesses

Rakuten Institute of Technology

Development center

Head Office / Regional Headquarters

Head Office

• E-commerce in 14 countries and regions

• All services and businesses in 28 countries

Page 20: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

20

JapanEU US

ASIA

Expanding Rakuten Ecosystem

Page 21: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

21

Taiwan SingaporeIndonesia

89% 54%72%

Brand Awareness

* Aided awareness among general internet users.

Survey Method: Internet survey (panel sampling), July

2014

Malaysia

54%

Page 22: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

22

1-3. Business KPI

Page 23: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

23

Rakuten Ichiba GMS

0

100

200

300

400

500

600

700

800

900

1,000

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

(JPY bn)

2010 2011 2020?

10,000

・・・

over 1 trillion$ 1 = JPY 79 over $ 13 Billion

Page 24: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

24

The BigData in Rakuten

• 93,870,000+ users

• 800,000,000+ purchase info

• 100,000,000+ reviews

• 3,000,000+ hotel booking per month

• 41,000+ merchants

• 60,000+ hotels

• Bank, Credit Card….

• Tremendous amount of search queries

• Several hundreds GB access log per day

• etc

• increasing more and more.

Page 25: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

25

Rakuten Data Explosion

0

20000000

40000000

60000000

80000000

100000000

120000000

140000000

160000000

180000000

200000000

1997 2005 2006 2007 2008 2009 2010 2011 2012 2013

The number of item

Page 26: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

26

2. System Situation

Page 27: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

27

2-1. Network

Page 28: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

28

Network Traffic Between 2005 to 2008

2.0G

4.0G

6.0G

8.0G

10.0G

12.0G

14.0G

2007/12 2008/12

16.0G

18.0G

20.0G

2009/12 2010/12 2011/12

トラフィックトレンドの実績と予測

2006/122005/12

15.0G

22.0G

Page 29: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

29

Network Traffic Between 2005 to 2008

2.0G

4.0G

6.0G

8.0G

10.0G

12.0G

14.0G

2007/12 2008/12

16.0G

18.0G

20.0G

2009/12 2010/12 2011/12

トラフィックトレンドの実績と予測

2006/122005/12

15.0G

22.0G

Page 30: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

30

Network Traffic From 2009.

2009 2010 2011 2012 2013

Victory sale 108Gbits/sec

Super sale 67Gbits/sec

Super sale 42Gbits/sec

Page 31: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

31

Reason of Data Increase

Page 32: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

32

2-2. Database

Page 33: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

33

History of Database

M9000

(2009)

ExaData

(2013)SF E25K

(2004)

SF 15K

(2002)

E10K

(2001)E420R

(1999)

E4500

(1999)

E450

(1997)

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

Page 34: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

34

Nothing changed for over 10 years

History of Database

App App App App App

App App App App

Few APP

Few DB

Many APP

Many DB

Page 35: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

35

Issues

We have to set the capacity size manually. It is a database specification.

- Max extents

- Max pagesize

- Highly loaded Database.

Data have been increasing day by day

We have to :

- Increase capacity

- Change HW/SW for EOSL

- Version up Software

- Patch Software

Too much maintenance

Causes:

- A large number of dependency ( Node=755, Edge=1,233 )

- Too many connections ( 1DB ⇔ 66 Application )

- Diverse versions ( OS, MW, programing language, script )

Difficult to manage/use the data

Page 36: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

36

2-3. Application

Page 37: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

37

DB has not been abstracted.Each App connect to DB directly.

Case 1

Page 38: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

38

2-3. Application

DB

App A App B App C

API α

App D

Merchant / Consumer

Front

API

Data

If we need to change our database…

Case 1.

Page 39: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

39

2-3. Application

DB

App A App B App C

API α

App D

Merchant / Consumer

Front

API

Data

We have to modify & test many applications.

Case 1.

Page 40: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

40

2-3. Application

DB

App A App B App C

API α

App D

Merchant / Consumer

Front

API

Data

We will integrate many connections to a single API.

Case 1.

Page 41: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

41

2-3. Application

2 phase commit.

Case 2

Page 42: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

42

API

InventoryOrder

Order Management tool for Merchant.

Mobile

Application

PC

Application

Order complete

R

It has to do INSERT to Order DB and do UPDATE to Inventory DB at the same time.If either one fails, data mismatch will occur. It is really risky.

Case2.

2-3. Application

Page 43: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

43

2-3. Application

New API

InventoryOrder

Order Management tool for Merchant.

Mobile

Application

PC

Application

R

Queue

We are abolishing 2 phase commit.Order DB will be separated from Order application.

Order complete

Case2.

Page 44: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

44

2-4. Summary

Page 45: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

45

Action plans for future development

1. Reduce management difficultiesReduce App server

Reduce DB server

2. Reconstruct ApplicationReduce too much dependence on DB

‐ Decrease test scope

‐ Decrease development scope

3. Stop 2 Phase Commit (Change the

architecture)

‐ Decrease difficult session control

4. Reconstruct DB schema and design

Page 46: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

46

3. Organization

Page 47: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

47

Hiring Talented Engineers

Many employees have come to Japan.

Page 48: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

48

Hiring Talented Engineers0 50 100 150 200

Japan

China

India

Korea

USA

Bangladesh

Philippines

Taiwan

England

Indonesia

Australia

Canada

Sri Lanka

Brazil

Algeria

Argentine

Swiss

Thailand

Chile

Nepal

Pakistan

Hungary

Republic

Vietnam

Hong Kong

Other

25+

Page 49: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

49

Next Challenge

Page 50: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

50

Ideal Goal

Engineers from our overseas companies will

become able to develop RMS together.

Page 51: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

51

Summary

The purpose of this presentation is to introduce our company ,Rakuten and share the issues for our future development.

Page 52: [Rakuten TechConf2014] [A-4] Rakuten Ichiba

52