method and system for transparent database connection pooling

US008484242B1

(12) United States Patent (10) Patent N0.: US 8,484,242 B1 Singh et a]. (45) Date of Patent: *Jul. 9, 2013

(54) METHOD AND SYSTEM FOR TRANSPARENT 7,685,131 B2 3/2010 Batra et a1. DATABASE CONNECTION POOLING AND g? * Earth eta1~ ~~~~~~~~~~~~~~~~~ ~~ 707/706

QUERY QUEUING 832803955 B1 10/2012 T321221. _ _ _ 8,341,287 B2* 12/2012 Khemaniet a1. ............ .. 709/232

(75) Inventors: Varun S1ngh,Nav1Mumba1(1N);Uday 2003/0217173 A1 11/2003 Butt et 31, V. Sawanat, Mumbai (1N); Prateek 2004/0225865 A1 * 11/2004 Cox et al. ...................... .. 712/34

Gm" Rwkee (1N); Nmsh G- 33838118331 21* 53882 81517 “51' 707/3 I ' a eI'WOO . . . . . . . . . . . . . . . . . . . ..

DeshaVen"Mumba1(IN) 2006/0168318 A1 7/2006 Twiss _ 2007/0033433 A1 2/2007 Pecone et a1.

(73) Asslgneel ScalArcJIlcw M61110 Park, CA(US) 2008/0249997 A1* 10/2008 Sun et a1. ........................ .. 707/3

( * ) Notice: Subject to any disclaimer, the term of this (Continued) patent is extended or adjusted under 35 OTHER PUBLICATIONS U.S.C. 154(b) by 51 days.

“Oracle Database 11g Product Family,” an Oracle White Paper, Sep. Tlhi's patent is subject to a terminal dis- 2009, 19 pp‘ c a1mer.

(Continued) (21) Appl.No.: 12/862,525

Primary Examiner * Jean M Corrielus

(22) Flled? Allg- 24: 2010 (74) Attorney, Agent, or Firm * Warren S. Wolfeld; Haynes (51) I Cl Beffel & Wolfeld LLP

nt. .

G06F 17/30 (2006.01) (57) ABSTRACT (52) US. C]. R . . . oughly descr1bed, a database accelerator 1s 1nstalled trans

USPC ........................................................ .. 707/770 . . _ _ _ parently 1n a netWork betWeen database cl1ent and server

(58) Fleld of Classl?catlon Search systems. It maintains a pool of connections to the database USPC ............... .. 707/715, 741, 758, 802, 770, 737,

(56)

707/769, 745, 719, 713; 709/217 See application ?le for complete search history.

References Cited

U.S. PATENT DOCUMENTS

5,544,357 A * 8/1996 Huei ........................... .. 707/715

5,978,577 A * 11/1999 Rierden et a1. ...................... .. 1/1

6,061,349 A 5/2000 Coile et a1. 6,604,046 B1* 8/2003 Van Watermulen et a1. .. 701/533 6,665,710 B1 12/2003 Bates et a1. 6,728,720 B1 4/2004 Lenzie 7,343,396 B2* 3/2008 Kausik et a1. ............... .. 709/217

7,343,398 B1 3/2008 Lownsbrough 7,426,436 B1* 9/2008 Van Watermulen et a1. .. 701/532 7,506,368 B1 3/2009 Kersey et a1. 7,668,811 B2* 2/2010 Janssens et a1. ..... .. 707/999.003

server(s) for re-use as needed. Connection requests from cli ent systems are received and completed by the accelerator, and queries received on such client-side connections are for Warded through pool connections toWard the database serv ers. If no appropriate connections are available When needed for a particular query, the accelerator forms one by emulating a client system requesting a connection to the desired data base server. If the maximum number of connections has already been reached, then the query is queued until a con nection becomes available in the pool. By installing the accel erator transparently in the network, no modi?cations are required to the software in either the client systems or the database servers to add connection pooling functionality to an existing client/ server database environment.

13 Claims, 23 Drawing Sheets

REcEwE nUERv FROM cUENT svsvEM YHROUGH DAYAEASE AccELERAmR NETWORK FORT

ouERv DEYERMWED NOT w CACHE, so uuERv 1s FORWARDED T0 DATABASE sERvER

REcEwE RESPONSE OODE 8 DATA rRoM DATABASE sERvER

SYORE QUERY HASH, RESPONSE CODE 8 DATA w nAmaAsE ACCELERATOR CACHE

: (SOMETIME LATER)

1420

RECEWE SAME ouERv FROM cUEMT SYSTEM THROUGH DAYAEASE AccELERAmR NETWORK FORT

1422

QUERY FOUND m DATABASE AwELERAToR CACHE, s0 RETURN RESPONSE 000E 5 DNA T0 CLlENT svsrEM FROM

CACHE, DATABASE sERvER NOT CONYACYED new

US 8,484,242 B1

US. PATENT DOCUMENTS

2008/0276234 A1* 11/2008 Taylor et al. ................ .. 717/177 2009/0119504 A1 5/2009 van Os et al. 2010/0174939 A1 7/2010 Vexler 2010/0235901 A1 9/2010 Simpkins et al.

2010/0262650 A1* 10/2010 Chauhan et al. 2011/0191442 A1* 8/2011 Ovsiannikov ..

2011/0264905 A1* 10/2011 Ovsiannikov ..

2012/0246316 A1* 9/2012 Ramu et al. 2012/0278293 A1* 11/2012 BulkoWski et al.

. 709/203

. 709/218

. 713/151

. 709/226

. 707/703

OTHER PUBLICATIONS

“Of?oading Remote Authentication for Servers,” F5 Networks, Inc. White Paper, Apr. 2006, 6 pp. Goyvaerts J ., “Regular Expressions, The Complete Tutorial,” Jul. 2007, available at http://WWW.regular-expressions.info/print.html (visited Jun. 15, 2010), 197 pp. Standard Template Library Programmer’s Guide, 1994, HeWlett Packard Company: Index page (http://WWW.sgi.com/tech/stl/index. html), Introduction page (http://WWW.sgi.com/tech/stl/stliintroduc tion.html) and hashimap<Key, Data, HashFcn, EqualKey, Alloc> page (http://WWW.sgi.com/tech/stl/hashimap.htrnl), total 10 printed pages, all visited Jun. 14, 2010.

“MySQL Internals ClientServer Protocol,” MySQL Forge, available at http://forge.mysql.com/Wiki/MySQLiInternalsiClientServeri Protocol (visited Jun. 7, 2010) 23 pp. Lloyd, Expresso Developer’s Guide, Chap. 15, Database Connection Pooling, http://WWW.jcorporate.com/expresso/doc/edg/edgicon nectionpool.html (visited Jun. 4, 2010) 3 pp. Gil?llan, MySQL’s Query Cache, Database Journal (2003), WWW. databasejournal.com/features/mysql/article.php/3110171, visited Jun. 2, 2010, 9 pp. Of?ce Action mailed Feb. 2, 2012 in SCAL 1001-0. Of?ce Action mailed Mar. 9, 2012 in SCAL 1004-0. Jun. 27, 2012 Response to Of?ce Action mailed Mar. 9, 2012 in SCAL 1004-0. Of?ce Action mailed Jul. 6, 2012 in SCAL 1002-0. May 15, 2012 Response to Of?ce Action mailed Feb. 2, 2012 in SCAL 1001-0. Final Of?ce Action mailed May 31, 2012 in SCAL 1001-0. MaltZ et al., “TCP Splicing for Application Layer Proxy Perfor mance,” IBM Resear Science/Mathematics), IBM Research Divi sion, Mar. 17, 1998, 19 pages. Of?ce Action mailed Sep. 26 , 2012 in US. Appl. No. 12/853,650. Response to Final Of?ce Action dated Oct. 1, 2012 in US. Appl. No. 12/853,649.

* cited by examiner

US. Patent Jul. 9, 2013 Sheet 1 0123 US 8,484,242 B1

110-1 110-2 110-3 110-n

CLIENT 1 CLIENT2 CLIENT 3 CLIENT n

DATABASE CONNECTIONS

111

ACCELERATOR §

\“mmmmmmmmmmmx

DATABASE CONNECTIONS

DATABASE\ DATABASE DATABAsE\ SERVER : SERVER SERVER

112-1 112-2 CLUSTER I 12'“

US. Patent Jul. 9, 2013 Sheet 2 0f 23

Z11

ACCEPT DATABASE CONNECTION FROM CUENT

APPLICATION

ACCEPT DATABASE f 202 QUERY FROM CIJENT

APPLIOA'I'ION

206B

SERVER-SIDE CONNECTION FOR WRITE QUERY

S REPLY TO THE QUERY WITH CACHED DATA

SUCCESSFULLY END TRANSACTION

US 8,484,242 B1

US. Patent Jul. 9, 2013 Sheet 3 0m US 8,484,242 B1

SELECT SERVER-SIDE CONNECTION FOR WRITE UERY

/- 206B

215

heMwdmumthmclm L'm'rtbeenreacfmforallreadl

wrilsservers?

Yes

cream Hdd the In uonnechnega mil 3 readlwrileserver 0mm

a1drna|kit Bemes awiaue Awiable

216

T

(To Fig. 2D)


Yes (XNNECTDN TOA

207 SELECT EXISTING AVAILABLE

READ-O SERVER

f212 CREATEANEW

CONNECTION TOAREAD/ WRITE SERVERAND MARK IT AVAlLABLE

242 /.

l-DLD THE QUERY lN SELECTEXISTING QUEUE UNTILA CONNECTION TO

CONNECTION BECOMES AREADMIRITE AVAILABLE

(To Fig. 2D)


252

MARK THE IDENTIFIED SERVER-SIDE CONNECTION ACTIVE

I 254 FORWARD THE QUERY ON THE ACTIVE

CONNECTION TO TARGET DATABASE SERVER

I 256 RECEIVE RESPONSE CODE & DATA

WRITE TO TEMPORARY BUFFER; MARK SERVER sIDE CONNCETION INACTIVE

* 258 FORWARD RESPONSE CODE & DATA TO CLIENT APPLICATION ON CLIENT-SIDE CONNECTION

YES STORE RESPONSE 260 DATABASE CODE & DATA IN

RESPONSE CODE INDICATES CACHE |F SUCCESS? APPROPRIATE

L 262 4 i

264

TRANSACTION COMPLETE


ACCEPT DATABASE CONNECTION FROM CLIENT APPLICATION

201 x

326 \ EMULATE DATABASE SERVER

310

'5 YES AUTHENTICATION OFFLOAD TO AUTHENTICATE CLIENT

ENABLED? SYSTEM

312

\ FORWARD CONNECTION REQUEST TO A SELECTED NO DATABASE SERvER

314 l 328

\ FORWARD AUTHENTICATION TRANSMISSIONS BACK & YES ‘5W FORTH BETWEEN DATABASE SERvER AND CLIENT SYSTEM

316 330 318

NO WEBB

YES

320 PASS-THROUGH AUTHENTICATED CLIENT-SIDE AUTHENTICATED —\// CONNECTION NOW ESTABLISHED CONNEC‘HON NOW ESTABUSHED

332

324 i ,( END SUCCESS)


\ \ \ 412 2 \ \

VIRTUAL Dig-RE DATABASE

410-1 CUENT ACCELERAT

OR PHYSICAL DATABASE

/ TCP Stack TCP Stack / CLIENT MACHINE 417 412-1

[ HYPERVISOR LAYER | TCP Stack 4154

411-1

PHYSICAL NETWORK 418 | A |'\/

411-2

415-2 HYPERVISOR LAYER TCP Stack ‘ 412.3

412-4 ' ' ’

PHYSICAL DATABASE SERVER ' TCP Stack TCP Stack

MACHINE VIRTUAL VIRTUAL DATABASE DATABASE SERVER SERVER

\L \L L 4191 L\418-2 419.2

416 ' 410-2


C.05


IS THE UERY DATA

EBE?ENLlNMQHEZ

l GENERATE A HASH OF

THE QUERY ALONG WITH 610 RELATED METADATA

[204

CHECK THE HASH 612 AGAINST THE lN-MEMORY HAsH TABLE To FIND AN

EXACT MATCH

AS A MATC FOUND IN HASH

TABLE‘?

618

No HAS 'l'l'L FOR THE HASH KEY PRESENT EXPIRED? ' NOT PRESENT

620 616


REPLY TO THE QUERY WITH CACHED DATA

/ 205

READ THE MEMORY LOCATION OF THE CACHE 710

RESPONSE FROM THE HASH / TABLE

l REPLY TO THE QUERY WITH 712

THE CACHED RESPONSE / READ FROM THE MEMORY

LOCATION

US 8,484,242 B1

US. Patent Jul. 9, 2013 Sheet 11 m3 US 8,484,242 B1

T RERE P N E DE DATA 262

LA!

810

S RESPONSE DATA THIN SIZE LIMIT SPECIFIE FOR INDIVIDUAL MAXIMUM

CACHE ENTRIES? No

814

OES THE QUER MATCH ANY OF THE

USER-SPECIFIED REGEX PATTERNS‘?

816 GENERATE HAsH OF THE uuERY ALONG WITH RELATED MErAnATA

812 I \ cLEAR THE TEMPORARY

818 MAKE NEW ENTRY IN BUFFER HAsH TABLE FOR THE

HAsH KEY GENERATED

820 ADD POINTER To THE BUFFER

\ LOCATION OF THE RESPONSE CODE & DATA; ADD USER

CONFIGURED 1'I'L

I 822 ADD THIS NEW ENTRY T0

PERSISTENT INSERT QUEUE TO PERIODICALLY WRITE IN-MEMORY CACHE TO PERSISTENT STORAGE

Fig. 8


PROCESS FOR IN-MEMORY GARBAGE COLLECTION

910 READ SYSTEM TIMER, AND IF TRIG

ARRIVED. START IN-MEMO BAGE COLLECTION PROCESS

8 THE MEMORY OCCUPIED BY CACHE MOR THE THRESHOLD PULSORY GAR

C N?

ARE SUFFICIE PU RESOURCES AVAILABLE TO FORM R COLLECTION WITHOUT AC

PERFORMANCE?

Yes 914 YES

918

L THROUGH / EN HASH TABLE

Lo G FOR EX D TTLs

I 920 DELETE RED ENT No CLEAR TED MEM OCCUPIED BY EXP

RESPONSE DAT

922

ADD HASH TO QUEUE /— REMOVA L FROM SISTENT CACHE

916

WRITE EM TIMER FOR A F RE /_ SCH LED RUN OF GAR

COLLECTION

US. Patent

1012

1016

Jul. 9, 2013

PR E F RBA KIN T PERITENTTRAE

Sheet 13 0f 23

P A HED DATA

READ SYSTEM TIMER, AND IF TRIGGER TIME ARRIVED, START PERSISTENT STORAGE WRITE

PROCESS

/- 1010

OES THE WRIT QUEUE HAVE ANY NEW

HASH ENTRIES?

READ ALL NEW ENTRIES, INSERT THEM INTO PERSISTENT

STORAGE ALONG WITH RELEVANT METADATA

WRITE SYSTEM TIMER FOR A FUTURE SCHEDULED RUN OF PERSISTENT STORAGE WRITE

PROCESS / 1014

Fig. 10

US 8,484,242 B1

US. Patent Sheet 14 0f 23 Jul. 9, 2013

TIME ARRIVED, START PERSISTENT STORAGE GARBAGE COLLECTION

1110 \ READ SYSTEM TIMER, AND IF TRIGGER

PROCESS

DOES THE DELETE

ENTRIES?

Yes

1116 \ READ ALL ENTRIES, FIND RELEVANT HASH

ENTRIES IN PERSISTENT STORAGE, ISSUE DELETE REQUESTS

No

SCHEDULED RUN OF GARBAGE 1114 WRITE SYSTEM TIMER FOR A FUTURE

\ COLLECTION

US 8,484,242 B1


Process for Deleting Selected Data from Cache via User Interface

1210\ RECEIVE DELETE COMMAND FROM USER, WITH METADATA ABOUT QUERY PATTERNS

TO DELETE

I 1212\ QUERY PERSISTENT STORAGE FOR

MATCHING ENTRIES; DELETE THEM FROM PERSISTENT STORAGE AND ADD THEM TO

A DELETE LIST

I 1214\ CHECK PERSISTENT INSERT QUEUE FOR ANY

FURTHER ENTRIES MATCHING QUERY PATTERNS TO DELETE; REMOVE THEM FROM INSERT QUEUE

AND ADD THEM TO DELETE LIST

I 1216\ FOR ALL ENTRIES ON DELETE LIST, CLEAR HASH KEYS AND CORRESPONDING DATA

BUFFERS FROM IN-MEMORY CACHE

1218

FIG. 12

US 8,484,242 B1


Process for Restorin Data from Persistent Stora e to

MemmmBetm

1310 UPON REBOOT, START SYSTEM RESTORE

PRO CESS

1312

START ALL SYSTEM SERVICES / EXCEPT WRITING TO CACHE

DOES THE PERSISTENT DATA STORE HAVE ANY

CHE ENTRI .

1318

CHECK EACH CACHE / ENTRY AND VALIDATE

ITS TTL

1322\ f 1320 READ VALID ENTRIES FROM MARK ENTRY FOR DELETION AT A

PERSISTENT STORAGE, INSERT THEM FUTURE T|ME BY PERS|STENT INTO ‘II-MEMORY HASH TABLE- STORAGE GARBAGE COLLECTOR

CREATE AND FILL MEMORY AREAS FOR CACHED RESPONSE DATA

DECLARE SYSTEM FULLY FUNCTIONAL, ENABLE WRITING

TO CACHE 1316


SAMPLE SEQUENCE USING W

I, RECEIVE QUERY FROM CLIENT SYSTEM THROUGH

DATABASE ACCELERATOR NETWORK PORT

I, QUERY DETERMINED NOT IN CACHE, SO QUERY IS

FORWARDED TO DATABASE SERVER

I, RECEIVE RESPONSE CODE & DATA FROM DATABASE

SERVER

I, FORWARD RESPONSE CODE & DATA TO CLIENT SYSTEM

I, STORE QUERY HASH, RESPONSE CODE & DATA IN

DATABASE ACCELERATOR CACHE

I I I : (SOMETIME LATER) I I

RECEIVE SAME QUERY FROM CLIENT SYSTEM THROUGH DATABASE ACCELERATOR NETWORK PORT

QUERY FOUND IN DATABASE ACCELERATOR CACHE, SO RETURN RESPONSE CODE & DATA TO CLIENT SYSTEM FROM

CACHE; DATABASE SERVER NOT CONTACTED AGAIN

I

US 8,484,242 B1

A 4; A 4;

1420

1422


SAMPLE SE UENCE USING AUTHENTICATION OFFLOAD

RECEIVE PRELIMINARY CONNECTION REQUEST FROM J- 1510 CLIENT SYSTEM THROUGH DATABASE ACCELERATOR

NETWORK PORT

I, 1512 I RUN AUTHENTICATION PROCESS FOR THIS USER I-—f

I’ 1514 I CLIENT-SIDE AUTHENTICATED CONNECTION NOW ESTABLISHED U

I’ 1 1 RECEIVE QUERY FROM CLIENT SYSTEM THROUGH CLIENT-SIDE f 5 6

AUTHENTICATED CONNECTION ON DB ACCELERATOR NETWORK PORT

+ 1 17 I NO EXISTING CONNECTION CURRENTLY AVAILABLE TO DATABASE SERVER I\/— 5

1518 RUN AUTHENTICATION PROCESS FOR THIS USER TO DATABASE SERVER | ‘I I\/_ 1519 I SERVER-SIDE AUTHENTICATED CONNECTION NOW ESTABLISHED

I 1520 I FORWARD QUERY THROUGH SERVER-SIDE AUTHENTICATED CONNECTION /

6 1521 RECEIVE RESULT CODE & DATA FROM DATABASE SERVER THROUGH f

DATABASE ACCELERATOR NETWORK PORT

I, 1522 FORWARD RESULT TO CLIENT SYSTEM AND STORE IN CACHE IF APPROPRIATEI—/—

| :: (SOMETIME LATER) i 1524

RECEIVE SUBSEQUENT DATABASE CONNECTION REQUEST FROM CLIENT SYSTEM

RUN AUTHENTICATION PROCESS FOR THIS USER f

+ CLIENT-SIDE AUTHENTICATED CONNECTION NOW ESTABLISHED

+ RECEIVE SUBSEQUENT QUERY FROM CLIENT SYSTEM THROUGH CLIENT-SIDE

AUTHENTICATED CONNECTION ON DB ACCELERATOR NETWORK PORT 1530

1526

1528

E

i RETURN RESPONSE FROM CACHE OR FORWARD TO DATABASE SERVER 1532 THROUGH THE ABOVE SERVER-SIDE AUTHENTICATED CONNECTION OR _/— THROUGH ANOTHER SERVER-SIDE AUTHENTICATED CONNECTION

I

Fig. 15

method and system for transparent database connection pooling

Documents