designing role-based database systems to achieve unlimited database scalability
TRANSCRIPT
![Page 1: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/1.jpg)
![Page 2: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/2.jpg)
2
Designing Role-Based Database Systems to Achieve Unlimited Database Scalability
Fernando G. GuerreroCEOSolid Quality Mentors
Session Code: DAT451
![Page 3: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/3.jpg)
5
What forced me to prepare this session
The trigger
![Page 4: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/4.jpg)
6
The "unpredictable workload" case
Millions of copies of a win app around the worldAll of them waiting for file safety, before using them
At undefined timesUnlimited number of potential files to checkUnpredictable level of workload at any date and timeUnpredictable geographical distribution of workload
Potentially, millions of requests per hourCan we solve it?
![Page 5: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/5.jpg)
7
The "unreasonable scalability" case
The system already reached its limitsThey need to support workload 50 times higher
Scaling up is not an optionScaling out is not a simple task
They need to support over 15K transactions per second, with textual based data
Can we solve it?
![Page 6: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/6.jpg)
8
Fernando G. GuerreroCEO of Solid Quality Global LLC, USAPresident of the SolidQ subsidiaries around the [email protected]
Microsoft Regional Director for Spain since 2004SQL Server MVP from year 2000 till 2007Usual suspect at many international conferences
Introductions
![Page 7: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/7.jpg)
9
The SolidQ recipe
4 SQL Server MVPs old friends speaking at a conference in Palm SpringsA pleasant dinner at a nice restaurant2 bottles of my favorite Spanish wine1 bottle of a great single Malt WhiskyLots of honest and trustful work for 6 years
![Page 8: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/8.jpg)
10
What are you going to see and hear today
This session
![Page 9: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/9.jpg)
11
This session is about…
Lesson learned from “unreasonable” and amazing real systems from real enterprise customersBased on actual work performed by our Solid Quality mentorsLearning to think out of the box when designing DB systemsThinking first, acting later
![Page 10: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/10.jpg)
12
This session is NOT about…
CodeDemosObscure trace flags and registry hacksAny particular commercial product or service offeringDescription about standard scalability techniques
![Page 11: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/11.jpg)
13
First things First
![Page 12: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/12.jpg)
14
First… Listen
![Page 13: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/13.jpg)
15
Then, try to understand what their needs really are…
And what they really want
![Page 14: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/14.jpg)
16
A database system is just one more tool to help achieving
business goals
![Page 15: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/15.jpg)
17
Your technical common sense is your most powerful tool
![Page 16: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/16.jpg)
18
If you don’t need to buy anything to implemnent a
solution, it does not mean that this is the cheapest solution
![Page 17: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/17.jpg)
19
Role based DB system design
A single box might be sufficientDifferent tasks might imply different rolesRoles might be incompatibleYou’ll never know until you measure it
![Page 18: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/18.jpg)
20
The unpredictable workload case
The Problem
![Page 19: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/19.jpg)
21
Role based DB system design
A single box might be sufficientDifferent tasks might imply different rolesRoles might be incompatibleYou’ll never know until you measure it
![Page 20: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/20.jpg)
22
Massive attack
![Page 21: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/21.jpg)
23
The unpredictable workload case
The Solution
![Page 22: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/22.jpg)
24
A common sense powered queue
Multi-layered server roles:The ReaderThe WriterThe Master Root
“Magic array” of virtual server namesSelf updated definitionsProprietary advanced hash-based checkingLocal vs remote storage
![Page 23: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/23.jpg)
25
The Magic Array
![Page 24: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/24.jpg)
26
The Reader role
Organized as a Readers tree with a Reader Root at the topVerifies signature of requested fileReplies to client with simple response:
SafeUnsafeUnknown
If unknown, checks parent Reader for further informationReceives updates from parent Reader
![Page 25: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/25.jpg)
27
The Writer role
Works as a Reader as wellOrganized as a flexible array of WritersSends signature to the Master Root
and sends file asynchronously if necessaryReplies immediately to client with simple response:
SafeUnsafeUnknown
Receives updates from Distributor
![Page 26: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/26.jpg)
28
The Master Root
Stores new files and signaturesCalls the qualification engine to qualify new filesDistributes updates to root ReaderDistributes updates to array of Writers
![Page 27: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/27.jpg)
29
Massive dynamic deployment
![Page 28: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/28.jpg)
30
The unreasonable scalability case
The problem
![Page 29: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/29.jpg)
31
Scalability needs Jan 2005
10
10090 90
300 300
Workload vs CapacityWorkload Initial C Improved C Series4
![Page 30: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/30.jpg)
32
Scalability needs Aug 2006
10
200
300
90 90 90
300 300
600
Workload vs CapacityWorkload Initial C Improved C
![Page 31: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/31.jpg)
33
Side note: is there such thing as a write-
only transaction?or…
how much writing activity do you really expect?
![Page 32: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/32.jpg)
34
Scalability needs Dec 2008
10 200 300 500
1500
3000
90 90 90 90 90 90600 600 600 600
2000
4000
Workload vs CapacityWorkload Initial CImproved C Role Based C
![Page 33: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/33.jpg)
35
Scalability needs beyond 2009
10 200 3005001500
3000
6000
12000
90 90 90 90 90 90 90 90600600 600 600 600 6002000
4000
8000
16000
Workload vs CapacityWorkload Initial CImproved C Role Based C
![Page 34: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/34.jpg)
36
The unreasonable scalability case
The solution
![Page 35: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/35.jpg)
37
Role-based architecture
![Page 36: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/36.jpg)
38
Detailed role assignment
![Page 37: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/37.jpg)
39
Technology alone cannot beat what an experienced engineer can design on the back of an
envelope
![Page 38: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/38.jpg)
40
Q & A
![Page 39: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/39.jpg)
www.microsoft.com/teched Tech·Talks Tech·Ed BloggersLive Simulcasts Virtual Labs
http://microsoft.com/technet
Evaluation licenses, pre-released products, and MORE!
Resources for IT Professionals
![Page 40: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/40.jpg)
Related Content
Breakout Sessions (session codes and titles)
Interactive Theater Sessions (session codes and titles)
Hands-on Labs (session codes and titles)
Hands-on Labs (session codes and titles)
![Page 41: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/41.jpg)
Track Resources
Resource 1
Resource 2
Resource 3
Resource 4
![Page 42: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/42.jpg)
1 Year Subscription!
Complete anevaluation onCommNet andenter to win!
![Page 43: Designing Role-Based Database Systems to Achieve Unlimited Database Scalability](https://reader037.vdocuments.mx/reader037/viewer/2022103115/556bef90d8b42a65458b4e7a/html5/thumbnails/43.jpg)
46
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED
OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.