mysql performance metrics you can't measure - using regression instead
TRANSCRIPT
KNOWING THE UNKNOWABLE
TO KNOW THAT WE KNOW WHAT WE KNOW, AND TO KNOW THAT WE DO NOT KNOW WHAT WE DO NOT KNOW, THAT IS TRUE KNOWLEDGE.
― NICOLAUS COPERNICUS
CERTAIN INVENTIONS DISCLOSED IN THIS PRESENTATION MAY BE CLAIMED WITHIN PATENTS OWNED OR PATENT APPLICATIONS FILED BY VIVIDCORTEX, INC.
LOGISTICS
SLIDES AND RECORDING WILL BE EMAILED TO YOU AFTER THE WEBINAR
TWEET QUESTIONS TO @VIVIDCORTEX AT ANY TIME (AND FOLLOW US!)
PLEASE OBSERVE THE LIGHTED EXIT SIGNS AND TURN OFF ALL ELECTRONIC DEVICES ;-)
BARON SCHWARTZ
@XAPRB ON TWITTER [email protected] LINKEDIN.COM/IN/XAPRB
Optimization, Backups, Replication, and more
Baron Schwartz, Peter Zaitsev &
Vadim Tkachenko
High PerformanceMySQL
3rd Edition
Covers Version 5.5
VARIATIONS
CONSTRAINED REGRESSION SMOOTHING AND SAMPLING STEP REGRESSION LOCAL REGRESSION LOGISTIC REGRESSION DECISION TREES AND RANDOM FORESTS BAYESIAN REGRESSION; MLE ENSEMBLES MACHINE LEARNING WAVELET DECOMPOSITION AND FFT COMMERCIAL SOFTWARE
PROBLEMS
TOO COMPLEX; TOO GENERAL
TOO SLOW AND COSTLY; O(N2) IN X-VARS
PARTIAL RESULTS
FOOLED BY CORRELATED X-VARS
0 100 200 300 400 500 600 700
2e+0
63e
+06
4e+0
65e
+06
6e+0
67e
+06
Index
q.63
68bf
5907
564a
9f ti
me
0 100 200 300 400 500 600 7000
5000
1000
015
000
2000
0Index
e.78
5f8a
c3c1
ea1c
93 ti
me
0 100 200 300 400 500 600 700
5.0e
+07
1.0e
+08
1.5e
+08
Index
Serv
er C
PU ti
me
QUERY TIMES / SERVER CPU TIME
2e+06 4e+06 6e+06
5.0e
+07
1.5e
+08
Query q.6368bf5907564a9f time
Serv
er C
PU ti
me
0 5000 10000 15000 20000
5.0e
+07
1.5e
+08
Query e.785f8ac3c1ea1c93 time
Serv
er C
PU ti
me
QUERIES VS CPU
REQUIREMENTS
MEMORY & CPU EFFICIENT FOR LARGE DATASETS
FULL RESULTS
NO PRECOMPUTATION
REASONABLE ACCURACY
SIMPLE & PHYSICALLY REALISTIC
INSIGHTS
NON-NEGATIVE SLOPES
IDENTICAL DIMENSIONS
INDEPENDENC OF X-VARS
ALL VARS SIGNIFICANT
VARS ROUGHLY SIMILAR
ION CANNONS NOT NEEDED
2e+06 4e+06 6e+06
5.0e
+07
1.5e
+08
Query q.6368bf5907564a9f time
Serv
er C
PU ti
me
0 5000 10000 15000 20000
5.0e
+07
1.5e
+08
Query e.785f8ac3c1ea1c93 time
Serv
er C
PU ti
me
2e+06 4e+06 6e+06
5.0e
+06
1.5e
+07
2.5e
+07
Query q.6368bf5907564a9f time
Allo
cate
d Se
rver
CPU
tim
e
0 5000 10000 15000 20000
020
000
4000
060
000
Query e.785f8ac3c1ea1c93 timeAl
loca
ted
Serv
er C
PU ti
me
DESCRIPTIVE STATS
QUERY CLASS SAMPLES R SLOPE T-VALUE INTERCEPT T-VALUE
Q.6368BF5907564A9F 719 0.98 3.65 0.0054 0.000083 0.98
E.785F8AC3C1EA1C93 711 0.98 3.09 0.0053 993.4 2.12
sample 001 actual CPU time
pred
icte
d C
PU ti
me
sample 002 actual CPU time
pred
icte
d C
PU ti
me
PREDICTIONS VERSUS ACTUAL MEASUREMENTS *PERFECT ACCURACY WOULD BE SLOPE = 1.0, R2 = 1.0, AND MAPE = 0%
SLOPE 0.97 R2 0.96
MAPE 5.9%
SLOPE 1.00 R2 0.91
MAPE 5.9%
0 100 200 300 400 500 600 700
−10
010
2030
40
sample 001
erro
r%
0 100 200 300 400 500 600 700
−20
−10
010
2030
4050
sample 002
erro
r%
RESIDUAL PERCENTAGES
sample 001 % error
Freq
uenc
y
−10 0 10 20 30 40 50
020
4060
80
sample 002 % error
Freq
uenc
y
−20 −10 0 10 20 30 40 50
020
4060
80
RESIDUAL HISTOGRAMS
−5.0e+07 0.0e+00 5.0e+07 1.0e+08 1.5e+08
5.0e
+07
1.0e
+08
1.5e
+08
Total Query Time
Serv
er C
PU T
ime
LITMUS TEST
QUESTIONS? [email protected] • @XAPRB
REFERENCES
DETAILED TECHNICAL WHITE PAPER: HTTPS://VIVIDCORTEX.COM/WHITE-PAPERS/WLR/
SAMPLE DATA AND CODE: HTTPS://GITHUB.COM/VIVIDCORTEX/WLR
FLICKR/AVDEZIGN/9731087346/
FLICKR/BENSTEVINSON/4908627827/
FLICKR/ANTONIOCASTAGNA/8491556471/
DESMOS.COM/CALCULATOR/PZMEB05TMG
ONLINECOURSES.SCIENCE.PSU.EDU/STAT557/
FLICKR/DGOODPHOTO/5264024028/
FLICKR/ORIGOMI/1377081666/
FLICKR/OBLONGPICTURES/5744192958/
FLICKR/21078769@N00/8456666615/
FLICKR/CHRIS_GIN/3535620878/
FLICKR/IKS_BERTO/3491779722/
FLICKR/KWERFELDEIN/1467887167/
FLICKR/SUN_SAND_SEA/10070098076/
FLICKR/JSMOORMAN/2298671281/
PHOTO CREDITS