![Page 1: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/1.jpg)
Iterative dashboards & monitors
CARMEL HINKS | SOF TWARE ENGINEER | ATLASSIAN
![Page 2: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/2.jpg)
Us, with our hands down
![Page 3: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/3.jpg)
You build it, you run it
![Page 4: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/4.jpg)
I addressed all operational concerns
You build it, you run it
Nice! We are finished forever!
![Page 5: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/5.jpg)
I addressed all operational
Nice! We are finished forever!
Past Present
![Page 6: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/6.jpg)
I addressed all operational
For now…
Past Present
![Page 7: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/7.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 8: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/8.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 9: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/9.jpg)
Dashboards MonitorsMetrics
Agenda
![Page 10: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/10.jpg)
MetricA measure of a software characteristic
![Page 11: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/11.jpg)
MetricA measure of a software characteristic
Analytic..?
![Page 12: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/12.jpg)
MetricWhat are our systems doing?
AnalyticWhat are our users doing?
![Page 13: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/13.jpg)
DashboardA visualisation of your metrics
![Page 14: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/14.jpg)
MonitorAn alert against one or more metrics
![Page 15: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/15.jpg)
Operational health
![Page 16: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/16.jpg)
Operational health
What went wrong?
![Page 17: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/17.jpg)
When did it go wrong?
Operational health
![Page 18: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/18.jpg)
Why did it go wrong?
Operational health
![Page 19: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/19.jpg)
Operational health
![Page 20: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/20.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 21: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/21.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 22: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/22.jpg)
![Page 23: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/23.jpg)
Multi-tenant
![Page 24: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/24.jpg)
![Page 25: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/25.jpg)
Database
App App App
Queue
![Page 26: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/26.jpg)
Database
App App App
Queue
![Page 27: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/27.jpg)
Database
App App App
Queue
What about cross-region latency?
![Page 28: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/28.jpg)
Database
App App App
Queue
What about cross-region latency?What about scale?
![Page 29: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/29.jpg)
Database
App App App
Queue
What about cross-region latency?What about scale? What about progressive rollouts?
![Page 30: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/30.jpg)
Database
App App App
Queue
What about cross-region latency?What about scale? What about progressive rollouts?What about outage blast radius?
![Page 31: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/31.jpg)
Database
App App App
Queue
What about cross-region latency?What about scale? What about progressive rollouts?What about outage blast radius?What about data sovereignty?
![Page 32: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/32.jpg)
Database
App App App
Queue
What about cross-region latency?What about scale? What about progressive rollouts?What about outage blast radius?What about data sovereignty?
What about noisy neighbours?
![Page 33: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/33.jpg)
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
![Page 34: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/34.jpg)
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Shard
![Page 35: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/35.jpg)
Sign me up to Jira!
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Provisioning pipeline
![Page 36: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/36.jpg)
Sign me up to Jira!
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Provisioning pipeline
![Page 37: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/37.jpg)
Sign me up to Jira!
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Provisioning pipeline
![Page 38: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/38.jpg)
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Provisioning pipeline
![Page 39: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/39.jpg)
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
![Page 40: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/40.jpg)
Shard Servic
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Shard Service
![Page 41: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/41.jpg)
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Shard Service
Database
App App App
Queue
![Page 42: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/42.jpg)
Shard Service
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
![Page 43: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/43.jpg)
Shard Service
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
![Page 44: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/44.jpg)
Shard Service
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
![Page 45: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/45.jpg)
Shard Service
Europe Australia USA
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
![Page 46: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/46.jpg)
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Database
App App App
Queue
Shard Service
Europe Australia USA
![Page 47: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/47.jpg)
Shard Service
Database
App App
Queue
App
![Page 48: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/48.jpg)
Shard Service
70% 20% 90% 5%
Shard A
![Page 49: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/49.jpg)
Shard Service
70% 20% 90% 5%
Shard A
Metrics from the shards, about the shards
![Page 50: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/50.jpg)
Shard Service
70% 20% 90% 5%
Shard A
![Page 51: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/51.jpg)
Wait…
![Page 52: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/52.jpg)
Understanding what to measure is hard
![Page 53: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/53.jpg)
Understanding what to measure is hard
What worked today, may not work tomorrow
![Page 54: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/54.jpg)
Understanding what to measure is hard
What worked today, may not work tomorrow
Keeping everyone & everything up to date is hard
![Page 55: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/55.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 56: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/56.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 57: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/57.jpg)
It is a capital mistake to theorise before one has data
SHERLOCK HOLMES
![Page 58: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/58.jpg)
Measure nothing
![Page 59: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/59.jpg)
Measure nothingMetrics aren’t verified before going live
![Page 60: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/60.jpg)
Measure nothingMetrics aren’t verified before going live
First incident is going to SUCK
![Page 61: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/61.jpg)
Measure nothingMetrics aren’t verified before going live
First incident is going to SUCK
This isn’t a solution, it’s a deferral
![Page 62: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/62.jpg)
Measure everything
![Page 63: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/63.jpg)
Measure everythingExpensive (time, money & resources)
![Page 64: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/64.jpg)
Measure everything
Lots of noise
Expensive (time, money & resources)
![Page 65: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/65.jpg)
Measure everythingExpensive (time, money & resources)
Lots of noise
Does not scale
![Page 66: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/66.jpg)
Measure stuff from out the box
![Page 67: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/67.jpg)
SHARD SERVICE DASHBOARD
![Page 68: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/68.jpg)
SHARD SERVICE DASHBOARD
![Page 69: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/69.jpg)
SHARD SERVICE DASHBOARD
![Page 70: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/70.jpg)
SHARD SERVICE DASHBOARD
![Page 71: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/71.jpg)
IF CPU > 80% for over 5 minutes THEN page
![Page 72: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/72.jpg)
Think of idea
Design service
Build service (MVP)
Reach operational maturity
Release
Iterate service
Iterate service
![Page 73: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/73.jpg)
Think of idea
Design service
Build service (MVP)
Reach operational maturity
Release
Iterate service
Iterate service
![Page 74: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/74.jpg)
Sign me up to Jira!
Database
Nod Nod Nod
Queue
Database
Nod Nod Nod
Queue
Database
Nod Nod Nod
Queue
![Page 75: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/75.jpg)
Sign me up to Jira!
Database
Nod Nod Nod
Queue
Database
Nod Nod Nod
Queue
Database
Nod Nod Nod
Queue
![Page 76: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/76.jpg)
Sign me up to Jira!
Database
Nod Nod Nod
Queue
Database
Nod Nod Nod
Queue
Database
Nod Nod Nod
Queue
![Page 77: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/77.jpg)
Sign me up to Jira!
Database
Nod Nod Nod
Queue
Database
Nod Nod Nod
Queue
Database
Nod Nod Nod
Queue
![Page 78: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/78.jpg)
Sign me up to Jira!
![Page 79: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/79.jpg)
What questions do we want to be able to answer with our operational resources?
TAKING A STEP BACK
![Page 80: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/80.jpg)
Shard ServicePerforms the selection of a suitable shard based on geographical location and dynamic capacity metrics
![Page 81: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/81.jpg)
Shard ServicePerforms the selection of a suitable shard based on geographical location and dynamic capacity metrics
Synchronous, http-facing
![Page 82: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/82.jpg)
Shard Service
Requests slow down significantly
![Page 83: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/83.jpg)
Shard Service
Requests slow down significantly
Requests are accepted, but then fail
![Page 84: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/84.jpg)
Shard Service
Requests slow down significantly
Requests are accepted, but then fail
Requests start being rejected
![Page 85: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/85.jpg)
Shard Service
Requests slow down significantly
Requests are accepted, but then fail
Requests start being rejected
There are no suitable shards
![Page 86: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/86.jpg)
Shard Service
Requests slow down significantly
Requests are accepted, but then fail
Requests start being rejected
There are no suitable shards
Incorrect shards were selected
![Page 87: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/87.jpg)
Shard Service
Requests slow down significantly
Requests are accepted, but then fail
Requests start being rejected
There are no suitable shards
Incorrect shards were selected
There is insufficient data to make decisions
![Page 88: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/88.jpg)
Shard Service
Requests slow down significantly
Requests are accepted, but then fail
Requests start being rejected
There are no suitable shards
Incorrect shards were selected
There is insufficient data to make decisions
Infrastructure metrics
Application metrics
![Page 89: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/89.jpg)
Shard Service
Requests slow down significantly
Requests are accepted, but then fail
Requests start being rejected
There are no suitable shards
Incorrect shards were selected
There is insufficient data to make decisions
Infrastructure metrics
Application metrics
Infrastructure health Useful metrics tied to components in your techstack.
![Page 90: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/90.jpg)
Shard Service
Requests slow down significantly
Requests are accepted, but then fail
Requests start being rejected
There are no suitable shards
Incorrect shards were selected
There is insufficient data to make decisions
Infrastructure metrics
Application metrics
Infrastructure health Useful metrics tied to components in your techstack.
Application health Useful metrics tied to the domain of your application
![Page 91: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/91.jpg)
Application metricsInfrastructure metrics + =
![Page 92: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/92.jpg)
Application metricsInfrastructure metrics +
Latency
Memory utilisation
Load balancer errors
=
![Page 93: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/93.jpg)
Application metricsInfrastructure metrics
Shard capacity
Errors logged
Shard selection reason
Latency
Memory utilisation
Load balancer errors
+ =
![Page 94: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/94.jpg)
Application metricsInfrastructure metrics
Shard capacity
Errors logged
Shard selection reason
Latency
Memory utilisation
Load balancer errorsMetrics about the shards, from Shard Service
+ =
![Page 95: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/95.jpg)
`
![Page 96: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/96.jpg)
Confluence Apdex Count metric: current vs target (by shard) Jira Apdex Count metric: current vs target (by shard) Database utilisation metric: current vs target (by shard)
Provisioning failures metric: current vs target (by shard) Average remaining capacity by shard (Jira Apdex) Average remaining capacity by shard (Confluence Apdex)
Top selected region (internal) Top selected region (AWS) Top selection reasons
Top selected shards
![Page 97: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/97.jpg)
SHARD SERVICE INFRASTRUCTURE
![Page 98: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/98.jpg)
Region capacity exhausted
Monitors
Surge in errors logged
Shard capacity exhausted
![Page 99: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/99.jpg)
How can you…
![Page 100: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/100.jpg)
How can you… Figure out what to measure?
![Page 101: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/101.jpg)
What questions do you want to answer?
![Page 102: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/102.jpg)
What questions do you want to answer?
Why does your service exist (what are its roles and responsibilities)?
What does it look like for those roles and responsibilities to degrade?
How can you verify whether or not such a degradation is occurring?
![Page 103: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/103.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 104: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/104.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 105: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/105.jpg)
Everything is right.
![Page 106: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/106.jpg)
Fine… or exploding Never checked operational health unless it was on fire
Noisy alerts Frequent & un-actionable
As time went on…
Things changed Because, you know, agile
![Page 107: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/107.jpg)
Elastic load balancer
Shard Service Node 1
Shard Service Node 2
Shard Service Node 3
As time went on…
![Page 108: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/108.jpg)
Elastic load balancer
Shard Service Node 1
Shard Service Node 2
Shard Service Node 3
LatencyLoad balancer errors
Healthy hosts
![Page 109: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/109.jpg)
Elastic load balancerApplication load balancer
Shard Service Node 1
Shard Service Node 2
Shard Service Node 3
![Page 110: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/110.jpg)
Application load balancer
Shard Service Node 1
Shard Service Node 2
Shard Service Node 3
LatencyLoad balancer errors
Healthy hosts
As time went on…
![Page 111: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/111.jpg)
Noisy alerts Frequent & un-actionable
As time went on…
Things changed Because, you know, agile
Fine… or exploding Never checked operational health unless it was on fire
![Page 112: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/112.jpg)
Noisy alerts Frequent & un-actionable
As time went on…
Things changed Because, you know, agile
Fine… or exploding Never checked operational health unless it was on fire
![Page 113: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/113.jpg)
Our team
Team who could actually fix the problem
![Page 114: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/114.jpg)
Noisy alerts Frequent & un-actionable
As time went on…
Things changed Because, you know, agile
Fine… or exploding Never checked operational health unless it was on fire
![Page 115: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/115.jpg)
Noisy alerts Frequent & un-actionable
As time went on…
Fine… or exploding Never checked operational health unless it was on fire
Things changed Because, you know, agile
![Page 116: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/116.jpg)
What level of service you can commit to offer
SERVICE LEVEL OBJECTIVE
![Page 117: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/117.jpg)
What level of service you can commit to offer
SERVICE LEVEL OBJECTIVE
E.g. 99.99% requests should succeed
![Page 118: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/118.jpg)
We were not alone
![Page 119: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/119.jpg)
Process dedicated to regularly reviewing, discussing and iterating on operational health
TECHOPS
![Page 120: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/120.jpg)
Develop measurable goalsTechOpsCollect data
Prepare a report
Meet and discuss
Repeat and iterate
![Page 121: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/121.jpg)
Develop measurable goalsTechOpsCollect data
Prepare a report
Meet and discuss
Repeat and iterate
![Page 122: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/122.jpg)
Develop measurable goalsTechOpsCollect data
Prepare a report
Meet and discuss
Repeat and iterate
![Page 123: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/123.jpg)
Develop measurable goalsTechOpsCollect data
Prepare a report
Meet and discuss
Repeat and iterate
![Page 124: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/124.jpg)
Develop measurable goalsTechOpsCollect data
Prepare a report
Meet and discuss
Repeat and iterate
![Page 125: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/125.jpg)
TechOps for everyone!
![Page 126: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/126.jpg)
Goal
TechOps for everyone!
![Page 127: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/127.jpg)
GoalReduce the number of noisy alerts
![Page 128: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/128.jpg)
DataReduce the number of noisy alerts
![Page 129: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/129.jpg)
DataAlerts received in the past weekReduce the number of noisy alerts
87
![Page 130: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/130.jpg)
87Alerts received in the past week
![Page 131: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/131.jpg)
87Low priority alerts
![Page 132: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/132.jpg)
Reduce the number of noisy alerts
87
Report
![Page 133: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/133.jpg)
ReportAlerts, dashboard screenshots, incidents…Reduce the number of noisy alerts
87
![Page 134: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/134.jpg)
Reduce the number of noisy alertsMeet & discuss
![Page 135: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/135.jpg)
Meet & discussActionable? Discoverable? Useful?Reduce the number of noisy alerts
![Page 136: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/136.jpg)
Meet & discussActionable? Discoverable? Useful?Reduce the number of noisy alerts
![Page 137: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/137.jpg)
Meet & discussActionable? Discoverable? Useful?Reduce the number of noisy alerts
![Page 138: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/138.jpg)
Meet & discussActionable? Discoverable? Useful?Reduce the number of noisy alerts
![Page 139: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/139.jpg)
ALERTS (ALL SERVICES, STAGING + PRODUCTION)
0
25
50
75
100
Week 1 Week 3 Week 5 Week 7 Week 9 Week 11 Week 13 Week 15 Week 17 Week 19 Week 21
Total High priority Low priority
![Page 140: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/140.jpg)
CASE #2 - RELIABILITY INCREASETotal High priority Low priority
![Page 141: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/141.jpg)
CASE #3 - ALERT REDUCTION
![Page 142: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/142.jpg)
How can you…
![Page 143: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/143.jpg)
Verify you’re measuring the right things?How can you…
![Page 144: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/144.jpg)
Review your operational resources!
![Page 145: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/145.jpg)
…frequently
Review your operational resources!
![Page 146: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/146.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 147: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/147.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 148: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/148.jpg)
Shard Service
Database
App App
Queue
App
![Page 149: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/149.jpg)
Shard Service
70% 20% 90% 5%
Shard A
![Page 150: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/150.jpg)
Shard Service
70% 20% 90% 5%
Shard A
![Page 151: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/151.jpg)
Shard A
Database
App
Queue
App App
![Page 152: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/152.jpg)
Shard A
Database
App
Queue
App App
Shard Sevice
![Page 153: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/153.jpg)
Shard A
Database
App
Queue
App App
Shard Sevice
![Page 154: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/154.jpg)
Shard A
Database
App
Queue
App App
Shard Sevice
![Page 155: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/155.jpg)
Shard A
Database
App
Queue
App App
Shard Sevice
![Page 156: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/156.jpg)
Shard A
Database
App
Queue
App App
Shard Sevice
![Page 157: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/157.jpg)
What’s the big deal?
![Page 158: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/158.jpg)
Confluence Apdex Count metric: current vs target (by shard) Jira Apdex Count metric: current vs target (by shard) Database utilisation metric: current vs target (by shard)
Provisioning failures metric: current vs target (by shard) Average remaining capacity by shard (Jira Apdex) Average remaining capacity by shard (Confluence Apdex)
Top selected region (internal) Top selected region (AWS) Top selection reasons
Top selected shards
![Page 159: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/159.jpg)
Confluence Apdex Count metric: current vs target (by shard) Jira Apdex Count metric: current vs target (by shard) Database utilisation metric: current vs target (by shard)
Provisioning failures metric: current vs target (by shard) Average remaining capacity by shard (Jira Apdex) Average remaining capacity by shard (Confluence Apdex)
Top selected region (internal) Top selected region (AWS) Top selection reasons
Top selected shards
Panel per metric
![Page 160: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/160.jpg)
Confluence Apdex Count metric: current vs target (by shard) Jira Apdex Count metric: current vs target (by shard) Database utilisation metric: current vs target (by shard)
Provisioning failures metric: current vs target (by shard) Average remaining capacity by shard (Jira Apdex) Average remaining capacity by shard (Confluence Apdex)
Top selected region (internal) Top selected region (AWS) Top selection reasons
Top selected shards
Panel per metric
Slow
![Page 161: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/161.jpg)
Confluence Apdex Count metric: current vs target (by shard) Jira Apdex Count metric: current vs target (by shard) Database utilisation metric: current vs target (by shard)
Provisioning failures metric: current vs target (by shard) Average remaining capacity by shard (Jira Apdex) Average remaining capacity by shard (Confluence Apdex)
Top selected region (internal) Top selected region (AWS) Top selection reasons
Top selected shards
Panel per metric
Slow, error prone
![Page 162: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/162.jpg)
Confluence Apdex Count metric: current vs target (by shard) Jira Apdex Count metric: current vs target (by shard) Database utilisation metric: current vs target (by shard)
Provisioning failures metric: current vs target (by shard) Average remaining capacity by shard (Jira Apdex) Average remaining capacity by shard (Confluence Apdex)
Top selected region (internal) Top selected region (AWS) Top selection reasons
Top selected shards
Panel per metric
Slow, error prone, forgettable
![Page 163: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/163.jpg)
Confluence Apdex Count metric: current vs target (by shard)Jira Apdex Count metric: current vs target (by shard)Database utilisation metric: current vs target (by shard)
Provisioning failures metric: current vs target (by shard) Average remaining capacity by shard (Jira Apdex) Average remaining capacity by shard (Confluence Apdex)
Top selected region (internal) Top selected region (AWS) Top selection reasons
Top selected shards
{ }Confluence Apdex Count metric: current vs target (by shard) Jira Apdex Count metric: current vs target (by shard) Database utilisation metric: current vs target (by shard)
Provisioning failures metric: current vs target (by shard) Average remaining capacity by shard (Jira Apdex) Average remaining capacity by shard (Confluence Apdex)
Top selected region (internal) Top selected region (AWS) Top selection reasons
Top selected shards
![Page 164: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/164.jpg)
{ }
{ }Application
{ }Test
Shard Service repository
ConfluenJira Databas
Provisioni Average Average
Top Top Top
To
ConfluencJira Databas
Provision Average Average
Top Top Top
To
![Page 165: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/165.jpg)
{ }
Shard Service repository Dashboard tool
Marge’s Service
Homer’s Service
Shard Service
{ }Application
{ }Test
ConfluenJira Databas
Provisioni Average Average
Top Top Top
To
ConfluencJira Databas
Provision Average Average
Top Top Top
To
ConfluenJira Databas
Provisioni Average Average
Top Top Top
To
ConfluencJira Databas
Provision Average Average
Top Top Top
To
![Page 166: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/166.jpg)
{ }
Shard Service repository Dashboard tool
Marge’s Service
Homer’s Service
Shard Service
{ }Application
{ }Test
ConfluenJira Databas
Provisioni Average Average
Top Top Top
To
ConfluencJira Databas
Provision Average Average
Top Top Top
To
![Page 167: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/167.jpg)
Discoverable
Operational resources as code
Front of mind
Version control
![Page 168: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/168.jpg)
Discoverable
Operational resources as code
Front of mind
Version control
![Page 169: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/169.jpg)
Discoverable
Operational resources as code
Front of mind
Version control
![Page 170: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/170.jpg)
Going a step further…
286Operational resources as code
![Page 171: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/171.jpg)
286
![Page 172: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/172.jpg)
286Dashboards
![Page 173: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/173.jpg)
JIRA SHARD DASHBOARD (SUBSET)
286
![Page 174: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/174.jpg)
Cue, templates
![Page 175: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/175.jpg)
This can be solved at the platform level
SERGEJS SINICA, ATLASSIAN SENIOR DEVELOPER
Cue, templates
![Page 176: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/176.jpg)
91
![Page 177: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/177.jpg)
91respondents who owned 1 to 10+ services
![Page 178: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/178.jpg)
83.5%
![Page 179: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/179.jpg)
83.5%maintained operational resources through the UI
![Page 180: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/180.jpg)
67%
![Page 181: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/181.jpg)
67%kept their team up to date via “tribal knowledge”
![Page 182: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/182.jpg)
23.1%
![Page 183: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/183.jpg)
23.1%satisfied with their existing process
![Page 184: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/184.jpg)
“We've been saying for eons that we should put our monitors and dashboards into code, but the task is too big to start so we don't do it. Over time the job just gets bigger and bigger and less likely to get done :P”
ATLASSIAN DEVELOPER
![Page 185: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/185.jpg)
Introducing, Sauron
![Page 186: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/186.jpg)
The “all seeing eye” for dashboards & monitors
Introducing, Sauron
![Page 187: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/187.jpg)
Dashboard tool
Shard Service
Sauron
Marge’s Service
Homer’s Service
![Page 188: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/188.jpg)
SauronExport my dashboard!
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
![Page 189: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/189.jpg)
SauronExport my dashboard!
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
![Page 190: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/190.jpg)
SauronExport my dashboard!
{ }
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
ConfJira DatProvi Aver Avera
ToToT T
ConflJira DatProvi Aver Aver
ToTT T
![Page 191: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/191.jpg)
SauronExport my dashboard!
{ }
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
ConfJira DatProvi Aver Avera
ToToT T
ConflJira DatProvi Aver Aver
ToTT T
![Page 192: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/192.jpg)
Sauron
{ }
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
ConfJira DatProvi Aver Avera
ToToT T
ConflJira DatProvi Aver Aver
ToTT T
![Page 193: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/193.jpg)
Sauron
Application repository
{ }
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
ConfJira DatProvi Aver Avera
ToToT T
ConflJira DatProvi Aver Aver
ToTT T
![Page 194: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/194.jpg)
Sauron
Application repository
{ }
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
![Page 195: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/195.jpg)
Sauron
Application repository
{ }{ }
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
![Page 196: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/196.jpg)
Sauron
Application repository
{ }
{ }
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
![Page 197: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/197.jpg)
Sauron
Application repository
{ }
Dashboard tool
Shard Service
Marge’s Service
Homer’s Service
![Page 198: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/198.jpg)
Monitors
Dashboards
Screenboards
shard-service operations
![Page 199: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/199.jpg)
> 50
![Page 200: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/200.jpg)
> 50Services adopted Sauron
![Page 201: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/201.jpg)
How can you…
![Page 202: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/202.jpg)
Help your team keep up to date with change?
How can you…
![Page 203: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/203.jpg)
Define operational resources in code
![Page 204: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/204.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 205: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/205.jpg)
Agenda
Iterative… what?
Setting some context
Deciding what to measure
Verifying your metrics
Keeping up with change
Summary
![Page 206: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/206.jpg)
![Page 207: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/207.jpg)
SelectLearn what questions you want to answer
![Page 208: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/208.jpg)
Select
Verify
Learn what questions you want to answer
Review, review, review
![Page 209: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/209.jpg)
Select
VerifyKeep up
Learn what questions you want to answer
Review, review, reviewDefine all the things in code
![Page 210: Iterative dashboards & monitors - Agile Australiaagileaustralia.com.au/2019/slides/agileaus-2019-carmel-hinks.pdf · Provisioning pipeline. Sign me up to Jira! Database App App App](https://reader033.vdocuments.mx/reader033/viewer/2022052614/6052cf953117417d0e79ef58/html5/thumbnails/210.jpg)
Thank you!