dns shotgun - realistic dns benchmarking€¦ · dns resolving recap client resolver auth 1 auth 2...
TRANSCRIPT
![Page 2: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/2.jpg)
Motivation
● Running DNS resolver power, cooling⇒● Power, cooling €€€⇒● Benchmarking ⇔ optimization
● ⇒ cost reduction
![Page 3: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/3.jpg)
DNS resolving recap
Client Resolver
Auth 1
Auth 2
Auth 3Cache
?
?
cache hit
cachemiss
![Page 4: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/4.jpg)
● $ man resperf
● Query list: tcpdump => text
● Ramp-up query traffic
● Find max QPS
● Response rate drops
Classic benchmarking: QPS QPS QPS!
time
QP
S
![Page 5: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/5.jpg)
Classic pitfalls
● No query timing
● Ignores TTL ⇒ unrealistic cache hit rate● QPS ramp-up
● Waits for cache hit rate increase ⇒ unrealistic● Resolver restart!
● Over-focuses on QPS!
![Page 6: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/6.jpg)
DNS Shotgun: Client-based approach
● How many clients can the resolver handle?
● Result depends on clients!
● IoT, mobile, desktop, mail server, …
![Page 7: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/7.jpg)
DNS Shotgun: Introduction
● Realistic DNS benchmarking
● New toolset
● Based on dnsjit by DNS-OARC● https://www.dns-oarc.net/tools/dnsjit
● Open-source
● https://gitlab.labs.nic.cz/knot/shotgun/● Very much work-in-progress!
![Page 8: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/8.jpg)
DNS Shotgun: Principle
● Phase 1: Data preparation
● Phase 2: Traffic replay
● Phase 3: Drawing pretty charts
![Page 9: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/9.jpg)
DNS Shotgun: Data preparation
● Analyze PCAP
● Pre-generate traffic for N clients
● 100k● 200k● 300k● …
![Page 10: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/10.jpg)
DNS Shotgun: Client simulation
● Replay pre-generated traffic
● Keep ± 1 second query timing– Realistic cache hit rate– ⇒ QPS varies over time
● Want higher "QPS"? Add clients!
![Page 11: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/11.jpg)
DNS Shotgun: Performance testing
● Simulate N clients
● Analyze respose rate + RCODEs● Monitor resource usage
● Increase N
● … as long as resolver can keep up● N = maximum # of clients
● for given input PCAP & connection parameters
![Page 12: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/12.jpg)
PowerDNS Recursor 4.2.0 with defaults
![Page 13: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/13.jpg)
PowerDNS Recursor 4.2.0 with defaults
Do not generalize!Do not generalize!
Measure it yourself!Measure it yourself!
Use your traffic capture!Use your traffic capture!
![Page 14: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/14.jpg)
BIND 9.14.6: synth-from-dnssec? tuning?
![Page 15: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/15.jpg)
BIND 9.14.6: synth-from-dnssec? tuning?
Do not generalize!Do not generalize!
Measure it yourself!Measure it yourself!
Use your traffic capture!Use your traffic capture!
![Page 16: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/16.jpg)
Knot Resolver 4.2.2 vs. to-be-4.3.0
![Page 17: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/17.jpg)
Knot Resolver 4.2.2 vs. to-be-4.3.0
Do not generalize!Do not generalize!
Measure it yourself!Measure it yourself!
Use your traffic capture!Use your traffic capture!
![Page 18: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/18.jpg)
Unbound 1.9.4
![Page 19: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/19.jpg)
Unbound 1.9.4
Do not generalize!Do not generalize!
Measure it yourself!Measure it yourself!
Use your traffic capture!Use your traffic capture!
![Page 20: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/20.jpg)
DNS Shotgun: Try it
● Very much work-in-progress
● Here be dragons! :-)● Try it anyway
● https://gitlab.labs.nic.cz/knot/shotgun● Sponsors needed!
● TCP/TLS/DoH support● Configurable connection reuse (pipelining, keepalive)
![Page 21: DNS Shotgun - realistic DNS benchmarking€¦ · DNS resolving recap Client Resolver Auth 1 Auth 2 Cache Auth 3? ? cache hit cache miss $ man resperf](https://reader034.vdocuments.mx/reader034/viewer/2022042513/5f5ae00dd8679915543546e3/html5/thumbnails/21.jpg)
Closing remarks
● DNS micro-benchmarks do not reflect real world
● HW & OS changes invalidate results
● Generalization is hard
● Compare using your config and your traffic● Interested in benchmarking?
● See full version of the talk!● https://ripe79.ripe.net/programme/meeting-plan/dns-wg/