Est
ablis
hing
the
Tech
nica
l Bas
is fo
r Tru
stw
orth
y N
etw
orki
ng
Robust Inter-Domain Routing
Addressing Systemic Vulnerabilities in BGP
Doug Montgomery ([email protected]) Manager, Internet and Scalable Systems Research
https://www.nist.gov/itl/antd/internet-scalable-systems-research
Systemic Vulnerabilities
• Faults / Accidents • Attacks
2
WIRED: HOW A TINY ERROR SHUT OFF THE INTERNET FOR PARTS OF THE US
https://www.wired.com/story/how-a-tiny-error-shut-off-the-internet-for-parts-of-the-us/
Targeted Internet Traffic Misdirection
https://dyn.com/blog/mitm-internet-hijacking/
Broad Range of Threats and Motivations • Financially motivated attacks
• Other motivations …
3
https://www.secureworks.com/research/bgp-hijacking-for-cryptocurrency-profit
https://arstechnica.com/information-technology/2017/04/russian-controlled-telecom-hijacks-financial-services-internet-traffic/
https://arstechnica.com/information-technology/2010/11/how-china-swallowed-15-of-net-traffic-for-18-minutes/
https://dyn.com/blog/iran-leaks-censorship-via-bgp-hijacks/
Broad Range of Risks • Attacks to leverage other vulnerabilities
• Attacks to Disable Infrastructure
4
https://blog.thousandeyes.com/amazon-route-53-dns-and-bgp-hijack/ https://www.csoonline.com/article/3258748/security/the-mirai-botnet-explained-how-teen-scammers-and-cctv-cameras-almost-brought-down-the-internet.html
BGP Systemic Vulnerabilities
5
• Threats • Route hijacks
– Steers traffic away from legitimate servers • Address squatting
– Hijacks a not-in-service prefix and sets up spam servers • Route detours
– Modifies path causing data to flow via the attacker • Route leaks
– Announces routes in violation of ISP policy.
– Ramifications – Exploitations commonly result in outages, spam,
misrouting of data traffic, eavesdropping on user data, DDoS, etc.
Route Hijacks • Pakistan Telecom Hijack of YouTube - 2008
6
Hijack of Routes to Critical Infrastructure
7
April, 2018
https://blog.thousandeyes.com/amazon-route-53-dns-and-bgp-hijack/
Route Leaks – Violations of Implicit Policy • Hathway / Airtel Route Leaks of Google Prefixes
8
March2015-Incidentanalysis:http://research.dyn.com/2015/03/routing-leak-briefly-takes-google/
Scale of Problem Space • Global Routing System • Recurring Threats
• Solution Constraints • Behavioral Impacts • Installed base • Economic Incentives • Business / trust models
9
https://securityintelligence.com/bgp-internet-routing-what-are-the-threats/
https://blog.apnic.net/2018/01/10/bgp-in-2017/
State of the Solutions Space • BGP Origin Validation (BGP-OV)
• Global public key infrastructure and protocol elements to enable BGP routers to verify that the origin AS in a BGP update, was authorized by the prefix owner to announce that route.
• BGP Path Validation (BGP-PV) • Leverages the same PKI to enable each AS to digitally sign a BGP update,
proving that each AS in the PATH authorized the route announcement to its next hop.
• BGP Route Leak Detection / Mitigation • BGP protocol modifications to allow networks to detect that a BGP routed path
violates typical customer-provider-peer policies for route redistribution.
10
BGP Origin Validation (BGP-OV) • BGP Origin Validation (BGP-OV)
• Core specifications are complete • IETF SIDR Working Group - https://datatracker.ietf.org/wg/sidr/documents/ • IETF SIDR Ops WG - https://datatracker.ietf.org/wg/sidrops/documents/
• Commercial implementations and production services exists. • All 5 RIRs operate production RPKI services • Commercial and opensource routers support RPKI-based origin validation.
• Initial pilot and operational deployments are emerging. • New specifications and implementations are emerging for:
• Deployment optimizations and implementation clarifications • Extensions to support additional security services.
11
BGP Origin Validation Components
12
Certification • Securing routes to
your addresses • Get certificates for your
address space • Sign and publish ROAs
Two Sides of RPKI
13
Resource Public Key Infrastructure
Origin Validation • (Securing your
routes to others’addresses) • Retrieve ROAs from
other CA repositories • Validate received routes
against the RPKI data
Two Sides of RPKI
14
• RPKI Certificate Hierarchy
• Rooted trust anchors at each RIR
• Sub allocations represented by CA certificates.
• ROAs signed by certificate holders.
• RPKI Objects published in repository.
• Hosted Model • All RPKI operations hosted by
RIR.
RPKI Resource Certificates
15
• Delegated Model • Up / Down protocol to
register resources. • Users operate their own
RPKI Certificate Authorities. • Publication protocol to
publish RPKI objects • Operates own RPKI
repository or uses public aggregator.
RPKI Resource Certificates
16
Validation in single AS • Local RPKI validating
caches synchronize with global repositories.
• Caches do all crypto / PKI validation operations.
• Routers only receive a digested lists of ROA data.
• No crypto on the router!
RPKI Origin Validation
17
Regional RPKI Services • Production RPKI services in RIRs:
• AFRINIC: • http://afrinic.net/en/initiatives/rpki-certification
• APNIC: • http://www.apnic.net/services/services-apnic-provides/resource-certification
• ARIN • https://www.arin.net/resources/rpki/
• LACNIC: • https://rpki.lacnic.net/rpki/
• RIPE NCC: • http://www.ripe.net/certification/
18
• To use RPKI data for BGP origin validation, you will want to deploy one or more “validating caches”. • These tools collect and cache
global RPKI data, perform X.509 validation on the objects,
• … and then provides a highly summarized version to eBGP speaking routers.
• The RPKI-to-RTR protocol enables eBGP routers to download this processed data for route filtering.
• Multiple open source validating cache implementations are available!
RPKI Validating Caches
19
• RPKI Origin Validation requires a router that can: • Interface with a RPKI validating cache to
download lists of authorized origins: • <prefix, max_length, origin_AS>, …..
• Match incoming BGP updates against the list of authorized origins.
• Enforce local policies based upon on the results of these matches: • Valid, Invalid, Unknown
• Major router vendors support these capabilities in shipping products today!
RPKI Router Implementations
20
Est
ablis
hing
the
Tech
nica
l Bas
is fo
r Tru
stw
orth
y N
etw
orki
ng
Implementations & Tools • RPKI Infrastructure • RIPE validator 2
• https://www.ripe.net/manage-ips-and-asns/resource-management/certification/tools-and-resources
• RIPE validator 3 • https://github.com/RIPE-NCC/rpki-validator-3/wiki
• Routinator - Open source Relying Party - NLnet Labs • https://github.com/NLnetLabs/routinator
• RPKI.net Open Source Implementation of RPKI Tools • https://github.com/dragonresearch/rpki.net/
• RPSTIR - BBN Validation Tool • https://sourceforge.net/projects/rpstir/
• Router Implementations • Cisco
• https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/command/irg-cr-book/bgp-m1.html#wp3677719851
• Juniper • https://www.juniper.net/documentation/en_US/
junos12.2/topics/topic-map/bgp-origin-as-validation.html • Nokia
• https://infoproducts.alcatel-lucent.com/cgi-bin/dbaccessfilename.cgi/9300731102_V1_7750%20SR%20OS%20Router%20Configuration%20Guide%2012.0.R4.pdf
• Quagga / FRR, BIRD • https://www.nist.gov/services-resources/software/bgp-
secure-routing-extension-bgp-srx-prototype • http://rtrlib.realmv6.org/
• Go BGP • https://github.com/osrg/gobgp
21
Est
ablis
hing
the
Tech
nica
l Bas
is fo
r Tru
stw
orth
y N
etw
orki
ng
RPKI Based Origin Validation Trends • Significant Deployments
• RIPE • https://rpki-monitor.antd.nist.gov/?
p=2&s=0 • Implementation of RPKI and IRR
filtering on the AMS-IX platform • https://www.ripe.net/support/training/
ripe-ncc-educa/presentations/use-cases-stavros-konstantaras.pdf
• Cloudflare
• https://blog.cloudflare.com/rpki-details/
• Ongoing BGP-OV Specifications • SIDROPs WG
• https://datatracker.ietf.org/wg/sidrops/documents
• Autonomous System Provider Authorization
• https://datatracker.ietf.org/doc/draft-azimov-sidrops-aspa-profile/
• https://datatracker.ietf.org/doc/draft-azimov-sidrops-aspa-verification/
• Drop Invalid if Still Routable • https://datatracker.ietf.org/doc/draft-
sriram-sidrops-drop-invalid-policy/ • Cloudflare GoRTR
• https://github.com/cloudflare/gortr
22
Est
ablis
hing
the
Tech
nica
l Bas
is fo
r Tru
stw
orth
y N
etw
orki
ng
BGP Path Validation (BGP-PV) • BGPSec (BGP-PV)
• Core specifications are complete • IETF SIDR Working Group - https://datatracker.ietf.org/wg/sidr/documents/ • IETF SIDR Ops WG - https://datatracker.ietf.org/wg/sidrops/documents/
• RFC 8205 – BGPsec Protocol Specification • RFC 8208 – BGPsec Cryptographic Algorithms • RFC 8206 – Operational BCP • RFC 8374 – BGPsec design rationale / discussion
• Commercial implementations and production services are lagging. • Two open source reference implementations exist.
• Ongoing Research in Performance Optimizations and Deployment Architectures.
23
Est
ablis
hing
the
Tech
nica
l Bas
is fo
r Tru
stw
orth
y N
etw
orki
ng
BGPSec Design • BGP Path Validation (BGP-PV)
• RPKI Support for Router Keys. • Validating Cache’s provide public keys to routers. • BGP-PV routers digitally sign and validate each hop in the BGP path.
• “BGPsec Protocol Specification”, IETF RFC 8205.• "BGPsec Algorithms, Key Formats, and Signature Formats", IETF RFC 8208.
• Crypto on the router!
24
BGP-Secure Routing eXtension (BGP-SRx) • Open Source Reference Implementation
• Software router with extensions for: RPKI/RTR protocol, maintenance ROA distilled data, ROV and RPKI-aware BGP route policies.
• Full support of BGPSec path validation • Designed to support experimentation with different
architectural configurations of SRx and RPKI components and different trade-offs performance and router impact.
• BGP-SRx Status • SRx Server • SRx API • Quagga SRx (integrates SRx API into Quagga
router) • src & yum repository:
• https://www-x.antd.nist.gov/bgpsrx/
25
Est
ablis
hing
the
Tech
nica
l Bas
is fo
r Tru
stw
orth
y N
etw
orki
ng
BGPsec Performance Optimization • Protocol Optimizations
• "Design and analysis of optimization algorithms to minimize cryptographic processing in BGP security protocols.", Computer Communications.
• Cryptographic
Optimizations • "
High Performance BGP Security: Algorithms and Architectures(link is external)",NANOG69.
26
0
2,000
4,000
6,000
8,000
10,000
12,000
0 20 40 60 80 100 120
Num
berofVerificatio
ns
durin
gPe
akSecon
d
Time(minutes)
UnoptimizedCCS-ECBPO-EC
WorkloadinthePeak-SecondofeachMinuteduringa Two-HourTimePeriod
40,000
24,649
12,3258216
61624930
4108 3521 3081 2739 2465
1,000
10,000
100,000
1 2 3 4 5 6 7 8 9 10
Upd
ates
Pro
cess
ed p
er S
econ
d
AS Path Length (#Sigs in BGPsec Update)
Sign onlyVerify only
Xeon®CPUE3-1285v43.5GHz
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
50,000
SignP256 VerifyP256 SignP384 VerifyP384
Ope
ratio
nsperse
cond
OpenSSL1.1.0
taraEcCRYPT-3
Est
ablis
hing
the
Tech
nica
l Bas
is fo
r Tru
stw
orth
y N
etw
orki
ng
BGP Route Leak Detection
27
• IETF IDR and Grow WG • https://tools.ietf.org/html/draft-ietf-idr-route-leak-detection-mitigation
GeneralPrinciplesofDesignC:SolutionUsingBGPCommunities• Considering Community based encoding of RLP info for faster adoption • Wish to limit the number of RLP entries so that they can be accommodated in 1
or 2 Community attributes per update. Reason: Avoid having a long string of Community attributes per BGP update because the more they are, the lesser the chance that they will all make it through. If some get dropped, then the rest become useless. Also, save memory, simplify processing, and improve robustness.
• Based on the analysis and knowledge we have so far about RLP/eOTC, independent of encoding (Attribute or Community), at the minimum the RLP info must include: Ø ASN of the RLP aware AS that most recently asserted that it sent update to a
customer or peer; let us call this DO = Down Only indication Ø Leak warning: L = Leak indication
Ø L = ASN of the first RLP aware AS in the path that is forwarding route from customer or lateral peer in spite of detecting a leak § AS in question is avoiding unreachability (absence of alternative route)
Note: RLP = Route Leak Protection; DO and L together constitute RLP
IllustrationofDownOnly(DO)andLeak(L)indications–1of2
Note: RLP = Route Leak Protection; DO alone or DO and L together constitute RLP
1
2
[DO = 1]
6
3
P2Corp2pC2P
q
AS does not participate in RLP and starts/restarts a leak
n
Legend:
[DO = 1]
4
[DO = 1, L = 3]
[DO = 3, L = 3]
C2P
5[DO = 1, L = 3]
C2P
7
[DO = 6, L = 3]
P2C
P2C
Once a route is tainted with L = X, it remains tainted with the same (L = X) when it propagates. This is “stickiness” of L.
8[DO = 3, L = 3] p2p
9
[DO = 8, L = 3]
P2C
No alternative route for q
1[DO = 1]
2
q
AS does not participate in RLP and starts/restarts a leak
n
Legend:
3[DO = 1] p2p
4[DO = 3, L = 3]
P2C
P2CNo alternative route for q
5C2P
P2C
[DO = 5]
IllustrationofDownOnly(DO)andLeak(L)indications–2of2
1
2
q
3[DO = 2] p2p
4
[DO = 4, L = 4]
P2C
No alternative route for q
6
C2P
P2C
[DO = 6]
C2P[DO = 2]
p2p
5
Note: RLP = Route Leak Protection; DO alone or DO and L together constitute RLP
EncodingRLPinBGPCommunities
Relevant RFCs: RFC 4360: BGP Extended Communities Attribute RFC 7153: IANA Registries for BGP Extended Communities RFC 8092: BGP Large Communities Attribute