Belle II Conditions Database statusCarlos Fernando Gamboa ([email protected]), Ruslan Mashinistov, Benedikt Hegner. 32nd Belle II General Meeting, February 4th – 8th 2019
Overview
• Status of Conditions Database (CDB) server. • Development activities and future work.
2
Status of CDB server
3
Status of CDB server
3
Belle II CDB Remote service accessibility (WAN/LAN)
June 19 2018 30th Belle II General Meeting 4
ConditionsCDB database
(Tier 1,US BNL)
WAN
LANTIER 1
BNLWorker Nodes
KEKDAQ
Tier 1sGE,CA,IT
Interactive Users
Interactive Users
KEKcc
Calibration cycle
Belle II CDB Metadata servicereview
Architecture deployed using Kubernetes / Docker framework and controlled via puppetkubelet-1.9docker-1.12b2s(0.6.9.8): Java application on Payara micro 4.1.2
Database replicated for reliability purposesPostgres 9.6 hot standby replication
Database service
Read OnlyRead/Write
Node 1 Node 2
Belle2db service
5
Production activities
Software Infrastructure• Upgraded Squid’s Docker image with Squid 3.5 version and package distribution
• Legacy PNNL’s container image uses an OSG non supported repository.• Deprecated redundant CDB web User Interface code.
CDB code and configuration updates• b2s: Enhanced protection for a Global Tag to prevent a change to its status once it is published.
b2s:0.6.9.6
• b2s:0.6.9.8 fix baseUrl dependency on HTTP/HTTPS and literal default port assignation.
• UI: Swagger interface using secure HTTP by default.• Password protected, contact me or DB coordinators to request access.
Authentication/Authorization for CDB write requests set in place.Isolation between delete and write/update requests.
1. cdbreader access to the swagger interface.2. commonDBuser have access to POST and PUT HTTP.3. cdbcoordinator restricted to HTTP DELETE method.
Updates applied on production system
6
7
CDB metadata (database component)in numbers
950K rows
340MB
520 connections
Day
Number of table/database rows read 1/1/19 to 1/1/27
Usage driven by Read access
Day
103.4k READs
Performance CDB service (belle2db services) The year so far CDB
CDB service resource usability within the capacity of resources provisioned.
8
Network CDB activity
CPU database activity
Transmitted Received
9
Performance CDB service (belle2db services)The year so far CDBOverall service response in terms of HTTP response codes
Cumulative accounting of HTTP response HTTP response
Metadatabelle2db
HTTP response 2xx
HTTP response 4xxClient side
1.7M of HTTP 2XX succeeded
No significant HTTP 5xx server side errors
Need to identify/understand along with Belle2 database group and other experts root cause of client related errors.
Day
Day
10
Performance CDB service (belle2db-file services)The year so far CDB
Cumulative accounting of HTTP response
Overall service response in terms of HTTP response codes
HTTP response 4xxClient side Errors
HTTP response 2xxsuccessful response 4M/requests
Day
Day
5M of HTTP 2XX succeeded
No significant HTTP 5xx server side errors
Development activities and future work.
11
Authentication filter
Authentication filter
● Extracts JWT from the Request’s header
● Verifies the signature● Parce the JWT’s
Payload● Confront User/Group
against requested API endpoint
API
Request+
JWT
● Developing the AuthenticationFilter
○ Intercept request and do some pre-processing before hit to the API
● JWT signature based on shared secret
● Future plans: Users/Groups managing
Development and Future work
12
Authentication filter
Authentication filter
● Extracts JWT from the Request’s header
● Verifies the signature● Parce the JWT’s
Payload● Confront User/Group
against requested API endpoint
API
Request+
JWT
● Developing the AuthenticationFilter
○ Intercept request and do some pre-processing before hit to the API
● JWT signature based on shared secret
● Future plans: Users/Groups managing
Json Web Token (JWT) technology been reviewed
Currently working in identifying a test scenario that allows full test integration with client and server.
Development and Future work (cont.)• Secure HTTPs enabled
• To support dynamic protocol (HTTPS and HTTP) for CDB payloads.
• Consideration of database replication technology• Prototype for multi site standby replica of CDB
metadata.
13
CDB readiness for real data takingRequesting feedback from Belle II Software and Computing communities:Are there any items that are critical to address before final data taking?
What are the items on the critical path?Otherwise,
Are we fully ready?
14
Conclusion• CDB production service have been consistently working
stable.• Need to coordinate along with Belle II database group
and community to reduce client side related errors.• Robustness in terms of security has been increased
• Working along with the Belle II database group to evolve a granular secure Authentication/Authorization infrastructure.
• Replication of CDB metadata to external site being considered.
15
Backup slides
16
17
Performance CDB service (belle2db-file services)The year so far CDB
CDB files/payloadsbelle2db-files Network CDB activity
CPU database activity