Download - Diamon/Laser Design proposal
![Page 1: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/1.jpg)
Diamon/Laser Design proposal
Marek Misiowiec BE/CO/APMay 2010
![Page 2: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/2.jpg)
Core OverviewMulti-tier, distributed, fail-safe system processing diagnostic data collected repeatedly from a variety of sources. Raw data ofdifferent shapes is transformed into unified formats and furtheranalyzed on a middle layer according to well-defined rules.
Actions may be taken, in form of alarms or monitoring informationdelivered to clients. Processed data is made available as JAPCparameters or for dedicated GUIs. It is stored in long term archive
too. Downward communication with the source devices is possible.2
devs
apps DAQ
raw data
unifieddata
SRV
clients
client view
![Page 3: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/3.jpg)
Core Overview Diamon/Laser core responsibilities:
• data acquisition for various sources – push & pull modes• communication through layers, publishing• data transformation, processing, analysis, conditioning• failover, database separation, scaling
complementary functionalities:• rules engine• logging, archiving, configuration updates
non-core:• definitions & rules, data provider’s workflow• GUIs, dedicated client applications
3
![Page 4: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/4.jpg)
Importance of a changeNowadays: upgrades mess
• stale sources, legacy constraints, dusty components meeting only part of the requirements
• partial data, simplistic thresholds, no conditioning monitoring based on alarms - Diamon on Laser
• misuse of the system, overhead• mediocre scaling• badly-behaving sources/clients cannot be detached/isolated
legacy solutions hiding in Laser world• SonicMQ, OC4J, EJBs, Hibernate, Sleepycat
...and no standard CO components in place
4
![Page 5: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/5.jpg)
Importance of a change
We would like to: widely use CO solutions and help to establish others worth it better drive interactions with the bottom and upper tiers consolidate logics in system core be able to draw more conclusions on collected data sustain upgrades, updates, dependency changes
5
![Page 6: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/6.jpg)
Transition
6
JMSNotifier
oc4j-diamon
AlarmArchive
Agent Messages
Checks surveillance
Alarms, filters non registered
Alarms
Result Messages
Sends update/Requests config
CLIC Agents
Central Agents
CMWJMXWIE TWIDO SNMP
PING
jdaemon
Configuration
Agentsto be
configured
configures
commands
commands
Router
CCC Consoles
LASER
ALARMS
YAMI protocol
JMS
![Page 7: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/7.jpg)
Transition
7
JMSNotifier
oc4j-diamon
AlarmArchive
Agent Messages
Checks surveillance
Alarms, filters non registered
Alarms
Result Messages
Sends update/Requests config
CLIC Agents
Central Agents
CMWJMXWIE TWIDO SNMP
PING
jdaemon
Configuration
Agentsto be
configured
configures
commands
commands
Router
CCC Consoles
LASER
ALARMS
YAMI protocol
JMS
OC4J, EJBs, Hibernate,
BerkeleyDB
OC4J, EJBs, Hibernate,
BerkeleyDB
secondSonicMQ
firstSonicMQ
In-house DB
In-house DB
SNMP, CMW, variety of data
formats
directly to devices
not enough data
YAMI specific protocol
![Page 8: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/8.jpg)
Transition
8
JMS
spring AlarmArchive
Agent Messages
Result Messages
Sends update/Requests config
CLIC Agents
Central Agents
CMWJMXWIE TWIDO SNMP
PING
jdaemon
Configuration
Agentsto be
configured
configures
commands
commands
CCC Consoles
YAMI protocol
JMS
LASERPROXY
LASERCMW
![Page 9: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/9.jpg)
Transition
9
spring
AlarmArchive
Result MessagesSends update/
Requests config
CLIC Agents
Central Agents
CMWJMXWIE TWIDO SNMP
PING
jdaemon
Configuration
configures
commands
CCC Consoles
YAMI protocol
JMS
LASERPROXY
LASERCMW
![Page 10: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/10.jpg)
Transition
10
spring
AlarmArchive
Result Messages
Data Acquisition Layer
CLIC AGENTS
CMWJMXWIE TWIDO SNMP
PINGConfiguration
commands
CCC Consoles
JMS
LASERPROXY
LASERCMW
Rules, thresholds, distribution, analysis, publishing...
JAPC
other applications
![Page 11: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/11.jpg)
Technical outcomeThen: unified view of complete data sound construction on CO materials
• OC4J -> Spring, Hibernate -> JDBC Templates, EJBs -> POJOsSonicMQ -> ActiveMQ, multitude of data formats -> JAPCin-house database -> CCDB + Logging
• wider choice of middleware: CMW, YAMI, JMS focusing logics in one place
• rules, thresholds, alarm generation• conditioning: machine mode, working hours, state of subsystems
driving the whole process • detachable sources, clients• less restart/reboot downtime• convergence with similar efforts
11
![Page 12: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/12.jpg)
information exchange
12
japc-ext-yami japc-ext-cmw japc-ext-snmp jms-to-japc proxy japc-ext-?
Core Services
rules thresholds
alarm generator
externalsconditions
publisher w-flow
consoles
client middleware: JMSclientinter
actions
other apps
JAPCprovider
ConfigDB
archiverArchive
DB
Core
DAQ [1] DAQ [2] DAQ [n]
tackleproblematic
sources
Clicagents CMW
srcsLaserWie, Twi
?other
sources
source middleware protocols
![Page 13: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/13.jpg)
Available technologies
Building blocks: CO standard components eg. JAPC CO less standard, but demanded eg. japc-ext-yami Laser dev code base, ready to reuse eg. caching, distribution TIM being refurbished and struggling with similar future other CO projects: INCA
13
![Page 14: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/14.jpg)
Lessons learned from Laser eradication of legacy concepts & technologies pays off to scale well, we need to distribute processing power
• a supervisor over multiple workers• dividing is difficult
separation of concerns single processing unit as a set of loosely coupled actions
• patterns: chain of responsibilities, observable• upward flow: source input -> processing -> publishing
cater for database outages through local caching JMS+RMI as a middleware for all parties involved
• more advanced features of JMS channel help ...but no JAPC concepts were introduced at all
14
![Page 15: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/15.jpg)
Device/property model to clarify the current incoherent view strong incentives:
• speak one language• let everyone fit (Logging)• common tools, well-established in CO world
migration of legacy sources• expose properties• eradicate laser-source as it is
interaction with devices through JAPC calls domain entities:
• Alarm, Alarms, Diagnostic,...
15
![Page 16: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/16.jpg)
JAPC extensions to unify view on a variety of devices/applications
available now:• CMW (japc-ext-cmwrda) and JMS (remote) as most popular
we need some more:• YAMI through japc-ext-yami is already available,
used in new Clic agent• SNMP through japc-ext-snmp under development
16
![Page 17: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/17.jpg)
Bottom tierExperience gained in: current Diamon and Laser
• eg. CMW Alarm Monitor as a huge data acquisition process LHC Concentrators , similar problems INCA data acqusition layer
Progress:
17
source type current planned/done
Clic Agents JMS, YAMI japc-ext-yami
CMW Alarm Monitor Laser gateway JAPC deviceLaser sources C/Java laser-source Laser-to-JAPC proxyWiener, Twido SNMP central agents japc-ext-snmpPing central agent JAPC device
![Page 18: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/18.jpg)
Already developed JAPC extensions current version of YAMI meets our requirements Diamon Clic agent
• fully YAMI-based, JAPC values CMW Alarm Monitor refurbished
• 6000 subscriptions to GM and FESA devices• will be incorporated into new core
proxy between Laser production alarms and JAPC values• for the legacy sources
18
![Page 19: Diamon/Laser Design proposal](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816468550346895dd652ef/html5/thumbnails/19.jpg)
Open questionsStill to think over: best components and collaborations to choose level of distribution and scaling shared memory, distributed cache storage: short-term, long-term, Logging DB rules handling: homemade, proprietary ...and many others
19