managed by ut-battelle for the department of energy best ever archive utility, yet (beauty) kay...
TRANSCRIPT
Managed by UT-Battellefor the Department of Energy
Best Ever Archive Utility, Yet (BEAUtY)
Kay Kasemir
April 2013
2 Managed by UT-Battellefor the Department of Energy
CSS-based OPICSS-based OPI
ArchiveArchive EngineArchiveArchive Engineconfig.xmlconfig.xml
IOCIOC
Binary Data Files
Data ServerData Server
“ASCII” Config.“ASCII” Config.Channel AccessChannel Access
XML-RPCXML-RPC
Previous Setup with Channel Archiver
History~2000:
Started byBob Dalesio
~2003:Index Tools,Data Server
~2007:CSS Client
3 Managed by UT-Battellefor the Department of Energy
Details Older Java “archive viewer” also available.
Archive Engines running on “sampling” computer, data server on “serving” machine.
Scripts to restart engines, copy data, update indices.
Headaches with data maintenance:– Index time grows with data
– Stuck when index files reach 2GB
– SNS Users faced with ~80 sub-archives
– No clue what needs fixing after network/power problems
– No idea who contributes how many samples
– No way to remove selected channels or time ranges
– Improving on this means implementing an RDB
4 Managed by UT-Battellefor the Department of Energy
CSS-based OPICSS-based OPI
ArchiveEngineArchiveEngine
config.xmlconfig.xml
IOCIOC
RDB(Oracle/MySQL)
RDB(Oracle/MySQL)
Channel AccessChannel Access
New Setup
SamplesSamplesConfig.Config.
EngineConfig-Import
Other tools for config & samples
5 Managed by UT-Battellefor the Department of Energy
RDB Alternatives
Data Browser– read from old Channel Archiver’s
ArchiveDataServer.cgi
– Read from RDB
– use your own site-specific ‘reader’ plugins
Archive Engine– Pluggable Configuration
– Pluggable ‘Write’ support
6 Managed by UT-Battellefor the Department of Energy
CSS Data Browser Handles Both New URL
Just one‘RDB’sub-archive
Old and newdata can becombined inone plot
7 Managed by UT-Battellefor the Department of Energy
View Archive Data in CSS Data Browser
Open Data Browser Plot
Add PV– If default archive setup lists data source with historic
data for the PV, it will be displayed
8 Managed by UT-Battellefor the Department of Energy
Searching for PVs in Archive
Open Archive Search View– Data Browser Perspective,
or Plot’s context menu
Search for channelsvia pattern
Send to new Data Browser– Via context menu
Send toexisting Plot– Via drag
& drop
9 Managed by UT-Battellefor the Department of Energy
Configure Sample Engines
Configuration is in RDB– Directly use SQL
– EngineConfigImport for legacy config files
– View/Edit via web
Hierarchical (as before)– Sampling engine (name, where to run, …)
Groups– Channels
No more duplicate channels!
10 Managed by UT-Battellefor the Department of Energy
Web Interface to Engine Config in RDB
Tomcat/JSP/Servlets to view and edit
11 Managed by UT-Battellefor the Department of Energy
Web Config View: Channel Stats
12 Managed by UT-Battellefor the Department of Energy
Web Config View: Sample Stats
OK
??
13 Managed by UT-Battellefor the Department of Energy
Stats 34 sample engines, 79400 channels
Host that runs sampling engine:– CPU load 45%, zero disk I/O wait, very responsive
Oracle performance– 8000 samples/sec peak in write tests
– Operationally maybe ¼ of that
Better configurationwould likely havefewer samples/sec
14 Managed by UT-Battellefor the Department of Energy
RDB Supported
Currently supported: MySQL or Oracle– Main table difference: Oracle TIMESTAMP has
nanosecs, MySQL has separate nanosecs col.
– All tools auto-configure based on JDBC URL
PostgreSQL support from ITER
15 Managed by UT-Battellefor the Department of Energy
Which one to use? Channel Archiver
– Fast
– Can’t be maintained
RDB– Slow
– Super easy to maintain
Worth a look: – SLAC Archiver Appliance
http://epicsarchiverap.sourceforge.net/
– HyperArchiver http://www.lnl.infn.it/~epics/joomla/archiver.html
– Both of them work with CSS Data Browser
16 Managed by UT-Battellefor the Department of Energy
Thanks!