RDB Archiver based on CS-Studio

%FOREACH{"css-address" in="https://github.com/ControlSystemStudio/cs-studio"}%

Introduction


(from http://cs-studio.sourceforge.net/docbook/archive.png)
The CS-Studio RDB Archiver Engine archives EPICS Process Variables into an RDB (PostgreSQL, MySQL, Oracle). Stored data can be retrieved via CS-Studio RDB Archive Reader or any other SQL aware application.

Documents


Setup / Installation

Needs:
  • Database configuration / optional installation
  • Archiver Engine configuration

Database

%FOREACH{"file" in="postgres_schema.txt"}% %FOREACH{"path" in="master/applications/archive/archive-plugins/org.csstudio.archive.rdb/dbd"}% %FOREACH{"prefix" in="SCSArchiverCSStudioPostgreSql"}%

PostgreSQL hosted by GSI central services


PostgreSQL hosted "locally" w/ option for NOSSL


CS-Studio - RDB Archiver

Currently located at lxhadeb06

Requirements

Sources download

Current versions can be build on its own based on the GIT hub repository of CS-Studio: Prebuilt binaries can be found at https://ics-web.sns.ornl.gov/css/updates/apps/ and/or https://controlssoftware.sns.ornl.gov/css_rcp/nightly/ Look for:
  • archive_config...
  • archvie_engine...
Currently used at HADES are those:
  • archive-config-4.1.0-linux.gtk.x86_64.zip : local copy
  • archive-engine-4.1.0-linux.gtk.x86_64.zip : local copy


File locations

Install Area
  • /home/epics/EPICS/programs/css/...
    • directory read-only to group and others
    • modifications on <Application>.ini file to allow multiuser operation "shared configuration"/"shared install"
  • archive-config-4.1.0
%CODE{"java"}% -Dosgi.configuration.area.default=@user.home/.css/ArchiveConfigTool_3.2.15/configuration -Dosgi.instance.area.default=@user.home/CSS-Workspaces/ArchiveConfigTool_3.2.15/workspace %ENDCODE%
  • archive-engine-4.1.0
%CODE{"java"}% -Dosgi.configuration.area.default=@user.home/.css/ArchiveEngine_4.0.0/configuration -Dosgi.instance.area.default=@user.home/CSS-Workspaces/ArchiveEngine_4.0.0/workspace %ENDCODE%
  • /home/epics/EPICS/bin/
    • Caller Scripts to provide correct java version of Oracle (java version "1.8.0_66", Sun/Oracle):
%CODE{"bash"}% #!/bin/bash export JAVA_HOME=/home/epics/EPICS/programs/java/jdk1.8.0_66; declare -F addToPath && addToPath $JAVA_HOME/bin || PATH=$(echo $JAVA_HOME/bin${PATH:+:$PATH} | sed -e 's/^://' -e 's/:$//') export PATH

/home/epics/EPICS/programs/css/archive-config-4.1.0/ArchiveConfigTool $* %ENDCODE%
%CODE{"bash"}% #!/bin/bash export JAVA_HOME=/home/epics/EPICS/programs/java/jdk1.8.0_66; declare -F addToPath && addToPath $JAVA_HOME/bin || PATH=$(echo $JAVA_HOME/bin${PATH:+:$PATH} | sed -e 's/^://' -e 's/:$//') export PATH

/home/epics/EPICS/programs/css/archive-engine-4.1.0/ArchiveEngine $* %ENDCODE%


Configuration Area
can be changed by runtime option
-configuration <location>
default
~/.css/ArchiveEngine_4.0.0/configuration
~/.css/ArchiveConfigTool_3.2.15/configuration

Instance Area
can be changed by runtime option
-data <location>
default
~/CSS-Workspaces/ArchiveEngine_4.0.0/workspace
~/CSS-Workspaces/ArchiveConfigTool_3.2.15/workspace



CS-Studio - Database credentials

There are several ways to provide the database passwords (archiving (write), retrieving (read):
  • every time via command line option :
    -rdb_password <plain text password>
  • every time via pluginCustomization :
    • command line option :
      -pluginCustomization <file>
    • where <file> contains the line:
      org.csstudio.archive.rdb/password=<plain text password>
  • one time command line option :
    -set_password org.csstudio.archive.rdb/password
    setting secure preferences interactively and storing it in secure_store.dat depending on the settings of org.csstudio.security/secure_preference_location in the (pluginCustomization) settings file (c.f Section "Secure Preferences", and Configuring Authentication, Authorization and Secure Preferences of "Chapter 13. Authentication and Authorization Part I. CS-Studio Guide" ):
    • "Default": Use the default location, which is typically the user's home directory.
      • Advantage: It's the default
      • Disadvantage: You won't always know where the preferences are.
    • "Instance": Use the Eclipse 'instance', i.e. the workspace.
      • Advantage: You know where it is, and each workspace will have its own settings.
      • Disadvantage: Each workspace has different settings.
    • "Install": Use the Eclipse 'install' location, i.e. where the product is installed.
      • Advantage: You know where it is, and every workspace for that product will have the same settings. Good for a "global" setup.
      • Disadvantage: Ordinary users cannot (should not) have write permissions.

Status

%FOREACH{"user" in="hades_scs_css_archive"}% At the moment the encrypted credentials for $user@pgsqltest.gsi.de:6846/hades_scs_css_archive are stored at the install location of the config tool or the engine respectively. Logged with installation credentials (e.g. user epics), either of these will set it: %CODE{"bash"}% $> su - epics $> . epics && ArchiveConfigTool -rdb_user $user -set_password org.csstudio.archive.rdb/password && ArchiveEngineCSS -set_password org.csstudio.archive.rdb/password %ENDCODE% %NEXT{"user"}%


GIT repository



Engine Configuration

  1. definitions
  2. convert old archiver "cfg" into new "XML"
  3. import configuration into configuration database
  4. list/check configurations
  5. goto operations

Code Repositories

CVS

  • GSI Linux Cluster
    CVSROOT
    /misc/hadesprojects/slowcontrol/cvsroot
    ==Module
    Archiver.css

git



Directory

%CODE{"bash"}% APPS_TOP=~scs/playground/EPICS/apps #APPS_TOP=~scs/apps ARCHIVER_CSS_TOP=$APPS_TOP/Archiver.css cd $ARCHIVER_CSS_TOP/configuration %ENDCODE%

Converting old archiver 'cfg' into new 'xml' configuration files

The configuration directory of the css archiver at $APPS_TOP/Archiver/css/configuration/old contains as a starting point the copy of the archiver configuration of the old archiver. By using the old/ConvertEngineConfig.pl script (which is a copy of the latest old archiver tool set) you could generate an xml configuration file:

%CODE{"bash"}% ConvertEngineConfig.pl -v -d old/engineconfig.dtd -o hadesArchive.xml hadesArchive.cfg %ENDCODE% or alternatively the wrapper script convert.sh which cares about existing files %CODE{"bash"}% old/convert.sh [[input:default=hadesArchive.cfg] output] %ENDCODE%


Import / Modify Engine Configuration Database

The XML output file of the conversion is used by ArchiveConfigTool to generate/modify the configuration of the archive engine, which is also stored in the database.
TIP pluginCustomization
to ease common rdb a plugin customization settings file is used, see plugin customization

hades_scs_css_archive

lxhadeb05_archive

lxhaddcs10_archive


List existing engine configurations

%CODE{"bash"}% pluginCustomization=$ARCHIVER_CSS_TOP/pluginCustomization/settings_hades_scs_css_archive.ini ArchiveConfigTool -pluginCustomization $pluginCustomization -list %ENDCODE%


Operation

Start Engine

by Hand

%CODE{"perl"}% engine_name=HadesArchiveEngine engine_port=4912 pluginCustomization=$ARCHIVER_CSS_TOP/pluginCustomization/settings_hades_scs_css_archive.ini

ArchiveEngineCSS -port $engine_port -engine $engine_name -data workspace/mdc -pluginCustomization $pluginCustomization %ENDCODE%

A running Engine will have an HTTP server at e.g. http://localhost:4912/main an will look like this:
ArchiveEngineWebInterface.png

ALERT! Make sure the engine port is the same as defined in the configuration, otherwise you might get something like this - (mismatch of defined 4812 (default) and requested 4912): %CODE{"java"}% 2016-06-02 15:56:51.010 INFO [Thread 1] org.csstudio.archive.engine.Application (start) - Archive Engine 4.0.0.201506301920 2016-06-02 15:56:51.233 INFO [Thread 1] org.csstudio.archive.engine.server.EngineServer () - Engine HTTP Server on http://localhost:4912/main 2016-06-02 15:56:51.233 INFO [Thread 1] org.csstudio.archive.engine.Application (start) - Reading configuration 'HadesArchiveEngine' 2016-06-02 15:56:51.858 SEVERE [Thread 1] org.csstudio.archive.engine.Application (start) - Cannot read configuration java.lang.Exception: Engine running on port 4912 while configuration requires http://localhost:4812/main

%ENDCODE%

using crontab / procServ

On the basis of the "by Hand" a suitable cronjob is set up: At ~scs/procServ the file archiverCSS.sh handles the configuration of the start routine. Its function userDefines holds the important informations, change it if necessary, and update it to the code revision.

The command crontab -l should contain the following line of code: %CODE{"bash"}% @reboot cd ~scs/procServ && ./archiverCSS.sh > ~/log/archiverCSS.crontab.start 2>&1 %ENDCODE%

With those scripts a procServ session with port 4853 is created, which can be locally reached and manipulated using telnet (see procServ man pages): %CODE{"bash"}% telnet localhost 4813 %ENDCODE%

Restarting Engine

via web browser

The RDB Channel Archiver offers the opportunity to restart the engine by simple calling its restart web page: http://localhost:4912/restart

script based using curl

Download this small bash script https://git.gsi.de/EPICS/Tools/scripts/Archiver/restartChannelArchiverArchiveEngine

via procServ - "brute force"

Login to the procServ session and kill/restart the child process with ^X.

Attach CSS Browser

lxhaddcs10

Settings needed in CSS preferences (restart needed) Just look at the line starting with jdbc:
  • Archiver preference settings 1:
    Archiver_preferences_settings_1.png
  • URL: jdbc:postgresql://lxhaddcs10:5432/archive

You must not enter a password:
  • Archiver preference settings 2:
    Archiver_preferences_settings_2.png
  • User: report
Example:
  • Archiver CSS Browser:
    Archiver_Example.png


Appendix:

pluginCustomization

with e.g. ~scs/playground/EPICS/apps/Archiver/css/pluginCustomization/settings_hades_scs_css_archive.ini %CODE{"perl"}% org.csstudio.archive.rdb/url=jdbc:postgresql://pgsqltest.gsi.de:6846/hades_scs_css_archive org.csstudio.archive.rdb/user=hades_scs_css_archive org.csstudio.archive.rdb/schema= org.csstudio.security/secure_preference_location=Instance %ENDCODE%

%NEXT{"css-address"}%


-- PeterZumbruch - 16 Jan 2018
-- PeterZumbruch - 2022-11-16

This topic: DaqSlowControl > HadesDaqDocumentation > DetectorControlSystem > SCSArchiverCSStudio
Topic revision: 2022-11-16, PeterZumbruch
Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki Send feedback | Imprint | Privacy Policy (in German)