4.4. Configuration

EOS has two parts of configuration:

Configuration File Types Description
External config file
shared library location implementing a component and static configuration
  • these files are typically once edited and during the lifetime of an instance very infrequently changed - a change of these files always requires a service restart!
Internal
configuration of filesystems, policies, mappings aso
  • these are stored using the EOS configuration engine inside EOS itself - any change is immediately active and never requires a server restart!

There are two methods to store external configuration: the classical EOS4 and the newer EOS5 approach. The EOS5 configuration aims to make the configuration more transparent and simpler using variable substitution for hostnames aso.

  • EOS4/Classical configuration
  • EOS5/New configuration

NOTE EOS5 configuration is not (yet) used in production instances at CERN.

Classical Configuration

The classical configuration includes the following configuration files:

Service Configuration File Type
MGM /etc/xrd.cf.mgm XRootD
FST /etc/xrd.cf.fst XRootD
MQ /etc/xrd.cf.mq XRootD
QDB /etxrootd/xrootd-quarkdb.cfg XRootD
ALL /etc/sysconfig/eos_env Sysconfig

You can find an example sysconfig file with explanation of configuration variables under /etc/sysconfig/eos_env.example The configuration files coming from an RPM installation have useful default settings. The main changes required in the static xrootd configuration files are concerning plug-ins (e.g. add HTTP(s) plug-ins) or authentication mechanisms and their logical ordering. Individual configuration settings are picked up in topical chapters.

Classical Systemd Services

The systemctl services for the four types of daemon are:

systemctl start|stop|status xrootd@quarkdb
systemctl start|stop|status eos@mq
systemctl start|stop|status eos@mgm
systemctl start|stop|status eos@fst

The logical startup order justified by their dependencies should be:

  1. QDB
  2. MQ
  3. MGM
  4. FSTs

The MGM service requires QDB and MQ to be up. The FST Service requires MGM, QDB and MQ to be up. QDB and MQ service have no dependencies.

Start systemd Services at Boot

To start EOS services when a machine boots one needs only to enable the corresponding systemd service:

systemctl enable xrootd@quarkdb
systemctl enable eos@mq
systemctl enable eos@mgm
systemctl enable eos@fst

Classical Service Logfiles

Service Logfile Location Type
MGM /var/log/eos/mgm EOS
FST /var/log/eos/fst EOS
MQ /var/log/eos/mq EOS
QDB /var/log/xrootd/quarkdb QDB

The EOS logfiles are usually called xrdlog.(service). In the MGM directory there are sublogfiles, which filter out log lines from the main logfile:

Logfile Contents
Balancer.log : log information for the balancer service
Clients.log : log information for client calls to the MGM
Converter.log : log information of the converter service
DrainJob.log : log information for draining
eosxd-logtraces.log : traces requested using ‘eos fusex evict … sendlog’ from clients
eosxd-stacktraces.log : traces requested using ‘eos fusex evict … stracktrace’ from clients
error.log : all message with ERROR level from FSTs
FileInspector.log : log information for the file inspector service
GeoBalancer.log : log information for the GEO balancer service
GeoTreeEngine.log : log information for the GEO tree engine
GroupBalancer.log : log information for the group balancer
GroupDrainer.log : log information for the group drainer
Grpc.log : log information for the GRPC server
Http.log : log information for the HTTP(S) server
logbook.log : annotated commands stored in the logbook on user request
LRU.log : log information for the LRU service
Master.log : log information for HA master transitions
MetadataFlusher.log : log information for the metadata flusher
Mounts.log : log information for FUSE mount/umount
OAuth.log : log information for OAUTH authentication
Recycle.log : log information for the Recycle (purging) service
ReplicationTracker.log : log information for the Replication tracker service
WFE::Job.log : log information for Workflow Engine jobs
WFE.log : log information for the Workflow Engine
xrdlog.mgm : main log file with all log messages
ZMQ.log : log information for the ZMQ server

EOS5 Configuration

Configuration Files

The configuration files for the EOS5 service management are located under /etc/eos/

Configuration Sections

They are internally structured into generic sections init sysonfig:

and daemon specific sections:

The first tag inside [daemon:xrootd:name] qdb mq fst mgm references the daemon where this applies. The second tag xrootd reflects that this is actually part of the XRootD configuration file generated for the respective daemon type. The last tag is the name of the daemon instance. It is possible to run one or several of each daemon type per machine. The default name is just the daemon type itself e.g. qdb daemon default name is qdb. On cane have several instances of one type of daemon e.g. mgm1 mgm2 mgm3 fst1 fst2 fst3 aso.

### Daemon Startup If you want to see the config for a specific daemon you can type:

This spits out the three active sections for init, sysconfig and xrootd configuration:

Init Sections

The init section are shell commands which are executed on startup. The default init sections create some of the required directories and change ownership accordingly. The init section of QDB also initializes a new QDB database automatically. Commands which should be executed for all daemons you put into /etc/eos/config/generic/all. Commands to be executed for a specific daemon you put into the daemon config file e.g. /etc/eos/config/qdb/qdb.

Sysconfig Sections

The sysconfig section contains variable definitions e.g. /etc/eos/config/generic/all contains:

# ------------------------------------------------------------ #
[sysconfig]
# ------------------------------------------------------------ #

# EOSHOST is replaced by the eos CLI with the current hostname
SERVER_HOST=${EOSHOST}
INSTANCE_NAME=eosdev
GEO_TAG=local

The configuration file syntax allows, that they can work on several hosts without changing host names etc. In this example you see that when you want to reference the machine where you run this command, you just use the variable ${EOSHOST}, so that you don’t have to write myhost1.foo myhost2.foo depending on the machine name. This is also the place where you define the name of your instance.

Unshare Section

The unshare section can be used to create a private mount namespace inside the environment of any XRootD process. This is useful if you want to mount a remote filesystem for FSTs, which are only visible to the FST process mount namespace, but to nobody else on the machine itself. A df as root will not show this external mount. You just write the needed mount commmand into the init section and it will be executed on daemon startup. It is possible also to encrypt commands in the init section, in case you have to use a mount key. To get an encrypted command for init sections you use:

eos daemon seal "mount -t nfs ... /nfs/"
enc:fmAWznYjTWqRGfeiDSpfQy3MzQpJOhVI

and you would place enc:fmAWznYjTWqRGfeiDSpfQy3MzQpJOhVI into your init section.

Service Management

The systemd command set to start each single daemon manually on a node is: .. code-block:

systemctl start eos5-qdb@qdb
systemctl start eos5-mq@mq
systemctl start eos5-mgm@mgm
systemctl start eos5-fst@fst

The syntax is eos5-daemon@name e.g. start fst daemon with name fst1: systemctl start eos5-fst@fst1

To enable all daemon on startup, you do:

systemctl enable eos5-qdb@qdb
systemctl enable eos5-mq@mq
systemctl enable eos5-mgm@mgm
systemctl enable eos5-fst@fst

EOS5 Service Logfiles

Service Logfile Location Type
MGM /var/log/eos/mgm EOS
FST /var/log/eos/fst EOS
MQ /var/log/eos/mq EOS
QDB /var/log/eos/qdb QDB

The EOS logfiles are usually called xrdlog.(service). In the MGM directory there are sublogfiles, which filter out log lines from the main logfile:

Logfile Contents
Balancer.log : log information for the balancer service
Clients.log : log information for client calls to the MGM
Converter.log : log information of the converter service
DrainJob.log : log information for draining
eosxd-logtraces.log : traces requested using ‘eos fusex evict … sendlog’ from clients
eosxd-stacktraces.log : traces requested using ‘eos fusex evict … stracktrace’ from clients
error.log : all message with ERROR level from FSTs
FileInspector.log : log information for the file inspector service
GeoBalancer.log : log information for the GEO balancer service
GeoTreeEngine.log : log information for the GEO tree engine
GroupBalancer.log : log information for the group balancer
GroupDrainer.log : log information for the group drainer
Grpc.log : log information for the GRPC server
Http.log : log information for the HTTP(S) server
logbook.log : annotated commands stored in the logbook on user request
LRU.log : log information for the LRU service
Master.log : log information for HA master transitions
MetadataFlusher.log : log information for the metadata flusher
Mounts.log : log information for FUSE mount/umount
OAuth.log : log information for OAUTH authentication
Recycle.log : log information for the Recycle (purging) service
ReplicationTracker.log : log information for the Replication tracker service
WFE::Job.log : log information for Workflow Engine jobs
WFE.log : log information for the Workflow Engine
xrdlog.mgm : main log file with all log messages
ZMQ.log : log information for the ZMQ server