III - Configuration guide

Now that we've installed all the required packages, let's configure and bring up our CTA instance step by step.

This guide is an approach to an easy, basic installation. However, note that:

• It is assumed that you've created an oracle or postgresql database to use as the Catalogue DB.

• It is assumed that you've created a VFS or NFS to create the ObjectStore later (this guide does not cover Ceph configuration steps).

• It is assumed that you have got a Kerberos server and that you're able to create a keytab.

1. Initialise the ObjectStore

First step is to initialize the space that our tape system will be using to store (shared) objects between our servers, as its name implies.

It is recommended to use Ceph in a production system, however it is not mandatory. This guide will cover the initialisation of a Network File System (NFS) as the ObjectStore.

First, check that you've mounted your NFS and installed the cta-objectstore-tools package:

cta-versionlock checkpkg | grep objectstore

Create the configuration file at /etc/cta/cta-objectstore-tools.conf and include the following line:

ObjectStore BackendPath /path/to/objectstore

Execute the following commands:

export OBJECTSTORETYPE=file
cta-objectstore-initialize /path/to/objectstore
chmod -R 0777 /path/to/objectstore

Let's use cta-objectstore-list. You should see the AgentRegister, DriveRegister and SchedulerGlobalLock processes for the moment. You're done.

cta-objectstore-list /path/to/objectstore 
Object store path: file:///path/to/objectstore
AgentRegister-cta-objectstore-initialize-<hostname>-21417-20230417-15:28:05-0-0
DriveRegister-cta-objectstore-initialize-<hostname>-21417-20230417-15:28:05-0-1
SchedulerGlobalLock-cta-objectstore-initialize-ctamon01.pic.es-21417-20230417-15:28:05-0-2
root

Note: do not manipulate or remove any of the processes or objects under any circumstances, except you know very well what you're doing!

2. Create the Catalogue

You can either use Oracle or PostgreSQL as your database to contain your Catalogue. Steps are the same.

Assuming that you've already created a database, let's write down the connection string at /etc/cta/catalogue.conf file. Use the correct format of the ones below according to your DB system:

oracle:username/password@database
# or
postgresql:postgresql://username:password@hostname/database

Make sure you have the cta-catalogueutils package installed. Now, let's use this connection string to create our database schema:

cta-catalogue-schema-create /etc/cta/cta-catalogue.conf

Check if the schema has been created correctly:

cta-catalogue-schema-verify /etc/cta/cta-catalogue.conf

There is a command for dropping the schema. Use this at your own risk!

cta-catalogue-schema-drop /etc/cta/cta-catalogue.conf

3. Create an admin user

Once you've created the schema for the catalogue, it's time to create now an admin user that will be able to query the catalogue's data through cta-admin commands.

Important: the name given to the admin user has to match the name given to the kerberos user.

cta-catalogue-admin-user-create /etc/cta/cta-catalogue.conf -u "username" -m "Comment to describe the creation of the admin user"

You can have multiple admin users, as long as they have their own kerberos keytab to generate valid tickets.

4. Set up the authentication

Before starting, verify that you have an user named 'cta' . It should be created when installing the packages. If not, create it. Then, you also have to create the group 'tape'. This user and group will be the owners of some of our keytabs:

useradd cta
groupadd tape

Let's start by configuring the Kerberos authentication. Make sure you have the krb5-workstation package installed. Then, modify your /etc/krb5.conf file and replace the information to point at your kerberos server.

As mentioned before, you'll also need to generate a kerberos keytab for your admin user to be able to execute admin commands from the krb server.

Simply copy your keytab to the /etc/cta directory and change the owner and group:

chown  cta:tape /etc/cta/cta-frontend.krb5.keytab

Then, generate a valid ticket with the following command:

kinit -k <username>@<realm> -t /etc/cta/cta-frontend.krb5.keytab

To check if your ticket has generated correctly, execute:

klist

Tickets are valid for 24 hours by default. It is up to you to change these settings on your krb server or create a cron executing the kinit command every day.

You're done with the Kerberos part of the authentication. Now, let's create the Simple Shared Secret (SSS) keytabs. These are also required for the frontend to work.

First, check that you have installed the xrootd-server package. Then, execute:

# This creates the /etc/cta/cta-frontend.sss.keytab
xrdsssadmin -k cta_eosdev -u eosdev -g cta add /etc/cta/cta-frontend.sss.keytab
chown cta  /etc/cta/cta-frontend.sss.keytab
chmod 600 /etc/cta/cta-frontend.sss.keytab

# This creates the /etc/eos.keytab
xrdsssadmin -k cta-taped -u cta -g cta add /etc/eos.keytab
xrdsssadmin -k eosdev -u daemon -g daemon add /etc/eos.keytab
chown daemon:daemon /etc/eos.keytab

# This creates the /etc/cta/cta-taped.sss.keytab
cp /etc/eos.keytab /etc/cta-taped.sss.keytab
xrdsssadmin -k eosdev del /etc/cta-taped.sss.keytab
chown cta:tape /etc/cta-taped.sss.keytab

5. Configure the CTA frontend

You've finished creating all these keytabs. Let's bring our first service up, which will be the cta-frontend daemon.

First of all, take a look at the contents of the frontend configuration file. It is located at /etc/cta/cta-frontend-xrootd.conf. For now, you can leave almost every parameter at its default value, except these ones:

cta.objectstore.backendpath /path/to/objectstore
sec.protocol krb5 /etc/cta/cta-frontend.krb5.keytab <username>@<realm>
sec.protocol sss -s /etc/cta/cta-frontend.sss.keytab
xrd.port 10955

Now, edit the /etc/sysconfig/cta-taped file and make sure it is pointing at the correct SSS keytab:

XrdSecSSSKT=/etc/cta/cta-taped.sss.keytab

Modify the cta.endpoint at file /etc/cta/cta-cli.conf. You can leave all other values at default. You should specify the machine where you are installing the frontend.

cta.endpoint ctafrontend.cern.ch:10955

Create the necessary logfiles:

touch /var/log/cta/cta-frontend.log
touch /var/log/cta/cta-frontend-xrootd.log
chmod a+w /var/log/cta/cta-frontend*

Start the daemon and check its status:

systemctl start cta-frontend
systemctl status cta-frontend

Now you should be able to do your first cta-admin command:

cta-admin version

If it does not work, try with the debug option or check the troubleshooting section.

XrdSecDEBUG=1 cta-admin version

Now, let's start the other frontend daemon, cta-frontend-grpc. This one is in charge of receiving the requests from dCache and transmitting them to the other CTA components.

Before starting the service, check the /etc/systemd/system/cta-frontend-grpc.service file. The user and tape specified must be the following:

User=cta
Group=tape

Save the changes:

systemctl daemon-reload

Create the configuration file for the daemon at /etc/cta/cta.conf. This file must include the following parameters:

taped BufferCount 50000
ObjectStore BackendPath /path/to/objectstore

The cta-frontend-grpc logs directly at /var/log/messages. However, you can redirect the logs with simple this rsyslog rule (must be in /etc/rsyslog.d directory):

if ($programname == "cta-frontend-grpc") then /var/log/cta/cta-frontend-grpc.log

Create the logfile and restart rsyslog:

touch /var/log/cta/cta-frontend-grpc.log
systemctl restart rsyslog

Now, let's start the daemon:

systemctl start cta-frontend-grpc
systemctl status cta-frontend-grpc

You're done setting up the whole frontend! Let's continue with the tapeserver part.

6. Creating the udev rules

In this step we are using an udev rule to create a symbolic link to the SCSI media changer device. Create the file /etc/udev/rules.d/00-cta.rules and add the following content:

# Create symlink /dev/smc pointing to the SCSI media changer
SUBSYSTEM=="scsi_generic", KERNEL=="sg*", ATTRS{type}=="8", SYMLINK:="smc"
# Set the permissions and group of the tape devices
KERNEL=="nst*", MODE:="0666"
KERNEL=="st*",  MODE:="0666"
KERNEL=="sg*",  MODE:="0666"

Once done, reboot your machine or apply the rules with the following command:

udevadm trigger

7. Configuring the cta-rmcd daemon

This service should run on the tapeserver where your drive(s) are connected to.

First, check that you have installed the cta-smc and cta-rmcd packages. Then, make sure to have this line at /etc/sysconfig/cta-rmcd file:

CTA_RMCD_OPTIONS=/dev/smc

Create the logfile and give permissions:

touch /var/log/cta/cta-rmcd.log
chmod a+w /var/log/cta/cta-rmcd.log

Start the service:

systemctl start cta-rmcd
systemctl status cta-rmcd

You can now use cta-smc commands as a replacement of mtx commands. It is useful to quickly check some information and do manual tape operations.

Here are a few examples of what you can do, but check the manpage of cta-smc to see the complete list of options.

# List the drives in the library, their status and the mounted tape (if any):
cta-smc -q D

# List general information about the physical library:
cta-smc -q L

# List the status of the library's first five slots:
cta-smc -q S -N 5

# List the status of the first five tapes:
cta-smc -q V -N 5

# Mount the tape V03647 on drive 0:
cta-smc -m -D 0 -V V03647

# Unmount the tape V03647 from drive 0:
cta-smc -d -D 0 -V V03647

8. Configuring the cta-taped daemon

First, let's check the configuration file, /etc/cta/cta-taped.conf. For now, you can leave all the default values except for these lines, which you'll have to uncomment:

ObjectStore BackendPath /path/to/objectstore
taped TpConfigPath /etc/cta/TPCONFIG
taped CatalogueConfigFile /etc/cta/cta-catalogue.conf
taped BufferCount 5000
taped TapeLoadTimeout 180
taped UseEncryption no

Now, go to /etc/sysconfig/cta-taped. Specify the logfile and the correct sss keytab:

CTA_TAPED_OPTIONS="--log-to-file=/var/log/cta/cta-taped.log"
XrdSecPROTOCOL=sss
XrdSecSSSKT=/etc/cta/cta-taped.sss.keytab

We have to register a drive manually. Add a line at /etc/cta/TPCONFIG. Take a look at the columns.

• DRIVE_NAME : the identifier that will be displayed when listing the drives with cta-admin, so think of a good nomenclature to identify them easily.

• LOGICAL_LIBRARY_NAME: the name you want your logical library to have, and under which the drive will be operating.

• DRIVE_DEVICE: Corresponds to the No-rewind SCSI Device (/dev/nstX). If we only have one drive on the tape server, it will always be nst0.

• LIBRARY_SLOT: Corresponds to the SCSI Media Changer Device. It is always smcX, where X corresponds to the drive ordinal assigned. Look up the drive ordinal with "cta-smc -q D"

An example of a TPCONFIG file with one drive would be:

# DRIVE_NAME LOGICAL_LIBRARY_NAME DRIVE_DEVICE LIBRARY_SLOT
IBML9541       cta                 /dev/nst0    smc0

Create the logfile and give permissions:

touch /var/log/cta/cta-taped.log
chmod a+w /var/log/cta/cta-taped.log

Finally, start the cta-taped service:

systemctl start cta-taped
systemctl status cta-taped

You're done configuring all of the necessary files and services of CTA, and have them all running. With all this done, we can jump to filling our logical library metadata with cta-admin commands. (See: Chapter III)