III - Configuration guide
Now that we've installed all the required packages, let's configure and bring up our CTA instance step by step.
This guide is an approach to an easy, basic installation. However, note that:
It is assumed that you've created an oracle or postgresql database to use as the Catalogue DB.
It is assumed that you've created a VFS or NFS to create the ObjectStore later (this guide does not cover Ceph configuration steps).
It is assumed that you have got a Kerberos server and that you're able to create a keytab.
1. Initialise the ObjectStore
First step is to initialize the space that our tape system will be using to store (shared) objects between our servers, as its name implies.
It is recommended to use Ceph in a production system, however it is not mandatory. This guide will cover the initialisation of a Network File System (NFS) as the ObjectStore.
First, check that you've mounted your NFS and installed the cta-objectstore-tools
package:
cta-versionlock checkpkg | grep objectstore
Create the configuration file at /etc/cta/cta-objectstore-tools.conf
and include the following line:
ObjectStore BackendPath /path/to/objectstore
Execute the following commands:
export OBJECTSTORETYPE=file
cta-objectstore-initialize /path/to/objectstore
chmod -R 0777 /path/to/objectstore
Let's use cta-objectstore-list
. You should see the AgentRegister, DriveRegister and SchedulerGlobalLock processes for the moment. You're done.
cta-objectstore-list /path/to/objectstore
Object store path: file:///path/to/objectstore
AgentRegister-cta-objectstore-initialize-<hostname>-21417-20230417-15:28:05-0-0
DriveRegister-cta-objectstore-initialize-<hostname>-21417-20230417-15:28:05-0-1
SchedulerGlobalLock-cta-objectstore-initialize-ctamon01.pic.es-21417-20230417-15:28:05-0-2
root
Note: do not manipulate or remove any of the processes or objects under any circumstances, except you know very well what you're doing!
2. Create the Catalogue
You can either use Oracle or PostgreSQL as your database to contain your Catalogue. Steps are the same.
Assuming that you've already created a database, let's write down the connection string at /etc/cta/catalogue.conf file. Use the correct format of the ones below according to your DB system:
oracle:username/password@database
# or
postgresql:postgresql://username:password@hostname/database
Make sure you have the cta-catalogueutils
package installed. Now, let's use this connection string to create our database schema:
cta-catalogue-schema-create /etc/cta/cta-catalogue.conf
Check if the schema has been created correctly:
cta-catalogue-schema-verify /etc/cta/cta-catalogue.conf
There is a command for dropping the schema. Use this at your own risk!
cta-catalogue-schema-drop /etc/cta/cta-catalogue.conf
3. Create an admin user
Once you've created the schema for the catalogue, it's time to create now an admin user that will be able to query the catalogue's data through cta-admin commands.
Important: the name given to the admin user has to match the name given to the kerberos user.
cta-catalogue-admin-user-create /etc/cta/cta-catalogue.conf -u "username" -m "Comment to describe the creation of the admin user"
You can have multiple admin users, as long as they have their own kerberos keytab to generate valid tickets.
4. Set up the authentication
Before starting, verify that you have an user named 'cta' . It should be created when installing the packages. If not, create it. Then, you also have to create the group 'tape'. This user and group will be the owners of some of our keytabs:
useradd cta
groupadd tape
Let's start by configuring the Kerberos authentication. Make sure you have the krb5-workstation
package installed. Then, modify your /etc/krb5.conf
file and replace the information to point at your kerberos server.
As mentioned before, you'll also need to generate a kerberos keytab for your admin user to be able to execute admin commands from the krb server.
Simply copy your keytab to the /etc/cta
directory and change the owner and group:
chown cta:tape /etc/cta/cta-frontend.krb5.keytab
Then, generate a valid ticket with the following command:
kinit -k <username>@<realm> -t /etc/cta/cta-frontend.krb5.keytab
To check if your ticket has generated correctly, execute:
klist
Tickets are valid for 24 hours by default. It is up to you to change these settings on your krb server or create a cron executing the kinit command every day.
You're done with the Kerberos part of the authentication. Now, let's create the Simple Shared Secret (SSS) keytabs. These are also required for the frontend to work.
First, check that you have installed the xrootd-server
package. Then, execute:
# This creates the /etc/cta/cta-frontend.sss.keytab
xrdsssadmin -k cta_eosdev -u eosdev -g cta add /etc/cta/cta-frontend.sss.keytab
chown cta /etc/cta/cta-frontend.sss.keytab
chmod 600 /etc/cta/cta-frontend.sss.keytab
# This creates the /etc/eos.keytab
xrdsssadmin -k cta-taped -u cta -g cta add /etc/eos.keytab
xrdsssadmin -k eosdev -u daemon -g daemon add /etc/eos.keytab
chown daemon:daemon /etc/eos.keytab
# This creates the /etc/cta/cta-taped.sss.keytab
cp /etc/eos.keytab /etc/cta-taped.sss.keytab
xrdsssadmin -k eosdev del /etc/cta-taped.sss.keytab
chown cta:tape /etc/cta-taped.sss.keytab
5. Configure the CTA frontend
You've finished creating all these keytabs. Let's bring our first service up, which will be the cta-frontend daemon.
First of all, take a look at the contents of the frontend configuration file. It is located at /etc/cta/cta-frontend-xrootd.conf
. For now, you can leave almost every parameter at its default value, except these ones:
cta.objectstore.backendpath /path/to/objectstore
sec.protocol krb5 /etc/cta/cta-frontend.krb5.keytab <username>@<realm>
sec.protocol sss -s /etc/cta/cta-frontend.sss.keytab
xrd.port 10955
Now, edit the /etc/sysconfig/cta-taped
file and make sure it is pointing at the correct SSS keytab:
XrdSecSSSKT=/etc/cta/cta-taped.sss.keytab
Modify the cta.endpoint at file /etc/cta/cta-cli.conf
. You can leave all other values at default. You should specify the machine where you are installing the frontend.
cta.endpoint ctafrontend.cern.ch:10955
Create the necessary logfiles:
touch /var/log/cta/cta-frontend.log
touch /var/log/cta/cta-frontend-xrootd.log
chmod a+w /var/log/cta/cta-frontend*
Start the daemon and check its status:
systemctl start cta-frontend
systemctl status cta-frontend
Now you should be able to do your first cta-admin command:
cta-admin version
If it does not work, try with the debug option or check the troubleshooting section.
XrdSecDEBUG=1 cta-admin version
Now, let's start the other frontend daemon, cta-frontend-grpc. This one is in charge of receiving the requests from dCache and transmitting them to the other CTA components.
Before starting the service, check the /etc/systemd/system/cta-frontend-grpc.service
file. The user and tape specified must be the following:
User=cta
Group=tape
Save the changes:
systemctl daemon-reload
Create the configuration file for the daemon at /etc/cta/cta.conf
. This file must include the following parameters:
taped BufferCount 50000
ObjectStore BackendPath /path/to/objectstore
The cta-frontend-grpc logs directly at /var/log/messages
. However, you can redirect the logs with simple this rsyslog rule (must be in /etc/rsyslog.d
directory):
if ($programname == "cta-frontend-grpc") then /var/log/cta/cta-frontend-grpc.log
Create the logfile and restart rsyslog:
touch /var/log/cta/cta-frontend-grpc.log
systemctl restart rsyslog
Now, let's start the daemon:
systemctl start cta-frontend-grpc
systemctl status cta-frontend-grpc
You're done setting up the whole frontend! Let's continue with the tapeserver part.
6. Creating the udev rules
In this step we are using an udev rule to create a symbolic link to the SCSI media changer device.
Create the file /etc/udev/rules.d/00-cta.rules
and add the following content:
# Create symlink /dev/smc pointing to the SCSI media changer
SUBSYSTEM=="scsi_generic", KERNEL=="sg*", ATTRS{type}=="8", SYMLINK:="smc"
# Set the permissions and group of the tape devices
KERNEL=="nst*", MODE:="0666"
KERNEL=="st*", MODE:="0666"
KERNEL=="sg*", MODE:="0666"
Once done, reboot your machine or apply the rules with the following command:
udevadm trigger
7. Configuring the cta-rmcd daemon
This service should run on the tapeserver where your drive(s) are connected to.
First, check that you have installed the cta-smc
and cta-rmcd
packages. Then, make sure to have this line at /etc/sysconfig/cta-rmcd
file:
CTA_RMCD_OPTIONS=/dev/smc
Create the logfile and give permissions:
touch /var/log/cta/cta-rmcd.log
chmod a+w /var/log/cta/cta-rmcd.log
Start the service:
systemctl start cta-rmcd
systemctl status cta-rmcd
You can now use cta-smc commands as a replacement of mtx commands. It is useful to quickly check some information and do manual tape operations.
Here are a few examples of what you can do, but check the manpage of cta-smc to see the complete list of options.
# List the drives in the library, their status and the mounted tape (if any):
cta-smc -q D
# List general information about the physical library:
cta-smc -q L
# List the status of the library's first five slots:
cta-smc -q S -N 5
# List the status of the first five tapes:
cta-smc -q V -N 5
# Mount the tape V03647 on drive 0:
cta-smc -m -D 0 -V V03647
# Unmount the tape V03647 from drive 0:
cta-smc -d -D 0 -V V03647
8. Configuring the cta-taped daemon
First, let's check the configuration file, /etc/cta/cta-taped.conf
. For now, you can leave all the default values except for these lines, which you'll have to uncomment:
ObjectStore BackendPath /path/to/objectstore
taped TpConfigPath /etc/cta/TPCONFIG
taped CatalogueConfigFile /etc/cta/cta-catalogue.conf
taped BufferCount 5000
taped TapeLoadTimeout 180
taped UseEncryption no
Now, go to /etc/sysconfig/cta-taped
. Specify the logfile and the correct sss keytab:
CTA_TAPED_OPTIONS="--log-to-file=/var/log/cta/cta-taped.log"
XrdSecPROTOCOL=sss
XrdSecSSSKT=/etc/cta/cta-taped.sss.keytab
We have to register a drive manually. Add a line at /etc/cta/TPCONFIG
. Take a look at the columns.
DRIVE_NAME : the identifier that will be displayed when listing the drives with cta-admin, so think of a good nomenclature to identify them easily.
LOGICAL_LIBRARY_NAME: the name you want your logical library to have, and under which the drive will be operating.
DRIVE_DEVICE: Corresponds to the No-rewind SCSI Device (/dev/nstX). If we only have one drive on the tape server, it will always be nst0.
LIBRARY_SLOT: Corresponds to the SCSI Media Changer Device. It is always smcX, where X corresponds to the drive ordinal assigned. Look up the drive ordinal with "cta-smc -q D"
An example of a TPCONFIG file with one drive would be:
# DRIVE_NAME LOGICAL_LIBRARY_NAME DRIVE_DEVICE LIBRARY_SLOT
IBML9541 cta /dev/nst0 smc0
Create the logfile and give permissions:
touch /var/log/cta/cta-taped.log
chmod a+w /var/log/cta/cta-taped.log
Finally, start the cta-taped service:
systemctl start cta-taped
systemctl status cta-taped
You're done configuring all of the necessary files and services of CTA, and have them all running. With all this done, we can jump to filling our logical library metadata with cta-admin commands. (See: Chapter III)
Last updated