[[ux_xnjs]]
Configuring the XNJS
--------------------

The XNJS is the UNICORE/X component that deals with the actual job 
execution and file access. It is configured using an XML file named
'xnjs.xml' or 'xnjs_legacy.xml'. The actual file that is used is set
in the uas.config property +coreServices.targetsystemfactory.xnjs.configfile+.

-----
#
# in uas.config
#
coreServices.targetsystemfactory.xnjs.configfile=conf/xnjs.xml
-----

Here's an overview of the most important properties that can be set in this file.

.Main XNJS properties
[options="header"]
|==================
| config file | property name		| range of values		| default value			| description
|xnjs.xml     | XNJS.filespace	|an absolute path on the target system's filesystem |"data/FILESPACE"	|the directory on the target system where job directories will be created
|	      | XNJS.statedir 		|a path on the UNICORE/X machine | "data/NJSSTATE"		|the directory on the UNICORE/X machine where the XNJS keeps its state
|				|XNJS.idbfile			|a file or directory name		|"conf/simpleidb"		|the IDB containing application definitions etc.
|				|XNJS.numberofworkers	|an integer					|"4"						|the number of worker threads used to process jobs
|=============================


Most of the other settings in this file are used to configure the internals of the XNJS and should usually be left
at their default values.

The UNICORE TSI
~~~~~~~~~~~~~~~

This section describes installation and usage of the UNICORE TSI.
This is a mandatory step if you want to interface to batch systems
such as Torque, SGE, or LoadLeveller to efficiently use a compute 
cluster.

[NOTE]
===========
Without this component, all jobs will run on the UNICORE/X server, under the
user id that started UNICORE/X.
===========

In a nutshell, you have to perform the following steps

  - Install the UNICORE TSI on your cluster front end node

  - Edit the tsi.properties file

  - On the UNICORE/X server, edit uas.config, simpleidb and xnjs_legacy.xml

  - Start the newly installed TSI (as root in a multiuser setting)

  - Restart UNICORE/X


==== Installation of the correct TSI

The TSI is a set of perl modules that is running on the target system.
In case of a cluster system, you'll need to install it on the frontend machine(s),
i.e. the machine from where your jobs are submitted to the batch system.
There are different variants available for the different batch systems such as Torque or SGE.

Usually installation and start of the TSI will be performed as the root user. 
The TSI will then be able to change to the current Grid user's id for 
performing work (Note: nothing will ever be executed as "root").
You can also use a normal user, but then all commands will be executed 
under this user's id.

 - First, download and install the UNICORE TSI package. The 
UNICORE core server bundle ("quickstart" package) includes the TSI in the 'tsi' 
subdirectory. You should copy this folder to the correct machine first. 
In the following this will be denoted by <tsidir>

 - Install the correct TSI variant by 

--------------------------------
 cd <tsidir>
  ./Install.sh
--------------------------------
When prompted for the path, choose an appropriate on, denoted '<your_tsi>' in the following

 - Check the tsi file in

--------------------------------
 <tsidir>/<your_tsi>/perl/tsi
-------------------------------
especially command locations, path settings etc.

 - set permissions using

------------------------
  cd <tsidir>
   ./Install_permissions.sh
------------------------

 - MAKE A NOTE of the exact location of the 'tsi_ls' and 'tsi_df' files
'<tsidir>/<your_tsi>/tsi_ls' and '<tsidir>/<your_tsi>/tsi_df'

==== Required TSI Configuration

Configuration is done by editing <tsi_dir>/conf/tsi.properties
At least the following settings are needed:

--------------------------------
# path to your tsi installation
tsi.path=<tsi_dir>/<your_tsi>

# UNICORE/X machine
tsi.njs_machine=<UNICORE/X host>

# UNICORE/X listener port (check unicorex/conf/xnjs_legacy.xml variable "CLASSICTSI.replyport"
tsi.njs_port=7654

# TSI listener port (check unicorex/conf/xnjs_legacy.xml variable "CLASSICTSI.port"
tsi.my_port=4433
-------------------------------



==== UNICORE/X configuration

Edit 'unicorex/conf/uas.config' and set the variable

--------------------------------
coreServices.targetsystemfactory.xnjs.configfile=conf/xnjs_legacy.xml
--------------------------------

Edit 'unicorex/conf/xnjs_legacy.xml'. Check the filespace location, this is where the local 
job directories will be created. On a cluster, these have to be on a shared part of the filesystem.

Check the 'CLASSICTSI' related properties. Set the correct value for the machine
and the ports (these can usually be left at their default values)

Set the value of +CLASSICTSI.TSI_LS+ to the path of 'tsi_ls' as noted above.

Set the value of +CLASSICTSI.TSI_DF+ to the path of 'tsi_df' as noted above.

Here is an example section for the classic TSI properties.

--------------------------------
    <eng:Property name="XNJS.tsiclass" value="de.fzj.unicore.xnjs.legacy.LegacyTSI"/>
    <!-- TSI machine and ports used -->
    <eng:Property name="CLASSICTSI.machine" value="localhost"/>
    <eng:Property name="CLASSICTSI.port" value="4433"/>
    <eng:Property name="CLASSICTSI.replyport" value="7654"/>
    <!-- location of the tsi_ls file -->
    <eng:Property name="CLASSICTSI.TSI_LS" value="tsi/tsi_ls"/>
    <!-- location of the tsi_df file -->
    <eng:Property name="CLASSICTSI.TSI_DF" value="tsi/tsi_df"/>
    <!-- commands on the target system -->
    <eng:Property name="CLASSICTSI.CP" value="/bin/cp"/>
    <eng:Property name="CLASSICTSI.RM" value="/bin/rm"/>
    <eng:Property name="CLASSICTSI.MV" value="/bin/mv"/>
    <eng:Property name="CLASSICTSI.MKDIR" value="/bin/mkdir -p"/>
    <eng:Property name="CLASSICTSI.CHMOD" value="/bin/chmod"/>
    <eng:Property name="CLASSICTSI.MKFIFO" value="/usr/bin/mkfifo"/>
    <eng:Property name="CLASSICTSI.PERL" value="/usr/bin/perl"/>
    <!-- interval between updates of job stati (milliseconds) -->
    <eng:Property name="CLASSICTSI.statusupdate.interval" value="5000"/>  
    <!-- how often the XNJS will re-try to get the status of a job
         in case the job is not listed in the status listing -->
    <eng:Property name="CLASSICTSI.statusupdate.grace" value="0"/>
    <!-- a user that is allowed to see all jobs on the batch system -->
    <eng:Property name="CLASSICTSI.priveduser" value="someuser"/>
    <!-- I/O buffer size, will greatly impact filetransfer performance -->
    <eng:Property name="CLASSICTSI.BUFFERSIZE" value="1000000"/>  
--------------------------------

==== Additional parameters

Some additional parameters exist for tuning the XNJS-TSI communication.
  
.XNJS-TSI communication settings
[options="header"]
|=======================
| property name		| range of values		| default value			| description	
|CLASSICTSI.BUFFERSIZE  | integer | 1000000 | Buffersize for filetransfers in bytes
|CLASSICTSI.socket.timeout  | integer | 300000 | Socket timeout in milliseconds 
|CLASSICTSI.socket.connect.timeout  | integer | 10000 | Connection timeout in milliseconds 
|=======================

==== Tuning the batch system settings

UNICORE uses the normal batch system commands (e.g. qstat) to get the status of running jobs.
There is a special case if a job is not listed in the qstat output. UNICORE will then assume
the job is finished. However, in some cases this is not true, and UNICORE will have a wrong job status.
To work around, there is a special property 

---------
 <!-- how often the XNJS will re-try to get the status of a job
      in case the job is not listed in the status listing -->
 <eng:Property name="CLASSICTSI.statusupdate.grace" value="2"/>
---------

If the value is larger than zero, UNICORE will re-try to get the job status. 
  

Start the TSI using (as root in a multiuser environment)

--------------------------------
cd <tsi_dir>/conf
 ../bin/start_tsi
--------------------------------

(or use the +unicore-tsi+ init script if available in your installation)

Finally, restart the UNICORE/X server

[NOTE]
============
When changing TSIs, it's a good idea to remove the UNICORE/X state and any
files before restarting. See xref:ux_persistence for details
=============

==== Enabling SSL for the XNJS to TSI communication

The UNICORE/X server can be set up to use SSL for communicating with the
Perl TSI. On the UNICORE/X side, this is very simple to switch on. In the XNJS config file,
set the following property to 'true' (by default it is set to false):

---------
    <!-- enable SSL using the normal UNICORE/X key and trusted certificates -->
    <eng:Property name="CLASSICTSI.ssl" value="true"/>
---------

On the TSI side it is a bit more complex, and you need to have the TSI from the 6.3.0 
distribution or later installed. First of all, your Perl installation must include the 
module "IO::Socket:SSL" and its dependencies. If you do not have it, you can get it from the 
CPAN archive.

In the 'tsi.properties' configuration file, you set the keystore and truststore to be used:
  
-------------
# SSL parameters
# Keystore must contain the private TSI key and certificate
# Trustore must contain the certificate of the CA
tsi.keystore=/certs/keystore.pem
tsi.keypass=yourpassword
tsi.truststore=/certs/keystore.pem
---------------

Both keystore and truststore are in pem format.


include::xnjs-accounting.txt[]

The IDB
-------

include::xnjs-idb.txt[]

include::xnjs-idb-applications.txt[]

include::xnjs-idb-executionenvironment.txt[]

include::xnjs-tsi-custom.txt[]

include::xnjs-dynamicincarnation.txt[]

include::xnjs-dynamicincarnation-context.txt[]



   
