installation-and-configuration-guide.md

Installation and Administrator Guide of the openBIS Server
==========================================================

## System Requirements

The minimal requirements of a system running openBIS are:

- Operating System: Linux / MacOS X (The binaries: `bash`, `awk`, `sed`, `unzip` need to be installed and in the `PATH` of the openBIS user.)
- Java Runtime Environment: recent versions of Oracle JRE 11
- PostgreSQL 11

We find Linux to be the easiest choice to get an openBIS server running quickly.

For recommended memory settings, see [Recommended CPU and memory settings for openBIS 20.10](https://openbis.readthedocs.io/en/latest/system-admin-documentation/installation/system-requirements.html).

An SMTP server needs to be accessible and configured if you want to obtain notifications.

## Installation

The server distribution is a `gzipped` `tar` file named `openBIS-installation-standard-technologies-<version>` `.tar.gz`. It contains:

- `console.properties:` configuration file for a console/non-gui installation

- `extract.sh:` helper script for installation

- `jul.config:` Log configuration for the openBIS install process

- `openBIS-installer.jar` Java archive containing openBIS

- `run-console.sh` Installation script for console/non-gui installation

- `run-ui.sh` Installation script for gui installation

### Installation steps

1. Create a service user account, i.e. an unprivileged, regular user account. **Do not run openBIS as root!**

2. Gunzip the distribution on the server machine into some temporary folder.

3. Run either the console/non-gui installation script or the gui installation script:

    **GUI-based installation**

    ```bash
    tar xvfz openBIS-installation-standard-technologies-S139.0-r26480.tar.gz
    cd openBIS-installation-standard-technologies-S139.0-r26480
    ./run-ui.sh
    ```

    In the non-gui version you have to edit the `console.properties`
    files:

    **Non-GUI installation**

    ```bash
    tar xvfz openBIS-installation-standard-technologies-S139.0-r26480.tar.gz
    cd openBIS-installation-standard-technologies-S139.0-r26480
    vi console.properties
    ./run-console.sh
    ```
    
    ```{note}
    Please be aware that the directory where openbis is installed should not already exist. Users should specify the directory where they want to install openBIS (in the console.properties or in the graphical installer) and this directory will be created by the installation procedure. If the directory already exists, the installation will fail.
    ```

After the successful installation you should have a look at the configuration file called s`ervice.properties`. It is located in `<server_folder>openBIS-server/jetty/etc/`

This file is a an [Extended Properties File](https://openbis.readthedocs.io/en/latest/system-admin-documentation/installation/optional-application-server-configuration.html). Here is an example which can be used as a template:

**service.properties**

```bash
# ---------------------------------------------------------------------------
# Database configuration
# ---------------------------------------------------------------------------
# The database instance local unique identifier. Used when the new database is created.
database-instance = DEFAULT

# Supported: currently only 'postgresql' is supported
database.engine = postgresql
database.url-host-part =
database.kind = prod
# User who owns the database. Default: Operating system user running the server.
database.owner =
database.owner-password =
# Superuser of the database. Default: database-dependent.
database.admin-user = 
database.admin-password =
# Max. number of active database connections. Default: 20. 
database.max-active-connections =
# Max. number of idle database connections to keep open. Default: 20. 
database.max-idle-connections = 
# Log interval (in seconds) between two regular log entries of the number of active database 
# connections. Default: 3600s.
database.active-connections-log-interval = 

# ---------------------------------------------------------------------------
# Master data by Excel sheets
# ---------------------------------------------------------------------------
# Path to the file which stores version information of master data imported from Excel sheets. 
# Default value: ../../../xls-import-version-info.json. The file will be created. 
# It should be <openbis installation path>/servers/openBIS-server. 
# Note, that the folder containing this file has to exist. 
# xls-import.version-data-file = ../../../xls-import-version-info.json

# ---------------------------------------------------------------------------
# Authentication configuration
# ---------------------------------------------------------------------------
# Supported Authentication options are:
# 'file-authentication-service'
# 'ldap-authentication-service'
# 'crowd-authentication-service'
# 'file-crowd-authentication-service'
# 'file-ldap-authentication-service'
# 'ldap-crowd-authentication-service'
# 'file-ldap-crowd-caching-authentication-service'
# For a detailed description, have a look at the Installation and Administrator
# Guide of the openBIS Server: https://wiki-bsse.ethz.ch/x/oYIUBQ 
authentication-service = file-ldap-crowd-caching-authentication-service

# ---------------------------------------------------------------------------
# Caching configuration (only used with 'file-ldap-crowd-caching-authentication-service')
# ---------------------------------------------------------------------------
# The time that the authentication cache keeps entries. Default: 28h
authentication.cache.time = 28h
# The time that the authentication cache does not perform re-validation on a cache entry. 
# Default: 1h
authentication.cache.time-no-revalidation = 1h

# ---------------------------------------------------------------------------
# Crowd configuration
# ---------------------------------------------------------------------------
#
# The Crowd host.
# Mandatory.
crowd.service.host = 
# The Crowd service port. Default: 443
crowd.service.port =
# The timeout (in s) to wait for a Crowd query to return, -1 for "wait indefinitely". Default: 10s. 
crowd.service.timeout =
# The Crowd application name. The value 'openbis' is just a suggestion.
# Mandatory. 
crowd.application.name = openbis
# The Crowd application password. 
# Mandatory.
crowd.application.password =

# ---------------------------------------------------------------------------
# LDAP configuration
# ---------------------------------------------------------------------------
# The space-separated URLs of the LDAP servers, e.g. "ldap://d.ethz.ch/DC=d,DC=ethz,DC=ch". 
# Mandatory.
ldap.server.url = 
# The distinguished name of the security principal, e.g. "CN=carl,OU=EthUsers,DC=d,DC=ethz,DC=ch".
# Mandatory.
ldap.security.principal.distinguished.name = 
# Password of the LDAP user account that will be used to login to the LDAP server to perform the queries. 
# Mandatory.
ldap.security.principal.password = 
# The security protocol to use, use "ssl" or "none", default is "ssl"
ldap.security.protocol =
# The authentication method to use: "none" (no authentication), "simple", "strong" (SASL), defaults to "simple"
ldap.security.authentication-method =
# The referral mode:
# "follow" - follow referrals automatically (the default)
# "ignore" - ignore referrals
# "throw" - throw ReferralException when a referral is encountered
ldap.referral =
# The attribute name for the user id, defaults to "uid"
ldap.attributenames.user.id =
# The attribute name for the email, defaults to "mail"
ldap.attributenames.email =
# The attribute name for the first name, defaults to "givenName"
ldap.attributenames.first.name =
# The attribute name for the last name, defaults to "sn"
ldap.attributenames.last.name =
# Set to true to also query for email aliases
ldap.queryEmailForAliases = true
# The query template, needs to contain %s which will be filled with the query term, e.g. uid=username
# The default is:
# ldap.queryTemplate = (&(objectClass=organizationalPerson)(objectCategory=person)(objectClass=user)(%s))
# which is known to work for many Active Directory installations.
# For OpenLDAP, replace by: 
# ldap.queryTemplate = (&(%s))
# For restriction to BSSE accounts in OpenLDAP, set to: 
# ldap.queryTemplate = (&(objectClass=bssePosixAccount)(%s))
ldap.queryTemplate =
# The number of times a failed LDAP query is retried at the max. Default: 1.
ldap.maxRetries = 
# The timeout (in s) to wait for an LDAP query to return, -1 for "wait indefinitely". Default: 10s. 
ldap.timeout = 
# The time (in s) to wait after a failure before retrying the query. Default: 10s. 
ldap.timeToWaitAfterFailure =

# ---------------------------------------------------------------------------
# Anonymous login configuration (optional)
# ---------------------------------------------------------------------------
# Login of the existing user whose settings will be used for anonymous login 
#user-for-anonymous-login = <user-login>

# ---------------------------------------------------------------------------
# Project authorization
# ---------------------------------------------------------------------------
# Enabled if set to 'true'. Default: disabled 
authorization.project-level.enabled = true
# Regular expression for user ids allowed to have a project role 
authorization.project-level.users = .*

# ---------------------------------------------------------------------------
# Project samples
# ---------------------------------------------------------------------------
# Enabled if set to 'true'. Default: disabled 
# Note: Changing to 'true' turns experiment samples to project samples 
# which can not be reverted after setting this flag back to 'false'. Also
# the sample identifier will change for such samples.
#project-samples-enabled = true

# ---------------------------------------------------------------------------
# Client configuration
# ---------------------------------------------------------------------------
# Name of the file that stores Web Client configuration
web-client-configuration-file = etc/web-client.properties

# A comma-separated list of trusted cross-origin domains, that are allowed to
# query openBIS content. Typically these are lightweight webapps that integrate with openBIS 
# via JSON-RPC services, but are not directly hosted within the openBIS application.
# 
# Example 1 (two different domains configured):
# 
# trusted-cross-origin-domains=https://myapp.domain.com:8443, http://other.domain.com
#
# Example 2 (match every domain):
#
# trusted-cross-origin-domains= *
#
# The '*' matches any arbitrary domain. It should be used with care as it opens openBIS 
# for potential cross-site scripting attacks.
#
#trusted-cross-origin-domains=

# ---------------------------------------------------------------------------
# Session configuration
# ---------------------------------------------------------------------------
# The time after which an inactive session is expired by the service (in minutes).
session-timeout = 720

# Session time (in minutes) in case of presents of file etc/nologin.html. Should be < 30. 
#session-timeout-no-login = 10

# Maximum number of sessions allowed per user. Zero means unlimited number of sessions. Default value is 1.
# max-number-of-sessions-per-user = 1

# Comma separated list of users allowed to have unlimited number of sessions. Default: Empty list.
# Note: The DSS (user 'etlserver' by default, see property 'username' of DSS service.properties)
# should be added to this list.
# users-with-unrestricted-number-of-sessions = 


# ---------------------------------------------------------------------------
# Business rules configuration
# ---------------------------------------------------------------------------
# When set to "true" enables the system to store material codes containing non-alphanumeric characters.
# Regardless of the value of this property no white spaces are allowed in the material codes.
#material-relax-code-constraints=false

# Comma-separated list of regular expression of data set types which do not require that the data set 
# is linked to an experiment. If not linked to an experiment a link to a sample with space is required.
data-set-types-with-no-experiment-needed = .*

# When set to 'true' the sequence of sample codes is gap less for each type if all samples are created by
# batch registrations. 
#create-continuous-sample-codes = false


# ---------------------------------------------------------------------------
# RPC Dropbox Default DSS configuration 
# ---------------------------------------------------------------------------
# Set this to the DSS code of the DSS handling RPC Dropboxes for this user.
# Note: This is only required if more than one DSS is connected to this openBIS server.
dss-rpc.put.dss-code =

# ---------------------------------------------------------------------------
# Hibernate Search
# ---------------------------------------------------------------------------
# The working directory.
hibernate.search.index-base = ./indices
# One of NO_INDEX, SKIP_IF_MARKER_FOUND, INDEX_FROM_SCRATCH.
# If not specified, default (SKIP_IF_MARKER_FOUND) is taken.
hibernate.search.index-mode = SKIP_IF_MARKER_FOUND
# Defines the maximum number of elements indexed before flushing the transaction-bound queue.
# Default is 1000.
hibernate.search.batch-size = 1000
# Maximum number of search results
hibernate.search.maxResults = 100000
# If 'async', the update of indices will be done in a separate thread.
hibernate.search.worker.execution=async
# How long fulltext searches can take (in seconds) before they are timed out. 
# When not defined, there is no timeout.
# fulltext-timeout = 30

# ---------------------------------------------------------------------------
# Online Help
# ---------------------------------------------------------------------------
# Online help is broken into two sections -- generic and specific. Generic help links back to
# the CISD. Specific help is provided by the host of the installation
#
# OpenBIS needs to know the root URL for the online help and a template for the individual pages.
# The template should have on parameter, called title, and should be constructed to automatically
# create the page if it does not already exist.
# The template can be created by going to the root page, adding a new link to the page, and
# replacing the title of the new page with the ${title}
onlinehelp.generic.root-url = https://wiki-bsse.ethz.ch/display/CISDDoc/OnlineHelp
onlinehelp.generic.page-template = https://wiki-bsse.ethz.ch/pages/createpage.action?spaceKey=CISDDoc&title=${title}&linkCreation=true&fromPageId=40633829
#onlinehelp.specific.root-url = https://wiki-bsse.ethz.ch/display/CISDDoc/OnlineHelp
#onlinehelp.specific.page-template = https://wiki-bsse.ethz.ch/pages/createpage.action?spaceKey=CISDDoc&title=${title}&linkCreation=true&fromPageId=40633829

# ---------------------------------------------------------------------------
# JMX memory monitor
# ---------------------------------------------------------------------------
# Interval between two runs of the memory monitor (in seconds). 
# Set to -1 to disable the memory monitor.
memorymonitor-monitoring-interval = 60
# Interval between two regular log call of the memory monitor (in seconds).
# Set to -1 to disable regular memory usage logging. 
memorymonitor-log-interval = 3600
# The percentage of memory that, if exceeded, triggers a notify log of the memory manager, 
# Set to 100 to disable.
memorymonitor-high-watermark-percent = 90

# ---------------------------------------------------------------------------
# Database Configurations for Query module (optional)
# ---------------------------------------------------------------------------
# Comma separated keys of databases configured for Query module.
# Each database should have configuration properties prefixed with its key.
# Mandatory properties for each <database> include: 
#   <database>.label                - name shown to the openBIS user when adding or editing a customized query
#       <database>.database-driver      - JDBC Driver of the database (e.g. org.postgresql.Driver)
#   <database>.database-url           - JDBC URL to the database (e.g. jdbc:postgresql://localhost/openbis)
# Optional properties for each <database> include:
#   <database>.database-user        - name of the database user (default: user.name from system properties)
#   <database>.database-password    - password of the database user
#   <database>.creator-minimal-role - minimal role required to create/edit queries on this database (default: POWER_USER)
#   <database>.data-space           - If NOT specified OBSERVER of any space will be allowed to perform 
#                                     queries and <creator-minimal-role> of any space will allowed 
#                                     to create/edit queries on this DB.
#                                   - If specified only OBSERVER of the space will be allowed to perform 
#                                     queries and <creator-minimal-role> of the space will allowed 
#                                     to create/edit queries on this DB.
#query-databases = openbisDB
#
#openbisDB.label = openBIS meta data
#openbisDB.data-space = CISD
#openbisDB.creator-minimal-role = SPACE_ADMIN
#openbisDB.database-driver = org.postgresql.Driver
#openbisDB.database-url = jdbc:postgresql://localhost/openbis_${database.kind}
#openbisDB.database-username =
#openbisDB.database-password =

# ---------------------------------------------------------------------------
# Maintenance plugins configuration (optional)
# ---------------------------------------------------------------------------
# Comma separated names of maintenance plugins. 
# Each plugin should have configuration properties prefixed with its name.
# Mandatory properties for each <plugin> include: 
#   <plugin>.class - Fully qualified plugin class name
#   <plugin>.interval - The time between plugin executions (in seconds)
# Optional properties for each <plugin> include:
#   <plugin>.start - Time of the first execution (HH:mm)
#   <plugin>.execute-only-once - If true the task will be executed exactly once, 
#                                interval will be ignored. By default set to false.
#maintenance-plugins = demo
#
#demo.class = ch.systemsx.cisd.openbis.generic.server.task.DemoMaintenanceTask
#demo.interval = 60
#demo.property_1 = some value
#demo.property_2 = some value 2

#
# Internal - do not change
#

# Authorization
# Supported: 'no-authorization' and 'active-authorization'
authorization-component-factory = active-authorization

script-folder = .

#
# Version of Jython to be used in plugins. 2.5 and 2.7 are supported
#
jython-version=2.7

##########
# V3 API #
##########

# -------------------------------------------------------------------------
# The configuration below reflects the default values used by the V3 API.
# Please uncomment and change the chosen values to overwrite the defaults. 
# -------------------------------------------------------------------------
#
# A path to a directory where operation execution details are stored.
#
#  api.v3.operation-execution.store.path = operation-execution-store
#
# A thread pool that is used for executing all asynchronous operations. 
#
# api.v3.operation-execution.thread-pool.name = operation-execution-pool
# api.v3.operation-execution.thread-pool.core-size = 10
# api.v3.operation-execution.thread-pool.max-size = 10
# api.v3.operation-execution.thread-pool.keep-alive-time = 0
#
# A name of a thread that updates operation execution progress information.
#
# api.v3.operation-execution.progress.thread-name = operation-execution-progress
#
# An interval that controls how often operation execution progress information gets updated. The interval is defined in seconds.
# 
# api.v3.operation-execution.progress.interval = 5
#
# Availability times control for how long information about an operation execution is stored in the system. 
# There are 3 levels of such information:
#
# * core information (code, state, owner, description, creation_date, start_date, finish_date)
# * summary information (summary of operations, progress, error, results)
# * detailed information (details of operations, progress, error, results)
#
# Each level of information can have a different availability time.
# The availability times can be defined at the moment of scheduling an operation execution.
# If a time is not provided explicitly then a corresponding 'default' value is used. 
# The maximum possible value that can be used for a given availability time is controlled with the 'max' property.
#
# All availability times are defined in seconds.
# Examples of values: 31536000 (1 year), 2592000 (30 days), 86400 (1 day), 3600 (1 hour).
#
# api.v3.operation-execution.availability-time.default = 31536000
# api.v3.operation-execution.availability-time.max = 31536000
# api.v3.operation-execution.availability-time.summary.default = 2592000
# api.v3.operation-execution.availability-time.summary.max = 2592000
# api.v3.operation-execution.availability-time.details.default = 86400 
# api.v3.operation-execution.availability-time.details.max = 86400
#
# Maintenance tasks responsible for marking and deleting timed out operation executions. Intervals are defined in seconds.
#
# api.v3.operation-execution.availability-update.mark-timeout-pending-task.name = operation-execution-mark-timeout-pending-task
# api.v3.operation-execution.availability-update.mark-timeout-pending-task.interval = 60
#
# api.v3.operation-execution.availability-update.mark-timed-out-or-deleted-task.name = operation-execution-mark-timed-out-or-deleted-task
# api.v3.operation-execution.availability-update.mark-timed-out-or-deleted-task.interval = 300
#
# Maintenance task responsible for marking new, scheduled and running operation executions as failed after server restart.
#
# api.v3.operation-execution.state-update.mark-failed-after-server-restart-task.name = operation-execution-mark-failed-after-server-restart-task
#
#
```

### Database Settings

All properties starting with `database.` specify the settings for the openBIS database. They are all mandatory.

| Property                         | Description                                                                                                                                                                                                           |
|----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `database.engine`                  | Type of database. Currently only postgresql is supported.                                                                                                                                                            |
| `database.create-from-scratch`     | If true the database will be dropped and an empty database will be created. In productive use always set this value to  false  .                                                                                     |
| `database.script-single-step-mode` | If true all SQL scripts are executed in single step mode. Useful for localizing errors in SQL scripts. Should be always false in productive mode.                                                                    |
| `database.url-host-part`           | Part of JDBC URL denoting the host of the database server. If openBIS Application Server and database server are running on the same machine this property should be an empty string.                                |
| `database.kind`                    | Part of the name of the database. The full name reads openbis_<  kind  >.                                                                                                                                      |
| `database.admin-user`              | ID of the user on database server with admin rights, like creation of tables. Should be an empty string if default admin user should be used. In case of PostgreSQL the default admin user is assumed to be postgres. |
| database.admin-password          | Password for admin user. Usual an empty string.                                                                                                                                                                      |
| `database.owner`                   | ID of the user owning the data. This should generally be openbis. The openbis role and password need to be created. In case of an empty string it is the same user who started up openBIS Application Server.        |
| `database.owner-password`          | Password of the owner.                                                                                                                                                                                               |
```{warning}
The credentials for the database user with the privilege to create a new database depends on the installation and configuration of the PostgreSQL database.
```

## Start Server

The server is started as follows:

```console
prompt> <installation folder>/bin/bisup.sh
```

On startup the openBIS server creates the database on PostgreSQL and
checks the connection with the remote authentication services (if they
are configured). Log files can be found in
`<installation folder>/servers/openBIS-server/jetty/logs`. Also the
following command shows the log: `<installation folder>/bin/bislog.sh`

```{warning}
The first user logged in into the system will have full administrator rights (role `INSTANCE_ADMIN`).
```

## Stop Server

The server is stopped as follows:

```console
prompt> <installation folder>/bin/bisdown.sh
```

## Authentication systems

Generic openBIS currently supports four authentication systems: a
self-contained system based on a UNIX-like passwd file, LDAP, the Crowd
system (see <http://www.atlassian.com/software/crowd>) and Single Sign
On (eg SWITCH AAI). Beside this there are also so called stacked
authentication methods available. Stacked authentication methods use
multiple authentication systems in the order indicated by the name. The
first authentication system being able to provide an entry for a
particular user id will be used. If you need full control over what
authentication systems are used in what order, you can define your own
stacked authentication service in the Spring application context file:
`<server folder>/openBIS-server/jetty/webapps/openbis/WEB-INF/classes/genericCommonContext.xml.`

### The default authentication configuration

In the template service properties, we set
`authentication-service = file-ldap-crowd-caching-authentication-service`,
which means that file-based authentication, LDAP and Crowd are used for
authentication, in this order. As LDAP and Crowd are not configured in
the template service properties, this effectively corresponds
to `file-authentication-service`, however when LDAP and / or Crowd are
configured, they are picked up on server start and are used to
authenticate users when they are not found in the local `passwd` file.
Furthermore, as it is a caching authentication service, it will cache
authentication results from LDAP and / or Crowd in
`file <server folder>/jetty/etc/passwd_cache`. See section
*Authentication Cache* below for details on this caching.

### The file based authentication system

This authentication schema uses the file
`<server folder>/jetty/etc/passwd` to determine whether a login to the
system is successful or not.

The script `<server folder>/jetty/bin/passwd.sh` can be used to maintain
this file. This script supports the options:

```bash
passwd list | [remove|show|test] <userid> | [add|change] <userid> [option [...]]
    --help                 : Prints out a description of the options.
    [-P,--change-password] : Read the new password from the console,
    [-e,--email] VAL       : Email address of the user.
    [-f,--first-name] VAL  : First name of the user.
    [-l,--last-name] VAL   : Last name of the user.
    [-p,--password] VAL    : The password.
```


A new user can be added with

```bash
prompt> passwd.sh add [-f <first name>] [-l <last name>] [-e <email>] [-p <password>] <username>
```

If no password is provided with the `-p` option, the system will ask for
a password of the new user on the console. Please note that providing a
password on the command line can be a security risk, because the
password can be found in the shell history, and, for a short time, in
the `ps` table. Thus `-p` is not recommended in normal operation.

The password of a user can be tested with

```bash
prompt> passwd.sh test <username>
```

The system will ask for the current password on the console and then
print whether the user was authenticated successfully or not.

An account can be changed with

```bash
prompt> passwd.sh change [-f <first name>] [-l <last name>] [-e <email>] [-P] <username>
```

An account can be removed with

```bash
prompt> passwd.sh remove <username>
```

The details of an account can be queried with

```bash
prompt> passwd.sh show <username>
```

All accounts can be listed with

```bash
prompt> passwd.sh list
```

The password file contains each user in a separate line. The fields of
each line are separated by colon and contain (in this order): *User Id*,
*Email Address*, *First Name*, *Last Name* and *Password Hash*.
The *Password Hash* field represents the
[salted](http://en.wikipedia.org/wiki/Salted_hash)
[SHA1](http://en.wikipedia.org/wiki/Sha1) hash of the user's password in
[BASE64 encoding](http://en.wikipedia.org/wiki/Base64).

### The interface to LDAP

To work with an LDAP server, you need to provide the server URL with
(example) and set the
`authentication-service = ldap-authentication-service`

```bash
ldap.server.url = ldap://d.ethz.ch/DC=d,DC=ethz,DC=ch
```

and the details of an LDAP account who is allowed to make queries on the
LDAP server with (example)

```bash
ldap.security.principal.distinguished.name = CN=carl,OU=EthUsers,DC=d,DC=ethz,DC=ch
ldap.security.principal.password = Carls_LDAP_Password
```


Note: A space-separated list of URLs can be provided if distinguished
name and password  are valid for all specified LDAP servers.

### The interface to Crowd

When setting `authentication-service = crowd-authentication-service` in
`service.properties`, the `passwd` file has no effect. Instead, the
following properties need to be configured via the following properties.

The URL (without port information):

    crowd.service.url = https://crowd.your.org

The Port of the URL:

    crowd.service.port = 443

The name of the application account in Crowd:

    crowd.application.name = openbis

The password of the application account in Crowd:

    crowd.application.password = <application password>

If Crowd is used as an authentication service, the IP of the openBIS
server and the name (of the openBIS application) has to be registered
with the Crowd server.

### Authentication Cache

If configuring a caching authentication service like
`file-ldap-crowd-caching-authentication-service`, authentication results
from remote authentication services like LDAP and / or Crowd are cached
locally in the openBIS Application Server. The advantage is a faster
login time on repeated logins when one or more remote authentication
services are slow. The disadvantage is that changes to data in the
remote authentication system (like a changed password or email address)
are becoming known to openBIS only with a delay. In order to minimize
this effect, the authentication caching performs "re-validation" of
authentication requests asynchronously. That means it doesn't block the
user from logging in because it is performed in different thread than
the login.

There are two service properties which give you control over the working
of the authentication cache:

- `authentication.cache.time` lets you set for how long (after putting
    it into the cache) a cache entry (read: "user name and password")
    will be kept if the user does not have a successful login to openBIS
    in this period of time (as successful logins will trigger
    re-validation and thus renewal of the cache entry). The default is
    28h, which means that users logging into the system every day will
    never experience a delay from slow remote authentication systems. A
    non-positive value will disable caching.
- `authentication.cache.time-no-revalidation` lets you set for how
    long (after putting it into the cache) a cache entry will *not* be
    re-validated if the login was successful. This allows you to reduce
    the load that openBIS creates on the remote authentication servers
    for successful logins of the same user. The default is 1h. Setting
    it to 0 will always trigger re-validation, setting it to
    `authentication.cache.time` will not perform re-validation at all
    and thus expire every cache entry after that time.

An administrator with shell access to the openBIS Application Server can
see and change the current cache entries in the
file `<server folder>/jetty/`etc/passwd\_cache. The format is the same
as for the file-based authentication system (see section *The file based
authentication system* above), but has an additional field *Cached At*
added to the end of each line. *Cached At* is the time (in milli-seconds
since start of the Unix epoch, which is midnight *Universal Time
Coordinated*, 1 January 1970) when the entry was cached. Removing a line
from this file will remove the corresponding cache entry. The
authentication cash survives openBIS Application Server restarts because
of this persisted file. If you need to clear the cache altogether, it is
sufficient to remove the `passwd_cache` file at any time. No server
restart is needed to make changes to this file take effect.

You can switch off authentication caching by either
setting `authentication.cache.time = -1`, or by choosing an
authentication service which does not have `caching` in its name.

### Anonymous Login

In order to allow anonymous login a certain user known by openBIS (not
necessarily by the authentication system) has to be specified. This is
done by the property `user-for-anonymous-login`. The value is the user
ID. The display settings and the authorization settings of this user are
used for the anonymous login.

Anonymous login is possible with URL parameter `anonymous` set to `true`
or by property `default-anonymous-login` in web configuration properties
(see [Web Client Customization](https://unlimited.ethz.ch/display/openBISDoc2010/Register+Master+Data+via+the+Admin+Interface)). Note, that for the ELN client the property `default-anonymous-login` isn't used. Anonymous login needs only the property `user-for-anonymous-login` for an existing user with some rights.

### Single Sign On Authentication

Currently only Shibboleth SSO is supported. For more details see [Single Sign On Authentication](https://openbis.readthedocs.io/en/latest/system-admin-documentation/installation/installation-and-configuration-guide.html#single-sign-on-authentication).

## Authorization

openBIS authorization is described here: [Authorization](https://openbis.readthedocs.io/en/latest/system-admin-documentation/installation/installation-and-configuration-guide.html#authorization).

## System Customization

### Login Page - Banners

To add banners to the main OpenBIS change `loginHeader.html` page. It is
stored in the same directory as `index.html`. Note that if the height of
`loginHeader.html` is too big, the content may overlap with the rest of
the OpenBIS login page.

Example of the `loginHeader.html`:

    <center><img src="images/banner.gif"></center>

For announcements you have to edit the `index.html` file. Here is an
example showing the tail:

```html
<input style="margin-left: 200px" type="submit" id="openbis_login_submit" value="Login">
<br>
<br>
<br>
<br>
<span style="color:red">
Due the server maintenance openBIS 
<br>
will not be available on 24th of 
<br>
    December 2010 from 10 am to 3 pm!
<br>
</span>
</form>
</div>
</body>
</html>
```


Note: the current work around with `br` tags between the lines ensures
that the login box is still centered.

### Client Customization

#### Configuration

To reconfigure some parts of the openBIS Web Client and Data Set Upload
Client, prepare the configuration file and add the path to the value of
`web-client-configuration-file` property in openBIS
`service.properties`.

    web-client-configuration-file = etc/web-client.properties

#### Web client customizations

- Enable the trashcan. When the trashcan is enabled, deleting entities
    only marks them as deleted but not deletes them physically (it is
    also called "logical deletion"). When clicking on the trashcan icon
    in the Web GUI, the user can see all of his deletion operations and
    can revert them individually. Only an admin can empty the trashcan
    and thus delete the entities physically. Only with enabled trashcan
    is it possible to delete complete hierarchies (e.g. an experiment
    with samples and datasets attached).
- Default view mode (`SIMPLE/NORMAL`) that should be used if user
    doesn't have it specified in a URL.
- Replacement texts for 'Experiment' and 'Sample' by `experiment-text`
    and `sample-text`, respectively.
- Anonymous login by default.
- Sample, material, experiment and data set `detail views `can be
    customized by:
    - hiding the sections (e.g. attachments)
- Additionally `data set detail view` can be customized by:
    - removing `Smart View` and `File View` from the list of available
        reports in `Data View` section
- Technology specific properties with property `technologies` which is
    a comma-separated list of technologies. For each technology
    properties are defined where the property names start with
    technology name followed by a dot character.

#### Data Set Upload Client Customizations

It is possible to restrict the set of data set types available to the
user in the data set uploader. This is useful when there are some data
set types that a user would never upload; for example, if there are data
set types that are used only internally exist only to support
third-party software.

The restriction is specified in the web-client.properties file using
either a whitelist or a blacklist. If both are specified, the whitelist
is used. To specify a whitelist, use the key
`creatable-data-set-types-whitelist`; for a blacklist, use the key
`creatable-data-set-types-blacklist`. The value for the property should
be a comma-separated list of regular-expression patterns to match. In
the case of the whitelist, data set types that match the specified
patterns are shown to the user, whereas for the blacklist, the data set
types that match the specified patterns are those that are not shown to
the user.

##### Examples

- Specifying a whitelist

**web-client.properties.**

    creatable-data-set-types-whitelist = .*IMAGE.*, ANALYSIS, THUMBNAIL[0-9]?

Assume we have the following data set types in our system:

*PROCESSED-DATA*, *MICROSCOPE-IMAGE*, *IMAGE-SCREENING*, *ANALYSIS*,
*ANALYSIS-FEATURES*, *THUMBNAIL1*, *THUMBNAIL-BIG*

In this case, the follwing data set types will be available to the user:

*MICROSCOPE-IMAGE*, *IMAGE-SCREENING*, *ANALYSIS*, *THUMBNAIL1*

- Specifying a blacklist

**web-client.properties.**

    creatable-data-set-types-blacklist = .*IMAGE.*, ANALYSIS, THUMBNAIL[0-9]?

Assume we have the following data set types in our system:

*PROCESSED-DATA*, *MICROSCOPE-IMAGE*, *IMAGE-SCREENING*, *ANALYSIS*,
*ANALYSIS-FEATURES*, *THUMBNAIL1*, *THUMBNAIL-BIG*

In this case, the follwing data set types will be available to the user:

*PROCESSED-DATA*, *ANALYSIS-FEATURES*, *THUMBNAIL-BIG*

#### Full web-client.properties Example

**web-client.properties**

```
# Enable the trash can and logical deletion.
# Default value: false
enable-trash = true

# Replacement texts for 'Experiment' and 'Sample' in the UI 
# sample-text = Object
# experiment-text = Collection

# Default view mode that should be used if user doesn't have it specified in URL.
# Options: 'NORMAL' (standard or application mode - default), 'SIMPLE' (read-only mode with simplified GUI)
#
default-view-mode = SIMPLE

# Flag specifying whether default login mode is anonymous or not. 
# If true a user-for-anonymous-login has to be defined in service.properties
# Default value: false
default-anonymous-login = true

# Configuration of entity (experiment, sample, data set, material) detail views.
#
# Mandatory properties:
#   - view (entity detail view id)
#   - types (list of entity type codes)
# Optional properties:
#   - hide-sections (list of section ids)
#   - hide-smart-view (removes "Smart View" from Data Set Detail View -> Data View) (generic_dataset_viewer)
#   - hide-file-view (removes "File View" from Data Set Detail View -> Data View) (generic_dataset_viewer)
# Available sections in entity-detail-views:
#   generic_dataset_viewer
#       data-set-data-section
#       data-set-parents-section
#       data-set-children-section
#       query-section
#   generic_experiment_viewer
#       data-set-section
#       attachment-section
#       query-section
#       container-sample-section
#   generic_sample_viewer
#       container-sample-section
#       derived-samples-section
#       parent-samples-section
#       data-set-section
#       attachment-section
#       query-section
#   generic_material_viewer
#       query-section
#
# Example:
#
#detail-views = sample-view, experiment-view, data-view
#
#sample-view.view = generic_sample_viewer
#sample-view.types = STYPE1, STYPE2
#sample-view.hide-sections = derived-samples-section, container-sample-section
#
#experiment-view.view = generic_sample_viewer
#experiment-view.types = ETYPE1, ETYPE2
#experiment-view.hide-sections = data-set-section
#
#data-view.view = generic_dataset_viewer
#data-view.types = DSTYPE
#data-view.hide-smart-view = false
#data-view.hide-file-view = true

#technologies = screening
#screening.image-viewer-enabled = true

#
# Only render these types when creating new data sets via the 
# Data Set Upload Client
#
#creatable-data-set-types-whitelist=WHITELISTED_TYPE1, WHITELISTED_TYPE2

#
# Do not render these types in the Data Set Upload Client. 
# The value of the property is only taken into account if  
# creatable-data-set-types-whitelist is not configured
#
#creatable-data-set-types-blacklist=BLACKLISTED_TYPE1, BLACKLISTED_TYPE2
```


### Configuring File Servlet

This service is specially tailored for web applications requiring to
upload files to the system without using the DataSet concept, it was
meant to be used for small images and rich text editors like CKEditor.

| Property Key                        | Default Value              | Description                                                                                               |
|-------------------------------------|----------------------------|-----------------------------------------------------------------------------------------------------------|
| file-server.maximum-file-size-in-MB | 10                         |  Max size of files.                                                                                      |
| file-server.repository-path         |  ../../../data/file-server | Path where files will be stored, ideally should be a folder on the same NAS you are storing the DataSets. |
| file-server.download-check          | true                       | Checks that the user is log in into the system to be able to download files.                             |

### Configuring DSS Data Sources

It is quite common that openBIS AS is using a database filled by DSS.
Depending on the DSS (specified by the DSS code) and the technology
different databases have to be used.

Configuration is best done by AS [core
plugins](https://openbis.readthedocs.io/en/latest/software-developer-documentation/server-side-extensions/core-plugins.html) of type
`dss-data-sources`. The name of the plugin is just the DSS code. The
following properties of `plugin.properties` are recognized:

| Property Key                                     | Description                                                                                                                                                                                                                             |
|--------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| technology                                       | Normally the technology/module folder of the core plugin specifies the technology/module for which this data source has to be configured. If this is not the case this property allows to specify the technology/module independently.  |
| database-driver                                  | Fully qualified class name of the data base driver, e.g. `org.postgresql.Driver`.                                                                                                                                                        |
| database-url                                     | URL of the database, e.g. `jdbc:postgresql://localhost/imaging_dev`                                                                                                                                                                       |
| username                                         | Optional user name needed to access database.                                                                                                                                                                                          |
| password                                         | Optional password needed to access database.                                                                                                                                                                                           |
| validation-query                                 | Optional SQL script to be executed to validate database connections.                                                                                                                                                                   |
| database-max-idle-connections                    | The maximum number of connections that can remain idle in the pool. A negative value indicates that there is no limit. Default: -1                                                                                                      |
| database-max-active-connections                  | The maximum number of active connections that can be allocated at the same time. A negative value indicates that there is no limit. Default: -1                                                                                         |
| database-max-wait-for-connection                 | The maximum number of seconds that the pool will wait for a connection to be returned before throwing an exception. A value less than or equal to zero means the pool is set to wait indefinitely. Default: -1                          |
| database-active-connections-log-interval         | The interval (in ms) between two regular log entries of currently active database connections if more than one connection is active. Default: Disabled                                                                                  |
| database-active-number-connections-log-threshold | The number of active connections that will trigger a NOTIFY log and will switch on detailed connection logging. Default: Disabled                                                                                                       |
| database-log-stacktrace-on-connection-logging    | If true and logging enabled also stack traces are logged. Default: `false`                                                                                                                                                                |

Properties `database-driver` and `database-url` can be omitted if a
`etc/dss-datasource-mapping` is defined. For more see [Sharing
Databases](https://unlimited.ethz.ch/display/openBISDoc2010/Sharing+Databases).

### Changing the Capability-Role map

openBIS uses a map of capabilities to roles to decide what role is
needed to perform a given action. The defaults can be overridden by
creating a file `etc/capabilities`. One line in this file has one of the
following formats:

```
<Capability>: <Role>[,<ROLE>...]
<Capability>: <Role>[,<ROLE>...][; <Parameter> = <Role>[, <Role>...]][; <Parameter> = <Role>[, <Role>]] ...
<Capability>: <Parameter> = <Role>[, <Role>...][; <Parameter> = <Role>[, <Role>]] ...
```


which sets a new (minimal) role for the given capability. There is a
special role `INSTANCE_DISABLED` which allows to completely disable a
capability for an instance. Note: to set multiple roles for single