-`run-console.sh` Installation script for console/non-gui installation
-`run-ui.sh` Installation script for gui installation
### Installation steps
1. Create a service user account, i.e. an unprivileged, regular user account. **Do not run openBIS as root!**
2. Gunzip the distribution on the server machine into some temporary folder.
3. Run either the console/non-gui installation script or the gui installation script:
**GUI-based installation**
```bash
tar xvfz openBIS-installation-standard-technologies-S139.0-r26480.tar.gz
cd openBIS-installation-standard-technologies-S139.0-r26480
./run-ui.sh
```
In the non-gui version you have to edit the `console.properties`
files:
**Non-GUI installation**
```bash
tar xvfz openBIS-installation-standard-technologies-S139.0-r26480.tar.gz
cd openBIS-installation-standard-technologies-S139.0-r26480
vi console.properties
./run-console.sh
```
```{note}
Please be aware that the directory where openbis is installed should not already exist. Users should specify the directory where they want to install openBIS (in the console.properties or in the graphical installer) and this directory will be created by the installation procedure. If the directory already exists, the installation will fail.
```
After the successful installation you should have a look at the configuration file called s`ervice.properties`. It is located in `<server_folder>openBIS-server/jetty/etc/`
This file is a an [Extended Properties File](./optional-application-server-configuration.md). Here is an example which can be used as a template:
| `database.engine` | Type of database. Currently only postgresql is supported. |
| `database.create-from-scratch` | If true the database will be dropped and an empty database will be created. In productive use always set this value to false . |
| `database.script-single-step-mode` | If true all SQL scripts are executed in single step mode. Useful for localizing errors in SQL scripts. Should be always false in productive mode. |
| `database.url-host-part` | Part of JDBC URL denoting the host of the database server. If openBIS Application Server and database server are running on the same machine this property should be an empty string. |
| `database.kind` | Part of the name of the database. The full name reads openbis_<kind>. |
| `database.admin-user` | ID of the user on database server with admin rights, like creation of tables. Should be an empty string if default admin user should be used. In case of PostgreSQL the default admin user is assumed to be postgres. |
| database.admin-password | Password for admin user. Usual an empty string. |
| `database.owner` | ID of the user owning the data. This should generally be openbis. The openbis role and password need to be created. In case of an empty string it is the same user who started up openBIS Application Server. |
| `database.owner-password` | Password of the owner. |
```{warning}
The credentials for the database user with the privilege to create a new database depends on the installation and configuration of the PostgreSQL database.
```
## Start Server
The server is started as follows:
```console
prompt><installation folder>/bin/bisup.sh
```
On startup the openBIS server creates the database on PostgreSQL and
checks the connection with the remote authentication services (if they
are configured). Log files can be found in
`<installation folder>/servers/openBIS-server/jetty/logs`. Also the
following command shows the log: `<installation folder>/bin/bislog.sh`
```{warning}
The first user logged in into the system will have full administrator rights (role `INSTANCE_ADMIN`).
```
## Stop Server
The server is stopped as follows:
```console
prompt><installation folder>/bin/bisdown.sh
```
## Authentication systems
Generic openBIS currently supports four authentication systems: a
self-contained system based on a UNIX-like passwd file, LDAP, the Crowd
system (see <http://www.atlassian.com/software/crowd>) and Single Sign
On (eg SWITCH AAI). Beside this there are also so called stacked
authentication methods available. Stacked authentication methods use
multiple authentication systems in the order indicated by the name. The
first authentication system being able to provide an entry for a
particular user id will be used. If you need full control over what
authentication systems are used in what order, you can define your own
stacked authentication service in the Spring application context file:
from remote authentication services like LDAP and / or Crowd are cached
locally in the openBIS Application Server. The advantage is a faster
login time on repeated logins when one or more remote authentication
services are slow. The disadvantage is that changes to data in the
remote authentication system (like a changed password or email address)
are becoming known to openBIS only with a delay. In order to minimize
this effect, the authentication caching performs "re-validation" of
authentication requests asynchronously. That means it doesn't block the
user from logging in because it is performed in different thread than
the login.
There are two service properties which give you control over the working
of the authentication cache:
-`authentication.cache.time` lets you set for how long (after putting
it into the cache) a cache entry (read: "user name and password")
will be kept if the user does not have a successful login to openBIS
in this period of time (as successful logins will trigger
re-validation and thus renewal of the cache entry). The default is
28h, which means that users logging into the system every day will
never experience a delay from slow remote authentication systems. A
non-positive value will disable caching.
-`authentication.cache.time-no-revalidation` lets you set for how
long (after putting it into the cache) a cache entry will *not* be
re-validated if the login was successful. This allows you to reduce
the load that openBIS creates on the remote authentication servers
for successful logins of the same user. The default is 1h. Setting
it to 0 will always trigger re-validation, setting it to
`authentication.cache.time` will not perform re-validation at all
and thus expire every cache entry after that time.
An administrator with shell access to the openBIS Application Server can
see and change the current cache entries in the
file `<server folder>/jetty/`etc/passwd\_cache. The format is the same
as for the file-based authentication system (see section *The file based
authentication system* above), but has an additional field *Cached At*
added to the end of each line. *Cached At* is the time (in milli-seconds
since start of the Unix epoch, which is midnight *Universal Time
Coordinated*, 1 January 1970) when the entry was cached. Removing a line
from this file will remove the corresponding cache entry. The
authentication cash survives openBIS Application Server restarts because
of this persisted file. If you need to clear the cache altogether, it is
sufficient to remove the `passwd_cache` file at any time. No server
restart is needed to make changes to this file take effect.
You can switch off authentication caching by either
setting `authentication.cache.time = -1`, or by choosing an
authentication service which does not have `caching` in its name.
### Anonymous Login
In order to allow anonymous login a certain user known by openBIS (not
necessarily by the authentication system) has to be specified. This is
done by the property `user-for-anonymous-login`. The value is the user
ID. The display settings and the authorization settings of this user are
used for the anonymous login.
Anonymous login is possible with URL parameter `anonymous` set to `true`
or by property `default-anonymous-login` in web configuration properties
(see [Web Client Customization](./installation-and-configuration-guide.md#web-client-customizations)). Note, that for the ELN client the property `default-anonymous-login` isn't used. Anonymous login needs only the property `user-for-anonymous-login` for an existing user with some rights.
### Single Sign On Authentication
Currently only Shibboleth SSO is supported. For more details see [Single Sign On Authentication](./installation-and-configuration-guide.md#single-sign-on-authentication).
## Authorization
openBIS authorization is described here: [Authorization](./installation-and-configuration-guide.md#authorization).
## System Customization
### Login Page - Banners
To add banners to the main OpenBIS change `loginHeader.html` page. It is
stored in the same directory as `index.html`. Note that if the height of
`loginHeader.html` is too big, the content may overlap with the rest of
the OpenBIS login page.
Example of the `loginHeader.html`:
<center><img src="images/banner.gif"></center>
For announcements you have to edit the `index.html` file. Here is an
| file-server.maximum-file-size-in-MB | 10 | Max size of files. |
| file-server.repository-path | ../../../data/file-server | Path where files will be stored, ideally should be a folder on the same NAS you are storing the DataSets. |
| file-server.download-check | true | Checks that the user is log in into the system to be able to download files. |
### Configuring DSS Data Sources
It is quite common that openBIS AS is using a database filled by DSS.
Depending on the DSS (specified by the DSS code) and the technology
different databases have to be used.
Configuration is best done by AS [core
plugins](../../../software-developer-documentation/server-side-extensions/core-plugins.md) of type
`dss-data-sources`. The name of the plugin is just the DSS code. The
following properties of `plugin.properties` are recognized:
| technology | Normally the technology/module folder of the core plugin specifies the technology/module for which this data source has to be configured. If this is not the case this property allows to specify the technology/module independently. |
| database-driver | Fully qualified class name of the data base driver, e.g. `org.postgresql.Driver`. |
| database-url | URL of the database, e.g. `jdbc:postgresql://localhost/imaging_dev` |
| username | Optional user name needed to access database. |
| password | Optional password needed to access database. |
| validation-query | Optional SQL script to be executed to validate database connections. |
| database-max-idle-connections | The maximum number of connections that can remain idle in the pool. A negative value indicates that there is no limit. Default: -1 |
| database-max-active-connections | The maximum number of active connections that can be allocated at the same time. A negative value indicates that there is no limit. Default: -1 |
| database-max-wait-for-connection | The maximum number of seconds that the pool will wait for a connection to be returned before throwing an exception. A value less than or equal to zero means the pool is set to wait indefinitely. Default: -1 |
| database-active-connections-log-interval | The interval (in ms) between two regular log entries of currently active database connections if more than one connection is active. Default: Disabled |
| database-active-number-connections-log-threshold | The number of active connections that will trigger a NOTIFY log and will switch on detailed connection logging. Default: Disabled |
| database-log-stacktrace-on-connection-logging | If true and logging enabled also stack traces are logged. Default: `false` |
Properties `database-driver` and `database-url` can be omitted if a
`etc/dss-datasource-mapping` is defined. For more see [Sharing
|`DELETE_DATASET` | |`SPACE_POWER_USER` |Delete datasets (this capability IS NOT enough to delete datasets with deletion_disallow flag set to true in their type - see `FORCE_DELETE_DATASET`) |
|`FORCE_DELETE_DATASET` | |`INSTANCE_DISABLED` |Delete datasets (this capability IS enough to delete datasets with deletion_disallow flag set to true in their type - see `DELETE_DATASET`) |
|`ARCHIVE_DATASET` | |`SPACE_POWER_USER` |Move dataset from data store into archive |
|`UNARCHIVE_DATASET` | |`SPACE_USER` |Copy back dataset from archive to data store |
|`LOCK_DATA_SETS` | |`SPACE_ADMIN` |Prevent data sets from being archived |
|`UNLOCK_DATA_SETS` | |`SPACE_ADMIN` |Release locked data sets |
|`WRITE_EXPERIMENT_SAMPLE_MATERIAL`| |`INSTANCE_ADMIN` |Registration / update of experiments, samples and materials in one go |
|`REGISTER_SPACE` | |`SPACE_ADMIN` |The user will become space admin of the freshly created space |
|`PURGE` | |`SPACE_ADMIN` |Permanently delete experiments, samples and datasets in the trashcan (this capability IS NOT enough to delete datasets with deletion_disallow flag set to true in their type - see `FORCE_PURGE`)|
|`FORCE_PURGE` | |`INSTANCE_DISABLED` |Permanently delete experiments, samples and datasets in the trashcan (this capability IS enough to delete datasets with deletion_disallow flag set to true in their type - see `PURGE`) |
|`RESTORE` | |`SPACE_USER` |Get back experiments, samples and datasets from the trashcan |
|`ASSIGN_SAMPLE_TO_EXPERIMENT` | |`SPACE_POWER_USER`, `SPACE_ETL_SERVER`|Re-assign a sample to a new experiment (called in 'register experiment', 'update experiment', 'update sample'') |
|`SEARCH_ON_BEHALF_OF_USER` | |`INSTANCE_OBSERVER` |All search or list operations being performed on behalf of another user. Supposed to be used by a service user for server-to-server communication tasks. |
Older versions of openBIS used to allow changing entity relationships to
regular `SPACE_USER`. If you want to get this behavior back, put these
lines into `etc/capabilities`:
```
ASSIGN_EXPERIMENT_TO_PROJECT: SPACE_USER
ASSIGN_EXPERIMENT_TO_PROJECT: SPACE_ETL_SERVER
ASSIGN_SAMPLE_TO_EXPERIMENT: SPACE_USER
ASSIGN_SAMPLE_TO_EXPERIMENT: SPACE_ETL_SERVER
UNASSIGN_SAMPLE_FROM_EXPERIMENT: SPACE_USER
UNASSIGN_SAMPLE_FROM_EXPERIMENT: SPACE_ETL_SERVER
ASSIGN_SAMPLE_TO_SPACE: SPACE_USER
ASSIGN_SAMPLE_TO_SPACE: SPACE_ETL_SERVER
ASSIGN_DATASET_TO_EXPERIMENT: SPACE_USER
ASSIGN_DATASET_TO_EXPERIMENT: SPACE_ETL_SERVER
ASSIGN_DATASET_TO_SAMPLE: SPACE_USER
ASSIGN_DATASET_TO_SAMPLE: SPACE_ETL_SERVER
ADD_PARENT_TO_SAMPLE: SPACE_USER
ADD_PARENT_TO_SAMPLE: SPACE_ETL_SERVER
REMOVE_PARENT_FROM_SAMPLE: SPACE_USER
REMOVE_PARENT_FROM_SAMPLE: SPACE_ETL_SERVER
ADD_CONTAINER_TO_SAMPLE: SPACE_USER
ADD_CONTAINER_TO_SAMPLE: SPACE_ETL_SERVER
REMOVE_CONTAINER_FROM_SAMPLE: SPACE_USER
REMOVE_CONTAINER_FROM_SAMPLE: SPACE_ETL_SERVER
ADD_PARENT_TO_DATASET: SPACE_USER
ADD_PARENT_TO_DATASET: SPACE_ETL_SERVER
REMOVE_PARENT_FROM_DATASET: SPACE_USER
REMOVE_PARENT_FROM_DATASET: SPACE_ETL_SERVER
ADD_CONTAINER_TO_DATASET: SPACE_USER
ADD_CONTAINER_TO_DATASET: SPACE_ETL_SERVER
REMOVE_CONTAINER_FROM_DATASET: SPACE_USER
REMOVE_CONTAINER_FROM_DATASET: SPACE_ETL_SERVER
```
#### Capability Role Map for V3 API
| Method of IApplicationServerApi | Default Roles | Capability |
This new user is added to the read-only group by the following command:
```sql
GRANT<read-onlygroup>TO<read-onlyuser>;
```
The name of the read-only group can be obtained by having a look into
the list of all groups:
```sql
SELECT*fromPG_GROUP;
```
*Note that by default openBIS creates a user*` openbis_readonly `
*which has read-only permissions to all database objects. You can use
this user to access the openBIS meta database through the openBIS query
interface.*
#### Enable Querying
To enable querying functionality for additional databases in openBIS Web application a [core plugin](../../../software-developer-documentation/server-side-extensions/core-plugins.md#core-plugins) of type query-databases has to be created. The following `plugin.properties` have to be specified:
| <database>.data-space | To which data-space this database belongs to (optional, i.e. a query database can be configured not to belong to one data space by leaving this configuration value empty). |
| <database>.creator-minimal-role | What role is required to be allowed to create / edit queries on this database (optional, default: INSTANCE_OBSERVER if data-space is not set, POWER_USER otherwise). |
The given parameters data-space and creator-minimal-role are used by
openBIS to enforce proper authorization.
For example, if
data-space = CISD
creator-minimal-role = SPACE_ADMIN
is configured, then for the query database configured with key `db1`:
- only a `SPACE_ADMIN` on data space `CISD` and an `INSTANCE_ADMIN`
are allowed to create / edit queries,
- only a user who has the `OBSERVER` role in data space `CISD` is
allowed to execute a query.
For query databases that do not belong to a space but that have a column with any of the [magic column names](../../../user-documentation/general-admin-users/custom-database-queries.md#hyperlinks), the query result is filtered on a per-row basis according to what the user executing the query is allowed to see. In detail this means: if the user executing the query is not an instance admin, filter out all rows which belong to a data space where the user doesn't have a least the observer role. The relationship between a row and a data space is established by means of the experiment / sample / data set whose `permId` is given by one of the magical column names.
For sensitive data where authorization needs to be enforced, there are
two setups possible:
1. Configure a query database that **does not** belong to a data space
and set the creator-minimal-role to `INSTANCE_ADMIN`. Any instance
admin can be trusted to understand authorization issues and ensure
that only queries are added for this query database that contain a
proper reference to an experiment / sample / data set. This way, it
can be ensured that only properly filtered query results are
returned to the user running the query.
2. Configure a query database that **does** belong to a specific data
space and set the creator-minimal-role to `POWER_USER`. The
datastore server (or whatever server maintains the query database)
ensures that only information related to the configured data space
is added to the query database. Thus whatever query the power user
writes for this database, it will only reveal information from this
data space. As only users with `OBSERVER` role on this data space
are allowed to execute the query, authorization is enforced properly
without the need of filtering query results.
### Master data import/export
The master data of openBIS comprises all entity/property types, property
assignments, vocabularies etc. needed for your customized installation
to work. The system offers a way to export/import master data via Jython
scripts. More information on how to do create such scripts and run them
manually see the advanced guide [Jython Master Data Scripts](https://unlimited.ethz.ch/display/openBISDoc2010/Jython+Master+Data+Scripts#JythonMasterDataScripts-Commandlinetools).
A master data script can be run automatically by start up of the AS if
it is defined in an AS core plugin. The script path should be
| log-service-calls | 'on', 'off' | 'off' | Turns on / off detailed service call logging.
When this feature is enabled, openBIS will log about start and end of every service call it executes to file <installationdirectory>/servers/openBIS-server/jetty/log/openbis_service_calls.txt |
| log-long-running-invocations | 'on', 'off' | 'on' | Turns on / off logging of long running invocations.
When this feature is enabled, openBIS will periodically create a report of all service calls that have been in execution more than 15 seconds to file <installationdirectory>/servers/openBIS-server/jetty/log/openbis_long_running_threads.txt. |
| debug-db-connections | 'on', 'off' | 'off' | Turns on / off logging about database connection pool activity.
When this feature is enabled, information about every borrow and return to database connection pool is logged to openBIS main log in file <installationdirectory>/servers/openBIS-server/jetty/log/openbis_log.txt |
| log-db-connections | no argument / minimum connection age (in milliseconds) | 5000 | When this command is executed without an argument, information about every database connection that has been borrowed from the connection pool is written into openBIS main log in file <installationdirectory>/servers/openBIS-server/jetty/log/openbis_log.txt
If the "minimum connection age" argument is specified, only connections that have been out of the pool longer than the specified time are logged. The minimum connection age value is given in milliseconds. |
| record-stacktrace-db-connections | 'on', 'off' | 'off' | Turns on / off logging of stacktraces.
When this feature is enabled AND debug-db-connections is enabled, the full stack trace of the borrowing thread will be recorded with the connection pool activity logs. |
| log-db-connections-separate-log-file | 'on', 'off' | 'off' | Turns on / off database connection pool logging to separate file.
When this feature is disabled, the database connection pool activity logging is done only to openBIS main log. When this feature is enabled, the activity logging is done ALSO to file <installationdirectory>/servers/openBIS-server/jetty/log/openbis_db_connections.txt. |
### Deleted Entity History
Logging the history of deleted entities can be enabled / disabled in
service.properties using setting
entity-history.enabled = \[true | false\]
Since 20.10.1 the default value is true (meaning, entity history is
enabled). Before 20.10.1 the default value was false.
Deleted entity history can be queried with script show-history.sh, which
is located in $OPENBIS\_INSTALL\_DIR/bin
## Troubleshooting Problems
### Samples with datasets and no experiments
In the openBIS UI users could detach samples with container data sets
from the experiment. This bug was fix on version S176 released on 14 of
march of 2014.
The following SQL script lists all samples with data sets but no
