pyBIS is a Python module for interacting with openBIS. pyBIS is designed to be most useful in a [Jupyter Notebook](https://jupyter.org) or IPython environment, especially if you are developing Python scripts for automatisation. Jupyter Notebooks offer some sort of IDE for openBIS, supporting TAB completition and immediate data checks, making the life of a researcher hopefully easier.
pyBIS is a Python module for interacting with openBIS. pyBIS is designed to be most useful in a [Jupyter Notebook](https://jupyter.org) or IPython environment, especially if you are developing Python scripts for automatisation. Jupyter Notebooks offer some sort of IDE for openBIS, supporting TAB completition and immediate data checks, making the life of a researcher hopefully easier.
...
@@ -62,9 +62,9 @@ pip install jupyterlab
...
@@ -62,9 +62,9 @@ pip install jupyterlab
-**property type:** a single property, as defined in the entity types above. It can be of a classic data type (e.g. INTEGER, VARCHAR, BOOLEAN) or its values can be controlled (CONTROLLEDVOCABULARY).
-**property type:** a single property, as defined in the entity types above. It can be of a classic data type (e.g. INTEGER, VARCHAR, BOOLEAN) or its values can be controlled (CONTROLLEDVOCABULARY).
-**plugin:** a script written in [Jython](https://www.jython.org) which allows to check property values in a even more detailed fashion
-**plugin:** a script written in [Jython](https://www.jython.org) which allows to check property values in a even more detailed fashion
# connect to OpenBIS
## connect to OpenBIS
## login
### login
In an **interactive session** e.g. inside a Jupyter notebook, you can use `getpass` to enter your password safely:
In an **interactive session** e.g. inside a Jupyter notebook, you can use `getpass` to enter your password safely:
...
@@ -215,7 +215,7 @@ Currently, mounting is supported for Linux and Mac OS X only.
...
@@ -215,7 +215,7 @@ Currently, mounting is supported for Linux and Mac OS X only.
All attributes, if not provided, are re-used by a previous login() command. If no mountpoint is provided, the default mounpoint will be `~/hostname`. If this directory does not exist, it will be created. The directory must be empty before mounting.
All attributes, if not provided, are re-used by a previous login() command. If no mountpoint is provided, the default mounpoint will be `~/hostname`. If this directory does not exist, it will be created. The directory must be empty before mounting.
# Masterdata
## Masterdata
OpenBIS stores quite a lot of meta-data along with your dataSets. The collection of data that describes this meta-data (i.e. meta-meta-data) is called masterdata. It consists of:
OpenBIS stores quite a lot of meta-data along with your dataSets. The collection of data that describes this meta-data (i.e. meta-meta-data) is called masterdata. It consists of:
...
@@ -230,7 +230,7 @@ OpenBIS stores quite a lot of meta-data along with your dataSets. The collection
...
@@ -230,7 +230,7 @@ OpenBIS stores quite a lot of meta-data along with your dataSets. The collection
- tags
- tags
- semantic annotations
- semantic annotations
## browse masterdata
### browse masterdata
```
```
sample_types = o.get_sample_types() # get a list of sample types
sample_types = o.get_sample_types() # get a list of sample types
**Samples** (objects), **experiments** (collections) and **dataSets** contain type-specific **properties**. When you create a new sample, experiment or datasSet of a given type, the set of properties is well defined. Also, the values of these properties are being type-checked.
**Samples** (objects), **experiments** (collections) and **dataSets** contain type-specific **properties**. When you create a new sample, experiment or datasSet of a given type, the set of properties is well defined. Also, the values of these properties are being type-checked.
...
@@ -334,7 +334,7 @@ To create a **tabular, spreadsheet-like property**, use `XML` as `dataType` and
...
@@ -334,7 +334,7 @@ To create a **tabular, spreadsheet-like property**, use `XML` as `dataType` and
**Note**: PropertyTypes that start with a \$ are by definition `managedInternally` and therefore this attribute must be set to True.
**Note**: PropertyTypes that start with a \$ are by definition `managedInternally` and therefore this attribute must be set to True.
## create sample types / object types
### create sample types / object types
The second step (after creating a property type, see above) is to create the **sample type**. The new name for **sample** is **object**. You can use both methods interchangeably:
The second step (after creating a property type, see above) is to create the **sample type**. The new name for **sample** is **object**. You can use both methods interchangeably:
...
@@ -365,7 +365,7 @@ sample_type.get_next_code() # e.g. FLY77
...
@@ -365,7 +365,7 @@ sample_type.get_next_code() # e.g. FLY77
From pyBIS 1.31.0 onwards, you can provide a `code` even for samples where its sample type has `autoGeneratedCode=True` to offer the same functionality as ELN-LIMS. In earlier versions of pyBIS, providing a code in this situation caused an error.
From pyBIS 1.31.0 onwards, you can provide a `code` even for samples where its sample type has `autoGeneratedCode=True` to offer the same functionality as ELN-LIMS. In earlier versions of pyBIS, providing a code in this situation caused an error.
## assign and revoke properties to sample type / object type
### assign and revoke properties to sample type / object type
The third step, after saving the sample type, is to **assign or revoke properties** to the newly created sample type. This assignment procedure applies to all entity types (dataset type, experiment type).
The third step, after saving the sample type, is to **assign or revoke properties** to the newly created sample type. This assignment procedure applies to all entity types (dataset type, experiment type).
The second step (after creating a **property type**, see above) is to create the **dataset type**. The third step is to **assign or revoke the properties** to the newly created dataset type.
The second step (after creating a **property type**, see above) is to create the **dataset type**. The third step is to **assign or revoke the properties** to the newly created dataset type.
Plugins are Jython scripts that can accomplish more complex data-checks than ordinary types and vocabularies can achieve. They are assigned to entity types (dataset type, sample type etc). [Documentation and examples can be found here](https://wiki-bsse.ethz.ch/display/openBISDoc/Properties+Handled+By+Scripts)
Plugins are Jython scripts that can accomplish more complex data-checks than ordinary types and vocabularies can achieve. They are assigned to entity types (dataset type, sample type etc). [Documentation and examples can be found here](https://wiki-bsse.ethz.ch/display/openBISDoc/Properties+Handled+By+Scripts)
...
@@ -453,7 +453,7 @@ pl = o.new_plugin(
...
@@ -453,7 +453,7 @@ pl = o.new_plugin(
pl.save()
pl.save()
```
```
## Users, Groups and RoleAssignments
### Users, Groups and RoleAssignments
Users can only login into the openBIS system when:
Users can only login into the openBIS system when:
...
@@ -496,7 +496,7 @@ ra = o.get_role_assignment(techId)
...
@@ -496,7 +496,7 @@ ra = o.get_role_assignment(techId)
ra.delete()
ra.delete()
```
```
## Spaces
### Spaces
Spaces are fundamental way in openBIS to divide access between groups. Within a space, data can be easily shared. Between spaces, people need to be given specific access rights (see section above). The structure in openBIS is as follows:
Spaces are fundamental way in openBIS to divide access between groups. Within a space, data can be easily shared. Between spaces, people need to be given specific access rights (see section above). The structure in openBIS is as follows:
...
@@ -533,7 +533,7 @@ space.attrs.all()
...
@@ -533,7 +533,7 @@ space.attrs.all()
space.delete('reason for deletion')
space.delete('reason for deletion')
```
```
## Projects
### Projects
Projects live within spaces and usually contain experiments (aka collections):
Projects live within spaces and usually contain experiments (aka collections):
exp.attrs.all() # returns all attributes as a dict
exp.attrs.all() # returns all attributes as a dict
...
@@ -682,7 +682,7 @@ exp.freezeForSamples = True
...
@@ -682,7 +682,7 @@ exp.freezeForSamples = True
exp.save() # needed to save/update the changed attributes and properties
exp.save() # needed to save/update the changed attributes and properties
```
```
### Experiment properties
#### Experiment properties
**Getting properties**
**Getting properties**
...
@@ -713,7 +713,7 @@ experiment.set_props({ key: value }) # set the values of some properties
...
@@ -713,7 +713,7 @@ experiment.set_props({ key: value }) # set the values of some properties
experiment.save() # needed to save/update the changed attributes and properties
experiment.save() # needed to save/update the changed attributes and properties
```
```
## Samples / Objects
### Samples / Objects
Samples usually live within experiments/collections:
Samples usually live within experiments/collections:
...
@@ -778,7 +778,7 @@ sample.add_attachment('testfile.xls') # deprecated, see above
...
@@ -778,7 +778,7 @@ sample.add_attachment('testfile.xls') # deprecated, see above
sample.delete('deleted for some reason')
sample.delete('deleted for some reason')
```
```
## create/update/delete many samples in a transaction
### create/update/delete many samples in a transaction
Creating a single sample takes some time. If you need to create many samples, you might want to create them in one transaction. This will transfer all your sample data at once. The Upside of this is the **gain in speed**. The downside: this is a **all-or-nothing** operation, which means, either all samples will be registered or none (if any error occurs).
Creating a single sample takes some time. If you need to create many samples, you might want to create them in one transaction. This will transfer all your sample data at once. The Upside of this is the **gain in speed**. The downside: this is a **all-or-nothing** operation, which means, either all samples will be registered or none (if any error occurs).
...
@@ -818,7 +818,7 @@ trans.commit()
...
@@ -818,7 +818,7 @@ trans.commit()
**Note:** You can use the `mark_to_be_deleted()`, `unmark_to_be_deleted()` and `is_marked_to_be_deleted()` methods to set and read the internal flag.
**Note:** You can use the `mark_to_be_deleted()`, `unmark_to_be_deleted()` and `is_marked_to_be_deleted()` methods to set and read the internal flag.
Datasets are by all means the most important openBIS entity. The actual files are stored as datasets; all other openBIS entities mainly are necessary to annotate and to structure the data:
Datasets are by all means the most important openBIS entity. The actual files are stored as datasets; all other openBIS entities mainly are necessary to annotate and to structure the data:
...
@@ -978,7 +978,7 @@ Datasets are by all means the most important openBIS entity. The actual files ar
...
@@ -978,7 +978,7 @@ Datasets are by all means the most important openBIS entity. The actual files ar
- sample / object
- sample / object
- dataset
- dataset
### working with existing dataSets
#### working with existing dataSets
**search for datasets**
**search for datasets**
...
@@ -1046,7 +1046,7 @@ ds.download_attachments(<path or cwd>) # Deprecated, as attachments are not com
...
@@ -1046,7 +1046,7 @@ ds.download_attachments(<path or cwd>) # Deprecated, as attachments are not com
# Attachments are an old concept and should not be used anymore.
# Attachments are an old concept and should not be used anymore.
```
```
### download dataSets
#### download dataSets
```
```
o.download_prefix # used for download() and symlink() method.
o.download_prefix # used for download() and symlink() method.
...
@@ -1066,7 +1066,7 @@ ds.download_path # returns the relative path (destination) of
...
@@ -1066,7 +1066,7 @@ ds.download_path # returns the relative path (destination) of
ds.is_physical() # TRUE if dataset is physically
ds.is_physical() # TRUE if dataset is physically
```
```
### link dataSets
#### link dataSets
Instead of downloading a dataSet, you can create a symbolic link to a dataSet in the openBIS dataStore. To do that, the openBIS dataStore needs to be mounted first (see mount method above). **Note:** Symbolic links and the mount() feature currently do not work with Windows.
Instead of downloading a dataSet, you can create a symbolic link to a dataSet in the openBIS dataStore. To do that, the openBIS dataStore needs to be mounted first (see mount method above). **Note:** Symbolic links and the mount() feature currently do not work with Windows.
...
@@ -1083,7 +1083,7 @@ ds.symlink(
...
@@ -1083,7 +1083,7 @@ ds.symlink(
ds.is_symlink()
ds.is_symlink()
```
```
### dataSet attributes and properties
#### dataSet attributes and properties
**Getting properties**
**Getting properties**
...
@@ -1115,7 +1115,7 @@ ds.p.set({'my_property':'value'}) # set the values of some properties
...
@@ -1115,7 +1115,7 @@ ds.p.set({'my_property':'value'}) # set the values of some properties
ds.set_props({ key: value }) # set the values of some properties
ds.set_props({ key: value }) # set the values of some properties
```
```
### search for dataSets
#### search for dataSets
- The result of a search is always list, even when no items are found
- The result of a search is always list, even when no items are found
- The `.df` attribute returns the Pandas dataFrame of the results
- The `.df` attribute returns the Pandas dataFrame of the results
create semantic annotation for sample type 'UNKNOWN':
create semantic annotation for sample type 'UNKNOWN':
...
@@ -1370,7 +1370,7 @@ sa.save()
...
@@ -1370,7 +1370,7 @@ sa.save()
sa.delete('reason')
sa.delete('reason')
```
```
## Tags
### Tags
```
```
new_tag = o.new_tag(
new_tag = o.new_tag(
...
@@ -1389,7 +1389,7 @@ tag.get_owner() # returns a person object
...
@@ -1389,7 +1389,7 @@ tag.get_owner() # returns a person object
tag.delete('why?')
tag.delete('why?')
```
```
## Vocabulary and VocabularyTerms
### Vocabulary and VocabularyTerms
An entity such as Sample (Object), Experiment (Collection), Material or DataSet can be of a specific _entity type_:
An entity such as Sample (Object), Experiment (Collection), Material or DataSet can be of a specific _entity type_:
...
@@ -1455,9 +1455,9 @@ term.save()
...
@@ -1455,9 +1455,9 @@ term.save()
term.delete()
term.delete()
```
```
## Change ELN Settings via pyBIS
### Change ELN Settings via pyBIS
### Main Menu
#### Main Menu
The ELN settings are stored as a **JSON string** in the `$eln_settings` property of the `GENERAL_ELN_SETTINGS` sample. You can show the **Main Menu settings** like this:
The ELN settings are stored as a **JSON string** in the `$eln_settings` property of the `GENERAL_ELN_SETTINGS` sample. You can show the **Main Menu settings** like this: