Skip to content
Snippets Groups Projects
Commit 99a82d16 authored by Marco Del Tufo's avatar Marco Del Tufo
Browse files

.

parent 52e5164b
No related branches found
No related tags found
1 merge request!40SSDM-13578 : 2PT : Database and V3 Implementation - include the new AFS "free"...
# Python (V3 API) - Welcome to pyBIS! # Python (V3 API) - pyBIS!
pyBIS is a Python module for interacting with openBIS. pyBIS is designed to be most useful in a [Jupyter Notebook](https://jupyter.org) or IPython environment, especially if you are developing Python scripts for automatisation. Jupyter Notebooks offer some sort of IDE for openBIS, supporting TAB completition and immediate data checks, making the life of a researcher hopefully easier. pyBIS is a Python module for interacting with openBIS. pyBIS is designed to be most useful in a [Jupyter Notebook](https://jupyter.org) or IPython environment, especially if you are developing Python scripts for automatisation. Jupyter Notebooks offer some sort of IDE for openBIS, supporting TAB completition and immediate data checks, making the life of a researcher hopefully easier.
...@@ -62,9 +62,9 @@ pip install jupyterlab ...@@ -62,9 +62,9 @@ pip install jupyterlab
- **property type:** a single property, as defined in the entity types above. It can be of a classic data type (e.g. INTEGER, VARCHAR, BOOLEAN) or its values can be controlled (CONTROLLEDVOCABULARY). - **property type:** a single property, as defined in the entity types above. It can be of a classic data type (e.g. INTEGER, VARCHAR, BOOLEAN) or its values can be controlled (CONTROLLEDVOCABULARY).
- **plugin:** a script written in [Jython](https://www.jython.org) which allows to check property values in a even more detailed fashion - **plugin:** a script written in [Jython](https://www.jython.org) which allows to check property values in a even more detailed fashion
# connect to OpenBIS ## connect to OpenBIS
## login ### login
In an **interactive session** e.g. inside a Jupyter notebook, you can use `getpass` to enter your password safely: In an **interactive session** e.g. inside a Jupyter notebook, you can use `getpass` to enter your password safely:
...@@ -215,7 +215,7 @@ Currently, mounting is supported for Linux and Mac OS X only. ...@@ -215,7 +215,7 @@ Currently, mounting is supported for Linux and Mac OS X only.
All attributes, if not provided, are re-used by a previous login() command. If no mountpoint is provided, the default mounpoint will be `~/hostname`. If this directory does not exist, it will be created. The directory must be empty before mounting. All attributes, if not provided, are re-used by a previous login() command. If no mountpoint is provided, the default mounpoint will be `~/hostname`. If this directory does not exist, it will be created. The directory must be empty before mounting.
# Masterdata ## Masterdata
OpenBIS stores quite a lot of meta-data along with your dataSets. The collection of data that describes this meta-data (i.e. meta-meta-data) is called masterdata. It consists of: OpenBIS stores quite a lot of meta-data along with your dataSets. The collection of data that describes this meta-data (i.e. meta-meta-data) is called masterdata. It consists of:
...@@ -230,7 +230,7 @@ OpenBIS stores quite a lot of meta-data along with your dataSets. The collection ...@@ -230,7 +230,7 @@ OpenBIS stores quite a lot of meta-data along with your dataSets. The collection
- tags - tags
- semantic annotations - semantic annotations
## browse masterdata ### browse masterdata
``` ```
sample_types = o.get_sample_types() # get a list of sample types sample_types = o.get_sample_types() # get a list of sample types
...@@ -263,7 +263,7 @@ o.get_terms(vocabulary='STORAGE') ...@@ -263,7 +263,7 @@ o.get_terms(vocabulary='STORAGE')
o.get_tags() o.get_tags()
``` ```
## create property types ### create property types
**Samples** (objects), **experiments** (collections) and **dataSets** contain type-specific **properties**. When you create a new sample, experiment or datasSet of a given type, the set of properties is well defined. Also, the values of these properties are being type-checked. **Samples** (objects), **experiments** (collections) and **dataSets** contain type-specific **properties**. When you create a new sample, experiment or datasSet of a given type, the set of properties is well defined. Also, the values of these properties are being type-checked.
...@@ -334,7 +334,7 @@ To create a **tabular, spreadsheet-like property**, use `XML` as `dataType` and ...@@ -334,7 +334,7 @@ To create a **tabular, spreadsheet-like property**, use `XML` as `dataType` and
**Note**: PropertyTypes that start with a \$ are by definition `managedInternally` and therefore this attribute must be set to True. **Note**: PropertyTypes that start with a \$ are by definition `managedInternally` and therefore this attribute must be set to True.
## create sample types / object types ### create sample types / object types
The second step (after creating a property type, see above) is to create the **sample type**. The new name for **sample** is **object**. You can use both methods interchangeably: The second step (after creating a property type, see above) is to create the **sample type**. The new name for **sample** is **object**. You can use both methods interchangeably:
...@@ -365,7 +365,7 @@ sample_type.get_next_code() # e.g. FLY77 ...@@ -365,7 +365,7 @@ sample_type.get_next_code() # e.g. FLY77
From pyBIS 1.31.0 onwards, you can provide a `code` even for samples where its sample type has `autoGeneratedCode=True` to offer the same functionality as ELN-LIMS. In earlier versions of pyBIS, providing a code in this situation caused an error. From pyBIS 1.31.0 onwards, you can provide a `code` even for samples where its sample type has `autoGeneratedCode=True` to offer the same functionality as ELN-LIMS. In earlier versions of pyBIS, providing a code in this situation caused an error.
## assign and revoke properties to sample type / object type ### assign and revoke properties to sample type / object type
The third step, after saving the sample type, is to **assign or revoke properties** to the newly created sample type. This assignment procedure applies to all entity types (dataset type, experiment type). The third step, after saving the sample type, is to **assign or revoke properties** to the newly created sample type. This assignment procedure applies to all entity types (dataset type, experiment type).
...@@ -383,7 +383,7 @@ sample_type.revoke_property('diff_time') ...@@ -383,7 +383,7 @@ sample_type.revoke_property('diff_time')
sample_type.get_property_assignments() sample_type.get_property_assignments()
``` ```
## create a dataset type ### create a dataset type
The second step (after creating a **property type**, see above) is to create the **dataset type**. The third step is to **assign or revoke the properties** to the newly created dataset type. The second step (after creating a **property type**, see above) is to create the **dataset type**. The third step is to **assign or revoke the properties** to the newly created dataset type.
...@@ -402,7 +402,7 @@ dataset_type.revoke_property('property_name') ...@@ -402,7 +402,7 @@ dataset_type.revoke_property('property_name')
dataset_type.get_property_assignments() dataset_type.get_property_assignments()
``` ```
## create an experiment type / collection type ### create an experiment type / collection type
The second step (after creating a **property type**, see above) is to create the **experiment type**. The second step (after creating a **property type**, see above) is to create the **experiment type**.
...@@ -422,7 +422,7 @@ experiment_type.revoke_property('property_name') ...@@ -422,7 +422,7 @@ experiment_type.revoke_property('property_name')
experiment_type.get_property_assignments() experiment_type.get_property_assignments()
``` ```
## create material types ### create material types
Materials and material types are deprecated in newer versions of openBIS. Materials and material types are deprecated in newer versions of openBIS.
...@@ -439,7 +439,7 @@ material_type.get_property_assignments() ...@@ -439,7 +439,7 @@ material_type.get_property_assignments()
``` ```
## create plugins ### create plugins
Plugins are Jython scripts that can accomplish more complex data-checks than ordinary types and vocabularies can achieve. They are assigned to entity types (dataset type, sample type etc). [Documentation and examples can be found here](https://wiki-bsse.ethz.ch/display/openBISDoc/Properties+Handled+By+Scripts) Plugins are Jython scripts that can accomplish more complex data-checks than ordinary types and vocabularies can achieve. They are assigned to entity types (dataset type, sample type etc). [Documentation and examples can be found here](https://wiki-bsse.ethz.ch/display/openBISDoc/Properties+Handled+By+Scripts)
...@@ -453,7 +453,7 @@ pl = o.new_plugin( ...@@ -453,7 +453,7 @@ pl = o.new_plugin(
pl.save() pl.save()
``` ```
## Users, Groups and RoleAssignments ### Users, Groups and RoleAssignments
Users can only login into the openBIS system when: Users can only login into the openBIS system when:
...@@ -496,7 +496,7 @@ ra = o.get_role_assignment(techId) ...@@ -496,7 +496,7 @@ ra = o.get_role_assignment(techId)
ra.delete() ra.delete()
``` ```
## Spaces ### Spaces
Spaces are fundamental way in openBIS to divide access between groups. Within a space, data can be easily shared. Between spaces, people need to be given specific access rights (see section above). The structure in openBIS is as follows: Spaces are fundamental way in openBIS to divide access between groups. Within a space, data can be easily shared. Between spaces, people need to be given specific access rights (see section above). The structure in openBIS is as follows:
...@@ -533,7 +533,7 @@ space.attrs.all() ...@@ -533,7 +533,7 @@ space.attrs.all()
space.delete('reason for deletion') space.delete('reason for deletion')
``` ```
## Projects ### Projects
Projects live within spaces and usually contain experiments (aka collections): Projects live within spaces and usually contain experiments (aka collections):
...@@ -589,7 +589,7 @@ project.freezeForExperiments = True ...@@ -589,7 +589,7 @@ project.freezeForExperiments = True
project.freezeForSamples = True project.freezeForSamples = True
``` ```
## Experiments / Collections ### Experiments / Collections
Experiments live within projects: Experiments live within projects:
...@@ -605,7 +605,7 @@ The new name for **experiment** is **collection**. You can use boths names inter ...@@ -605,7 +605,7 @@ The new name for **experiment** is **collection**. You can use boths names inter
- `new_experiment()` = `new_collection()` - `new_experiment()` = `new_collection()`
- `get_experiments()` = `get_collections()` - `get_experiments()` = `get_collections()`
### create a new experiment #### create a new experiment
``` ```
exp = o.new_experiment exp = o.new_experiment
...@@ -617,7 +617,7 @@ exp = o.new_experiment ...@@ -617,7 +617,7 @@ exp = o.new_experiment
exp.save() exp.save()
``` ```
### search for experiments #### search for experiments
``` ```
experiments = o.get_experiments( experiments = o.get_experiments(
...@@ -659,7 +659,7 @@ experiments = o.get_experiments( ...@@ -659,7 +659,7 @@ experiments = o.get_experiments(
``` ```
### Experiment attributes #### Experiment attributes
``` ```
exp.attrs.all() # returns all attributes as a dict exp.attrs.all() # returns all attributes as a dict
...@@ -682,7 +682,7 @@ exp.freezeForSamples = True ...@@ -682,7 +682,7 @@ exp.freezeForSamples = True
exp.save() # needed to save/update the changed attributes and properties exp.save() # needed to save/update the changed attributes and properties
``` ```
### Experiment properties #### Experiment properties
**Getting properties** **Getting properties**
...@@ -713,7 +713,7 @@ experiment.set_props({ key: value }) # set the values of some properties ...@@ -713,7 +713,7 @@ experiment.set_props({ key: value }) # set the values of some properties
experiment.save() # needed to save/update the changed attributes and properties experiment.save() # needed to save/update the changed attributes and properties
``` ```
## Samples / Objects ### Samples / Objects
Samples usually live within experiments/collections: Samples usually live within experiments/collections:
...@@ -778,7 +778,7 @@ sample.add_attachment('testfile.xls') # deprecated, see above ...@@ -778,7 +778,7 @@ sample.add_attachment('testfile.xls') # deprecated, see above
sample.delete('deleted for some reason') sample.delete('deleted for some reason')
``` ```
## create/update/delete many samples in a transaction ### create/update/delete many samples in a transaction
Creating a single sample takes some time. If you need to create many samples, you might want to create them in one transaction. This will transfer all your sample data at once. The Upside of this is the **gain in speed**. The downside: this is a **all-or-nothing** operation, which means, either all samples will be registered or none (if any error occurs). Creating a single sample takes some time. If you need to create many samples, you might want to create them in one transaction. This will transfer all your sample data at once. The Upside of this is the **gain in speed**. The downside: this is a **all-or-nothing** operation, which means, either all samples will be registered or none (if any error occurs).
...@@ -818,7 +818,7 @@ trans.commit() ...@@ -818,7 +818,7 @@ trans.commit()
**Note:** You can use the `mark_to_be_deleted()`, `unmark_to_be_deleted()` and `is_marked_to_be_deleted()` methods to set and read the internal flag. **Note:** You can use the `mark_to_be_deleted()`, `unmark_to_be_deleted()` and `is_marked_to_be_deleted()` methods to set and read the internal flag.
### parents, children, components and container #### parents, children, components and container
``` ```
sample.get_parents() sample.get_parents()
...@@ -848,7 +848,7 @@ sample.add_components('/MY_SPACE/COMPONENT_NAME') ...@@ -848,7 +848,7 @@ sample.add_components('/MY_SPACE/COMPONENT_NAME')
sample.del_components('/MY_SPACE/COMPONENT_NAME') sample.del_components('/MY_SPACE/COMPONENT_NAME')
``` ```
### sample tags #### sample tags
``` ```
sample.get_tags() sample.get_tags()
...@@ -857,7 +857,7 @@ sample.add_tags(['tag2','tag3']) ...@@ -857,7 +857,7 @@ sample.add_tags(['tag2','tag3'])
sample.del_tags('tag1') sample.del_tags('tag1')
``` ```
### Sample attributes and properties #### Sample attributes and properties
**Getting properties** **Getting properties**
...@@ -891,7 +891,7 @@ sample.set_props({ key: value }) # set the values of some properties ...@@ -891,7 +891,7 @@ sample.set_props({ key: value }) # set the values of some properties
sample.save() # needed to save/update the attributes and properties sample.save() # needed to save/update the attributes and properties
``` ```
### search for samples / objects #### search for samples / objects
The result of a search is always list, even when no items are found. The `.df` attribute returns The result of a search is always list, even when no items are found. The `.df` attribute returns
the Pandas dataFrame of the results. the Pandas dataFrame of the results.
...@@ -958,7 +958,7 @@ experiments = o.get_samples( ...@@ -958,7 +958,7 @@ experiments = o.get_samples(
``` ```
### freezing samples #### freezing samples
``` ```
sample.freeze = True sample.freeze = True
...@@ -968,7 +968,7 @@ sample.freezeForParents = True ...@@ -968,7 +968,7 @@ sample.freezeForParents = True
sample.freezeForDataSets = True sample.freezeForDataSets = True
``` ```
## Datasets ### Datasets
Datasets are by all means the most important openBIS entity. The actual files are stored as datasets; all other openBIS entities mainly are necessary to annotate and to structure the data: Datasets are by all means the most important openBIS entity. The actual files are stored as datasets; all other openBIS entities mainly are necessary to annotate and to structure the data:
...@@ -978,7 +978,7 @@ Datasets are by all means the most important openBIS entity. The actual files ar ...@@ -978,7 +978,7 @@ Datasets are by all means the most important openBIS entity. The actual files ar
- sample / object - sample / object
- dataset - dataset
### working with existing dataSets #### working with existing dataSets
**search for datasets** **search for datasets**
...@@ -1046,7 +1046,7 @@ ds.download_attachments(<path or cwd>) # Deprecated, as attachments are not com ...@@ -1046,7 +1046,7 @@ ds.download_attachments(<path or cwd>) # Deprecated, as attachments are not com
# Attachments are an old concept and should not be used anymore. # Attachments are an old concept and should not be used anymore.
``` ```
### download dataSets #### download dataSets
``` ```
o.download_prefix # used for download() and symlink() method. o.download_prefix # used for download() and symlink() method.
...@@ -1066,7 +1066,7 @@ ds.download_path # returns the relative path (destination) of ...@@ -1066,7 +1066,7 @@ ds.download_path # returns the relative path (destination) of
ds.is_physical() # TRUE if dataset is physically ds.is_physical() # TRUE if dataset is physically
``` ```
### link dataSets #### link dataSets
Instead of downloading a dataSet, you can create a symbolic link to a dataSet in the openBIS dataStore. To do that, the openBIS dataStore needs to be mounted first (see mount method above). **Note:** Symbolic links and the mount() feature currently do not work with Windows. Instead of downloading a dataSet, you can create a symbolic link to a dataSet in the openBIS dataStore. To do that, the openBIS dataStore needs to be mounted first (see mount method above). **Note:** Symbolic links and the mount() feature currently do not work with Windows.
...@@ -1083,7 +1083,7 @@ ds.symlink( ...@@ -1083,7 +1083,7 @@ ds.symlink(
ds.is_symlink() ds.is_symlink()
``` ```
### dataSet attributes and properties #### dataSet attributes and properties
**Getting properties** **Getting properties**
...@@ -1115,7 +1115,7 @@ ds.p.set({'my_property':'value'}) # set the values of some properties ...@@ -1115,7 +1115,7 @@ ds.p.set({'my_property':'value'}) # set the values of some properties
ds.set_props({ key: value }) # set the values of some properties ds.set_props({ key: value }) # set the values of some properties
``` ```
### search for dataSets #### search for dataSets
- The result of a search is always list, even when no items are found - The result of a search is always list, even when no items are found
- The `.df` attribute returns the Pandas dataFrame of the results - The `.df` attribute returns the Pandas dataFrame of the results
...@@ -1174,7 +1174,7 @@ datasets = o.get_experiment('/MY_NEW_SPACE/MY_PROJECT/MY_EXPERIMENT4')\ ...@@ -1174,7 +1174,7 @@ datasets = o.get_experiment('/MY_NEW_SPACE/MY_PROJECT/MY_EXPERIMENT4')\
.get_datasets(type='RAW_DATA') .get_datasets(type='RAW_DATA')
``` ```
### freeze dataSets #### freeze dataSets
- once a dataSet has been frozen, it cannot be changed by anyone anymore - once a dataSet has been frozen, it cannot be changed by anyone anymore
- so be careful! - so be careful!
...@@ -1188,7 +1188,7 @@ ds.freezeForContainers = True ...@@ -1188,7 +1188,7 @@ ds.freezeForContainers = True
ds.save() ds.save()
``` ```
### create a new dataSet #### create a new dataSet
``` ```
ds_new = o.new_dataset( ds_new = o.new_dataset(
...@@ -1201,7 +1201,7 @@ ds_new = o.new_dataset( ...@@ -1201,7 +1201,7 @@ ds_new = o.new_dataset(
ds_new.save() ds_new.save()
``` ```
### create dataSet with zipfile #### create dataSet with zipfile
DataSet containing one zipfile which will be unzipped in openBIS: DataSet containing one zipfile which will be unzipped in openBIS:
...@@ -1214,7 +1214,7 @@ ds_new = o.new_dataset( ...@@ -1214,7 +1214,7 @@ ds_new = o.new_dataset(
ds_new.save() ds_new.save()
``` ```
### create dataSet with mixed content #### create dataSet with mixed content
- mixed content means: folders and files are provided - mixed content means: folders and files are provided
- a relative specified folder (and all its content) will end up in the root, while keeping its structure - a relative specified folder (and all its content) will end up in the root, while keeping its structure
...@@ -1243,7 +1243,7 @@ ds_new = o.new_dataset( ...@@ -1243,7 +1243,7 @@ ds_new = o.new_dataset(
ds_new.save() ds_new.save()
``` ```
### create dataSet container #### create dataSet container
A DataSet of kind=CONTAINER contains other DataSets, but no files: A DataSet of kind=CONTAINER contains other DataSets, but no files:
...@@ -1258,7 +1258,7 @@ ds_new = o.new_dataset( ...@@ -1258,7 +1258,7 @@ ds_new = o.new_dataset(
ds_new.save() ds_new.save()
``` ```
### get, set, add and remove parent datasets #### get, set, add and remove parent datasets
``` ```
dataset.get_parents() dataset.get_parents()
...@@ -1276,7 +1276,7 @@ dataset.add_children(['20170115220259155-412']) ...@@ -1276,7 +1276,7 @@ dataset.add_children(['20170115220259155-412'])
dataset.del_children(['20170115220259155-412']) dataset.del_children(['20170115220259155-412'])
``` ```
### dataSet containers #### dataSet containers
- A DataSet may belong to other DataSets, which must be of kind=CONTAINER - A DataSet may belong to other DataSets, which must be of kind=CONTAINER
- As opposed to Samples, DataSets may belong (contained) to more than one DataSet-container - As opposed to Samples, DataSets may belong (contained) to more than one DataSet-container
...@@ -1301,7 +1301,7 @@ dataset.add_components(['20170115220259155-412']) ...@@ -1301,7 +1301,7 @@ dataset.add_components(['20170115220259155-412'])
dataset.del_components(['20170115220259155-412']) dataset.del_components(['20170115220259155-412'])
``` ```
## Semantic Annotations ### Semantic Annotations
create semantic annotation for sample type 'UNKNOWN': create semantic annotation for sample type 'UNKNOWN':
...@@ -1370,7 +1370,7 @@ sa.save() ...@@ -1370,7 +1370,7 @@ sa.save()
sa.delete('reason') sa.delete('reason')
``` ```
## Tags ### Tags
``` ```
new_tag = o.new_tag( new_tag = o.new_tag(
...@@ -1389,7 +1389,7 @@ tag.get_owner() # returns a person object ...@@ -1389,7 +1389,7 @@ tag.get_owner() # returns a person object
tag.delete('why?') tag.delete('why?')
``` ```
## Vocabulary and VocabularyTerms ### Vocabulary and VocabularyTerms
An entity such as Sample (Object), Experiment (Collection), Material or DataSet can be of a specific _entity type_: An entity such as Sample (Object), Experiment (Collection), Material or DataSet can be of a specific _entity type_:
...@@ -1455,9 +1455,9 @@ term.save() ...@@ -1455,9 +1455,9 @@ term.save()
term.delete() term.delete()
``` ```
## Change ELN Settings via pyBIS ### Change ELN Settings via pyBIS
### Main Menu #### Main Menu
The ELN settings are stored as a **JSON string** in the `$eln_settings` property of the `GENERAL_ELN_SETTINGS` sample. You can show the **Main Menu settings** like this: The ELN settings are stored as a **JSON string** in the `$eln_settings` property of the `GENERAL_ELN_SETTINGS` sample. You can show the **Main Menu settings** like this:
...@@ -1491,7 +1491,7 @@ settings_sample.props['$eln_settings'] = json.dumps(settings) ...@@ -1491,7 +1491,7 @@ settings_sample.props['$eln_settings'] = json.dumps(settings)
settings_sample.save() settings_sample.save()
``` ```
### Storages #### Storages
The **ELN storages settings** can be found in the samples of project `/ELN_SETTINGS/STORAGES` The **ELN storages settings** can be found in the samples of project `/ELN_SETTINGS/STORAGES`
...@@ -1517,7 +1517,7 @@ sto.props() ...@@ -1517,7 +1517,7 @@ sto.props()
sto.save() sto.save()
``` ```
### Templates #### Templates
The **ELN templates settings** can be found in the samples of project `/ELN_SETTINGS/TEMPLATES` The **ELN templates settings** can be found in the samples of project `/ELN_SETTINGS/TEMPLATES`
...@@ -1527,7 +1527,7 @@ o.get_samples(project='/ELN_SETTINGS/TEMPLATES') ...@@ -1527,7 +1527,7 @@ o.get_samples(project='/ELN_SETTINGS/TEMPLATES')
To change the settings, use the same technique as shown above with the storages settings. To change the settings, use the same technique as shown above with the storages settings.
### Custom Widgets #### Custom Widgets
To change the **Custom Widgets settings**, get the `property_type` and set the `metaData` attribute: To change the **Custom Widgets settings**, get the `property_type` and set the `metaData` attribute:
...@@ -1540,4 +1540,4 @@ pt.save() ...@@ -1540,4 +1540,4 @@ pt.save()
Currently, the value of the `custom_widget` key can be set to either Currently, the value of the `custom_widget` key can be set to either
- `Spreadsheet` (for tabular, Excel-like data) - `Spreadsheet` (for tabular, Excel-like data)
- `Word Processor` (for rich text data) - `Word Processor` (for rich text data)
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment