diff --git a/README.md b/README.md
index 72ee7f8206a261272aa4edbaaac315e2a0d94a17..2988e5d682d2e9df26ea3a1787dd617db8a21a6b 100644
--- a/README.md
+++ b/README.md
@@ -1,21 +1,99 @@
# Jupyter-OpenBIS-Extension
-## How to install the extension
+## Requirements
+
+The jupyter-openbis-extension needs at least **Python 3.3** in order to run. The Jupyter notebook server starts this extension during startup and must therefore run under the same Python version. The kernel can be anything (Python 2.7, Julia, R, Perl...)
+
+## Install the extension
+
+If you haven't done yet: download the latest jupyter-openbis-extension from pip. It will automatically install the various dependencies, e.g. Pandas and NumPy.
+
+```
+pip install --upgrade jupyter-openbis-extension
+```
+
+**Register the Jupyter server extension** which will communicate both with openBIS and the notebook:
+
+```
+jupyter serverextension enable --py jupyter-openbis-extension
+```
+
+**Register the notebook extension**, the client side of the Jupyter notebook, which will communicate with the server extension via AJAX calls:
+
+```
+ jupyter nbextension install --py jupyter-openbis-extension --user
+```
+If you are developing, you should consider creating a link to the JavaScript source instead:
+
+```
+ jupyter nbextension install --py jupyter-openbis-extension --user --symlink
+```
+
+Finally, activate the notebook extension.
+
+```
+jupyter nbextension enable jupyter-openbis-extension --user --py
+```
+
+**Create a configuration file**: `~/.jupyter/openbis-connections.yaml` It should contain connection information to your server(s), for example:
+
+```
+connections:
+ - name : TEST local openBIS instance
+ url : https://localhost:8443
+ verify_certificates : false
+ username : username
+ password : password
+ - name : PRODUCTION openBIS instance
+ url : https://openbis.example.com
+ verify_certificates : true
+ username : username
+ password : password
+```
+
+Now you are **ready to launch!**
+
+```
+$ jupyter notebook
+```
+Observe the terminal. It should tell you that the server(s) have been successfully connected:
+
+```
+$ jupyter notebook
+connected to TEST local openBIS instance
+connected to PRODUCTION openBIS instance
+```
+**Congratulations!**
+
+## Uninstall Jupyter extension
+
+```
+jupyter serverextension disable --py jupyter-openbis-extension
+jupyter nbextension disable --py jupyter-openbis-extension --user
+jupyter nbextension uninstall --py jupyter-openbis-extension --user
+```
+This should remove the registrations in `~/.jupyter/jupyter_notebook_config.json` and `~/.jupyter/nbconfig/notebook.json`. The symbolic link in `~/Library/Jupyter/nbextension/` (Mac OS X) should have been removed too. If not, you can remove it manually.
+
+
+
+## Manual installation (if above should fail)
### Install the Python module(s)
+- clone the project:
+```
+git clone git@sissource.ethz.ch:sispub/jupyter-openbis-extension.git
+```
- cd into the extension `cd jupyter-openbis-extension`
- install the python module normally: `pip install .`
- or, for development, install just a symbolic link: `pip install -e .`
-- this extension needs Jupyter and pyBIS in order to run, so it will load it if not yet present
-- be aware that, because pyBIS needs at least Python 3.3, your Jupyter needs to run under the
- same version of Python.
-
+- this extension needs Jupyter and pyBIS 1.7.2 in order to run, so it will load / upgrade it if not yet present
+- the jupyter-openbis-extension needs at least Python 3.3 in order to run. Therefore, your Jupyter notebook server (but not the kernel!) needs to run under at least Python 3.3 too.
### Register the Jupyter Notebook Server Extension (server-side, as the name suggests)
- in the terminal, enter the following:
``
-jupyter serverextension enable --py jupyter-openbis-extension.server
+jupyter serverextension enable --py jupyter-openbis-extension
``
- OR (1) register it manually:
@@ -33,7 +111,6 @@ c.NotebookApp.server_extensions = [
{
"NotebookApp": {
"nbserver_extensions": {
- "jupyter_nbextensions_configurator": true,
"jupyter-openbis-extension.server": true
}
}
@@ -41,83 +118,28 @@ c.NotebookApp.server_extensions = [
```
### Register the Jupyter Notebook Extension (client-side)
-- TODO
-
-
-## General thoughts (unordered)
-
-- need easy way to install the extension, i.e. just a pip install for both Python and Javascript
-- centralised infrastructure which runs Jupyter notebooks (or labs)
-- possibility to run your notebooks offline
-- every step within the notebook should be reproducible
-- notebook should be self-contained, which means all meta-information should be stored within the notebook
-- notebook should not be cluttered with meta-information
-- meta-information is already part of the notebook, contains mostly kernel information
-- metadata can be extended
-- downloading datasets from openBIS should be possible from within a menu inside Jupyter
-- with every download, the metadata inside the notebook should be extended
-- metadata in future should contain: Kernel-version, Packages-versions, Datasets that have been downloaded
-- all notebooks created by members of the lab should be set up in a standardised way, without troubling the user
-- pulling information is always better than pushing
-- current implementation of ELN-LIMS and Jupyter does not take care about different kernels, e.g R, Julia, C, etc.
-
-
-## nice-to-have example workflow
-- user logs into Jupyterhub
-- creates a new notebook (with any kernel)
-- opens a submenu «openBIS integration» and specifies a database, providing server, username, password
-- the notebook server (which is always running in Python) will open a new openBIS connection, the password is not stored
-- in another submenu, the user specifies the dataset(s) he/she wants to download and where to store them locally
-- in the background, these datasets are loaded
-- the metainformation of the notebook is automatically extended with this information
-- the user now does the analysis, producing
- - code (in whatever language the notebook runs)
- - Markdown-text
- - output files
-- every time the user saves the notebook, a new git version is commited (using a post-save hook)
-- when the user wants to **release** his results:
- - he specifies the openBIS connection the release should be published
- - he specifies the parent dataset and some additional information to create a new dataset
- - he specifies all necessary datasets which were needed to do the analysis (default: all downloaded datasets)
- - he specifies which cell outputs to keep (or all/none)
- - he specifies which output files to include
- - he specifies whether he wants to create an html/pdf/rst/.py version of the notebook
- - the Jupyter-kernel is asked about its environment, including all package versions
- - the meta-information is updated, all files are sent to openBIS as a new dataset
-
-
-## what's going wrong with the current development in ELN-LIMS
-
-- Jupyter should not be seen as part of openBIS and should not be «integrated» in any way
-- pushing data directly into a users' home folder is very problematic
-- pulling data is almost always a better idea than pushing data
-- ELN-LIMS currently only creates Pytho 3.6 notebooks.
-- Users should be allowed to use any kernel they want.
-- having a URL provided by ELN-LIMS which downloads a notebook template that includes the download of a dataset – would make more sense
-- ELN-LIMS should provide a quick overview of what's inside openBIS
-- browsing and filtering Experiments/Samples/Datasets is a nightmare and not as good as the classic GUI
-
-
-## next steps
-- create a proof-of-concept to show the functionality of a notebook-extension
-- show how meta-information of the notebook can be extended automatically
-
-## first productive implementation
-
-Implement a notebook extension which allows the following workflow:
-1. push a button to open up a Openbis search-field
-2. enter a sample code or permId of a dataset
-3. display the datasets which belong to that sample as a list
-4. tick the datasets you want to download
-5. click on a «download dataset(s)» button to start the download in the background
-6. wait until the confirmation message(s) appear
-7. after analysis, push another button to register the notebook in openBIS
-8. make sure the notebook is saved first
-9. specify a DataSet parent or Sample and include some metadata
-10. upload Notebook to openBIS
-
-**Time estimate**: About 8 days (1-6: 5d, 7-10: 3d)
-
-## open questions (from Andreas Cuny, in German)
-1. wenn der user Daten von irgendwoher importiert sagen wir Desktop. Werden die dann auch hochgeladen? Oder nur was in dem folder ist wo das notebook drin liegt?
-2. wenn diese Daten vom eg Desktop eigentlich Daten von OpenBis sind aber irgendwann mal runtergeladen wurden (ev nicht aktuell, Risiko von Daten dublikation wenn als dataset mit hochgeladen wuerden. Idealerweise müssten importierte Daten welche vom Desktop kommen aber noch nicht in openbis sind via pybis als sample erst registiert werden oder manuell via ui.
+
+Inside the folder `~/Library/Jupyter/nbextension/` (Mac OS X) create a symbolic **link to the folder that contains the Java Script**:
+
+```
+ln -s /path/to/jupyter-openbis-extension/nbextension/dialog.js jupyter-openbis-extension
+```
+In the file `~/.jupyter/nbconfig/notebook.json` the JavaScript gets activated like this:
+
+```
+{
+ "load_extensions": {
+ "jupyter-openbis-extension/dialog": true
+ }
+}
+```
+**Check** that the nbextension has been installed and activated:
+
+```
+$ jupyter nbextension list
+Known nbextensions:
+ config dir: /Users/your-username/.jupyter/nbconfig
+ notebook section
+ jupyter-openbis-extension/dialog enabled
+ - Validating: OK
+```
\ No newline at end of file
diff --git a/howto-implement-jupyter-extension.md b/howto-implement-jupyter-extension.md
deleted file mode 100644
index 4fc865b1cba5901da9d616dce7eecf1789a320a4..0000000000000000000000000000000000000000
--- a/howto-implement-jupyter-extension.md
+++ /dev/null
@@ -1,296 +0,0 @@
-# How to implement the Jupyter-Openbis extension
-
-## How the functionality of Jupyter notebooks is extended.
-
-There are four ways to extend a Jupyter notebook (see also [Parente's Mindtrove: Four Ways to Extend Jupyter Notebook](http://mindtrove.info/4-ways-to-extend-jupyter-notebook/) ):
-
-1. Kernels
-2. IPython Kernel Extensions
-3. Notebook Extensions
-4. Notebook Server Extensions
-
-**Kernels** are used to run your notebook in a specific environment, be it a specific language or a special running environment, e.g. on a cluster.
-
-**Kernel Extensions** allow to modify your kernel environment within a cell. They often introduce some *Magics*, e.g. `%run_with_extra_horsepower <my_function>`
-
-**Notebook Extensions** improve or alter the GUI / Webinterface in order to add new functionality, a new shortcut, a special button, etc. They are written in JavaScript. Some of them interact with the Notebook server via a *Notebook Server Extension* (see next).
-
-**Notebook Server Extensions** run as part of the Jupyter notebook server and as such, they are written in the same language as the server. Which is – depending on your installation – either Python 2.7 or Python 3.3 or greater. Notebook Server Extensions can listen to request sent by the user via a *Notebook Extension* (see previous).
-
-For our Jupyter-Openbis extension we need both Notebook Extension and Notebook Server Extension. We start with the latter.
-
-## Implementing a Notebook Server Extension
-
-The Jupyter notebook server includes a [tornado webserver](http://www.tornadoweb.org/en/stable/) which will talk to the GUI.
-
-Extensions need to implement a method called `load_jupyter_server_extension` which will provide an instance of the running tornado Application. We then easily can add new handlers, which means, we can set up new commmunication channels between the Web frontend and the backend. In our case, the channel has the name `/openbis/dataset`.
-
-Here is our script `pybis.jupyter_extension.py`:
-
-```
-from notebook.utils import url_path_join
-from notebook.base.handlers import IPythonHandler
-from pybis import Openbis
-
-o = Openbis(url='https://sprint-openbis.ethz.ch')
-o.login('username', '***password****')
-
-import json
-import os
-import urllib
-from subprocess import check_output
-from urllib.parse import urlsplit, urlunsplit
-
-
-def _jupyter_server_extension_paths():
- return [{'module': 'gitlab_commit_push'}]
-
-
-def load_jupyter_server_extension(nb_server_app):
- """Call when the extension is loaded.
-
- :param nb_server_app: Handle to the Notebook webserver instance.
- """
- web_app = nb_server_app.web_app
- host_pattern = '.*$'
-
- route_pattern = url_path_join(web_app.settings['base_url'], '/openbis/dataset')
- web_app.add_handlers(host_pattern, [(route_pattern, DataSetHandler)])
- print("pybis loaded: {}".format(Openbis))
-
-
-class DataSetHandler(IPythonHandler):
- """Download the requested DataSet"""
-
- def put(self):
- """Handle a user PUT request."""
-
- permId = self.get_json_body()
-
- try:
- ds = o.get_dataset(permId)
- except ValueError:
- self.write({
- 'status':
- 404,
- 'statusText':
- 'DataSet {} could not be found.'.format(permId)
- })
- return
-
- try:
- destination = ds.download()
- except Exception as e:
- self.write({
- 'status':
- 404,
- 'statusText':
- 'DataSet {} could not be downloaded: {}'.format(permId, e)
- })
- return
-
- path = os.path.join(destination, ds.permId)
-
- self.write({
- 'status':
- 200,
- 'permId': ds.permId,
- 'statusText':
- 'DataSet was successfully downloaded to {}.'.format(path)
- })
-```
-
-## Register Notebook Server Extension
-
-In order to let Jupyter know that it needs to run an extension, we have to register it in a file called `~/.jupyter/jupyter_notebook_config.py` like this:
-
-```
-c.NotebookApp.server_extensions = [
- 'pybis.jupyter_extension'
-]
-```
-See [a very basic example here](http://mindtrove.info/4-ways-to-extend-jupyter-notebook/#nb-server-exts) to demonstrate the concept.
-
-
-## Implementing a Notebook Extension
-
-As mentioned before, Notebook Extensions are written in JavaScript. Much like the Notebook Server Extensions, they need to implement a special method: `load_ipython_extension`. See [here for a basic Javascript example](http://mindtrove.info/4-ways-to-extend-jupyter-notebook/#nb-extensions).
-
-In our case, we want to offer an input box, which takes a **permId** of a Dataset and sends this information to the already implemented Notebook Server Extension via the communication channel `/openbis/dataset``. After the download succeeded, our script then receives some confirmation message, including a statusText for user feedback as well as the permId, which is stored within the metadata of the notebook itself in order to keep track of all datasets downloaded.
-
-Here is the content of the script `jupyter-openbis-extension/main.js`:
-
-```
-define([
- "base/js/namespace",
- "base/js/dialog",
- "base/js/utils",
- "jquery"
-], function(IPython, dialog, utils, $, mc) {
- var fetchDatasetFromOpenBis = {
- help: 'Download a openBIS dataset to your local harddrive',
- icon: 'fa-database',
- help_index: '',
- handler: function (env) {
- var p = $('<p/>').text('Please enter a DataSet identifier or permId to load from sprint-openbis.ethz.ch')
- var input = $('<textarea rows="4" cols="72"></textarea>')
- var div = $('<div/>')
- var commitUrl = env.notebook.base_url + 'openbis/dataset'
-
- div.append(p).append(input)
-
- // get the canvas for user feedback
- var container = $('#notebook-container')
-
- function onOk () {
- var notebook = IPython.notebook
- var re = /\/notebooks(.*?)$/
- var filepath = window.location.pathname.match(re)[1]
- //var payload = {'filename': filepath, 'msg': input.val()}
- var settings = {
- url: commitUrl,
- processData: false,
- type: 'PUT',
- dataType: 'json',
- //data: JSON.stringify(payload),
- data: JSON.stringify(input.val()),
- contentType: 'application/json',
- success: function (data) {
- // display feedback to user
- var container = $('#notebook-container')
- var feedback = '<div class="openbis-feedback alert alert-success alert-dismissible" role="alert">' +
- '<button type="button" class="close" data-dismiss="alert" aria-label="Close">' +
- '<span aria-hidden="true">×</span></button>' +
- data.statusText +
- '</div>'
-
- // display feedback
- $('.openbis-feedback').remove()
- container.prepend(feedback)
-
- // write statusText from returned data to notebooks metadata
- if ( typeof notebook.metadata.openbis === 'undefined') {
- notebook.metadata.openbis = {}
- }
- if ( typeof notebook.metadata.openbis.permIds === 'undefined' ) {
- notebook.metadata.openbis.permIds = {}
- }
- if ( data.permId ) {
- notebook.metadata.openbis.permIds[data.permId] = data.statusText
- }
- },
- error: function (data) {
- // display feedback to user
- var feedback =
- '<div class="openbis-feedback alert alert-danger alert-dismissible" role="alert">' +
- '<button type="button" class="close" data-dismiss="alert" aria-label="Close"><span aria-hidden="true">×</span></button>' +
- '<strong>Warning!</strong> Something went wrong.<div>' + data.statusText + '</div></div>'
-
- // display feedback
- $('.openbis-feedback').remove()
- container.prepend(feedback)
- }
- }
- //console.log(IPython.notebook)
-
- // display preloader during commit and push
- var preloader = '<img class="openbis-feedback" src="https://cdnjs.cloudflare.com/ajax/libs/slick-carousel/1.5.8/ajax-loader.gif">'
- container.prepend(preloader)
- $('.openbis-feedback').remove()
-
- // commit and push
- utils.ajax(settings)
- }
-
- if (IPython.notebook.dirty === true) {
- dialog.modal({
- body: 'Please, save the notebook before sharing.',
- title: 'Action required',
- buttons: {
- 'Back': {}
- },
- notebook: env.notebook,
- keyboard_manager: env.notebook.keyboard_manager
- })
- }
- else {
- dialog.modal({
- body: div,
- title: 'Download openBIS DataSet',
- buttons: {
- 'Download': {
- class: 'btn-primary btn-large',
- click: onOk
- },
- 'Cancel': {}
- },
- notebook: env.notebook,
- keyboard_manager: env.notebook.keyboard_manager
- })
- }
- }
- }
-
- function _onLoad () {
- // log to console
- console.info('Loaded Jupyter extension: openbis-jupyter-extension')
- // register new action
- var actionName = IPython.keyboard_manager.actions.register(
- fetchDatasetFromOpenBis, 'openbis-dataset-fetch', 'jupyter-gitlab')
-
- console.info(actionName)
- // add button for new action
- IPython.toolbar.add_buttons_group([actionName])
- }
-
- return {load_ipython_extension: _onLoad}
-})
-```
-
-## Register the Notebook Extension
-
-The Jupyter configuration which indicates which (JavaScript) Notebook Extension should be loaded is stored in this file: `.jupyter/nbconfig/notebook.json`. To maintain this file, Jupyter comes with some commands which will make things a bit easier.
-
-### a) install the extension
-
-```
-jupyter nbextension install /path/to/jupyter-openbis-extension --user --symlink
-```
-
-will create a symlink of our jupyter-openbis-extension folder containing `main.js` (and maybe more files) in its destination, which is (on a Mac): `~/Library/Jupyter/nbextensions/`. This is the recommended setup if you are still developing. If you omit the `--symlink` flag, your files will be *copied* instead. If you omit the `--user` flag, your extension will be installed system-wide. There are many more flags, type
-
-```
-jupyter nbextension install --help
-```
-to list them all.
-
-### b) register the extension
-
-`jupyter nbextension list` lists all extensions already installed. After you installed your extension, it needs to be enabled and validated:
-
-```
-jupyter nbextension enable jupyter-openbis-extension/main
-```
-
-This is Require.js - Style: You need to omit the `.js` extension of the entry-point of your Javascript file. If you omit `/main` altogether, the validation will fail:
-
-```
-$ jupyter nbextension enable jupyter-openbis-extension
-Enabling notebook extension jupyter-openbis-extension...
- - Validating: problems found:
- - require? X jupyter-openbis-extension
-```
-This is because Require.js does not find its entry point script (`main.js`) automatically. To remove the error, edit `~/.jupyter/nbconfig/notebook.js` and remove the faulty line. Watch out not to leave a trailing comma `,` in the list, it will produce yet another weird error when starting Jupyter.
-
-Once the extension is correctly installed, the output of `jupyter nbextension list` should look something like this:
-
-```
-$ jupyter nbextension list
-Known nbextensions:
- config dir: /Users/username/.jupyter/nbconfig
- notebook section
- jupyter-openbis-extension/main enabled
- - Validating: OK
-```
-
-Congratulations!
\ No newline at end of file
diff --git a/jupyter-openbis-extension/.server.py.swp b/jupyter-openbis-extension/.server.py.swp
deleted file mode 100644
index c294c69447df13190139740612a621e8cc044143..0000000000000000000000000000000000000000
Binary files a/jupyter-openbis-extension/.server.py.swp and /dev/null differ
diff --git a/jupyter-openbis-extension/__init__.py b/jupyter-openbis-extension/__init__.py
index 9fd8cd106ac6ffe3cc65d6f58d015f1d4696d482..37aaff17bd684cb440495db5a08089b53c52773a 100644
--- a/jupyter-openbis-extension/__init__.py
+++ b/jupyter-openbis-extension/__init__.py
@@ -12,9 +12,9 @@ def _jupyter_nbextension_paths():
# # the path relative to the `jupyter-openbis-extension` directory containing the JavaScript
'src': "nbextension",
# # directory in the `nbextension/` namespace
- 'dest': "jupyter-openbis",
+ 'dest': "jupyter-openbis-extension",
# _also_ in the `nbextension/` namespace
- 'require' : "jupyter-openbis/extension"
+ 'require' : "jupyter-openbis-extension/dialog"
}]
def load_jupyter_server_extension(nbapp):
diff --git a/jupyter-openbis-extension/__pycache__/__init__.cpython-36.pyc b/jupyter-openbis-extension/__pycache__/__init__.cpython-36.pyc
deleted file mode 100644
index 17c85d6c830fc23516d571d3b9fbc90fd636a634..0000000000000000000000000000000000000000
Binary files a/jupyter-openbis-extension/__pycache__/__init__.cpython-36.pyc and /dev/null differ
diff --git a/jupyter-openbis-extension/__pycache__/server.cpython-36.pyc b/jupyter-openbis-extension/__pycache__/server.cpython-36.pyc
deleted file mode 100644
index 0a1e17fe0c786f4007391b5c58e6c6c99190b531..0000000000000000000000000000000000000000
Binary files a/jupyter-openbis-extension/__pycache__/server.cpython-36.pyc and /dev/null differ
diff --git a/nbextension/openbis/main.js b/jupyter-openbis-extension/nbextension/dialog.js
similarity index 98%
rename from nbextension/openbis/main.js
rename to jupyter-openbis-extension/nbextension/dialog.js
index c6bf7738ac775ec85f9c41361a7cf9505f0af90b..79ef32eecc71211ae7d9c2f26c10beed0b75e197 100644
--- a/nbextension/openbis/main.js
+++ b/jupyter-openbis-extension/nbextension/dialog.js
@@ -186,19 +186,16 @@ define([
processData: false,
type: 'GET',
dataType: 'json',
- contentType: 'application/json',
- success: function(data) {
- show_datasets_table(data, datasets_table)
- },
- error: function(data) {
- console.log(data);
-
- console.log(data.statusText);
- alert('Error: ' + data.statusText)
- }
-
+ contentType: 'application/json'
}
+
utils.ajax(settings)
+ .done(function(data) {
+ show_datasets_table(data, datasets_table)
+ })
+ .fail(function(data) {
+ alert('Error:' + data.statusText)
+ })
})
sample.append(sampleIdentifier).append(show_datasets_btn).append(datasets_table)
diff --git a/jupyter-openbis-extension/server.py b/jupyter-openbis-extension/server.py
index 8a6781c7fc2b90b8518cfc8b2ce59e4c405934cc..c1d72c54495dea2d2b1190b2234b19df7264381e 100644
--- a/jupyter-openbis-extension/server.py
+++ b/jupyter-openbis-extension/server.py
@@ -1,10 +1,12 @@
from notebook.utils import url_path_join
from notebook.base.handlers import IPythonHandler
from pybis import Openbis
+import numpy as np
import json
import os
import urllib
+from urllib.parse import unquote, unquote_plus
from subprocess import check_output
from urllib.parse import urlsplit, urlunsplit
import yaml
@@ -53,24 +55,36 @@ def load_jupyter_server_extension(nb_server_app):
host_pattern,
[(url_path_join(
base_url,
- '/openbis/dataset/(?P<connection_name>.*)?/(?P<permId>.*)'),
- DataSetHandler
+ '/openbis/dataset/(?P<connection_name>.*)?/(?P<permId>.*)?/(?P<downloadPath>.*)'),
+ DataSetDownloadHandler
+ )]
+ )
+ web_app.add_handlers( host_pattern, [(
+ url_path_join(
+ base_url,
+ '/openbis/dataset/(?P<connection_name>.*)'
+ ),
+ DataSetUploadHandler
)]
)
web_app.add_handlers(
host_pattern,
- [(url_path_join(
- base_url,
- '/openbis/sample/(?P<connection_name>.*)?/(?P<permId>.*)'),
+ [(
+ url_path_join(
+ base_url,
+ '/openbis/sample/(?P<connection_name>.*)?/(?P<permId>.*)'
+ ),
SampleHandler
)]
)
web_app.add_handlers(
host_pattern,
- [(url_path_join(base_url, '/openbis/conn'), OpenBISHandler)]
+ [(
+ url_path_join(base_url, '/openbis/conn'),
+ OpenBISHandler
+ )]
)
-
print("pybis loaded: {}".format(Openbis))
@@ -86,20 +100,23 @@ def register_connection(connection_info):
)
openbis_connections[conn.name] = conn
- openbis = Openbis(
- url = conn.url,
- verify_certificates = conn.verify_certificates
- )
- conn.openbis = openbis
-
try:
+ openbis = Openbis(
+ url = conn.url,
+ verify_certificates = conn.verify_certificates
+ )
+ conn.openbis = openbis
+
openbis.login(
username = conn.username,
password = conn.password
)
conn.status = 'connected'
+ print('connected to {}'.format(conn.name))
except Exception as exc:
conn.status = 'FAILED: {}'.format(exc)
+ print('ERROR: could not connected to {}. Reason: {}'.format(conn.name, exc))
+
raise exc
@@ -128,7 +145,7 @@ class OpenBISConnection:
return self.openbis.is_session_active()
def reconnect(self):
- openbis.login(self.username, self.password)
+ self.openbis.login(self.username, self.password)
class OpenBISHandler(IPythonHandler):
@@ -141,12 +158,15 @@ class OpenBISHandler(IPythonHandler):
connections= []
for conn in openbis_connections.values():
connections.append({
- 'name': conn.name,
- 'url' : conn.url
+ 'name' : conn.name,
+ 'url' : conn.url,
+ 'status' : conn.status,
})
self.write({
- 'connections': connections
+ 'status' : 200,
+ 'connections': connections,
+ 'cwd' : os.getcwd()
})
return
@@ -154,19 +174,26 @@ class SampleHandler(IPythonHandler):
"""Handle the requests for /openbis/sample/connection/permId"""
def get_datasets(self, conn, permId):
+ if not conn.is_session_active():
+ try:
+ conn.reconnect()
+ except Exception as exc:
+ print(exc)
+ self.send_error(
+ reason = 'connection to {} could not be established: {}'.format(conn.name, exc)
+ )
+
sample = None
try:
sample = conn.openbis.get_sample(permId)
- except Exception:
- self.send_error(
- reason = 'No such sample found: {}'.format(permId)
- )
- return
+ except Exception as exc:
+ print(exc)
+ self.send_error( reason = 'No such sample found: {}'.format(permId) )
datasets = sample.get_datasets().df
- dataset_records = datasets.to_dict(orient='records')
- self.write({'dataSets': dataset_records })
- return
+ datasets.replace({np.nan:None}, inplace=True) # replace NaN with None, otherwise we cannot convert it correctly
+ return datasets.to_dict(orient='records') # is too stupid to handle NaN
+
def get(self, **params):
"""Handle a request to /openbis/sample/connection_name/permId
@@ -178,16 +205,21 @@ class SampleHandler(IPythonHandler):
self.send_error(
reason = 'connection {} was not found'.format(params['connection_name'])
)
- return
- results = self.get_datasets(conn, params['permId'])
+ datasets = self.get_datasets(conn, params['permId'])
+ if datasets is not None:
+ self.set_status(200)
+ self.write({
+ "dataSets": datasets
+ })
-class DataSetHandler(IPythonHandler):
+
+class DataSetDownloadHandler(IPythonHandler):
"""Handle the requests for /openbis/dataset/connection/permId"""
- def download_data(self, conn, permId):
+ def download_data(self, conn, permId, downloadPath=None):
if not conn.is_session_active():
try:
conn.reconnect()
@@ -207,25 +239,29 @@ class DataSetHandler(IPythonHandler):
# dataset was found, download the data to the disk
try:
- destination = dataset.download()
+ destination = dataset.download(destination=downloadPath)
except Exception as exc:
- print(exc)
self.send_error(
reason = 'Data for DataSet {} could not be downloaded: {}'.format(permId, exc)
)
return
# return success message
- path = os.path.join(destination, dataset.permId)
+ path = os.path.join(downloadPath, dataset.permId)
self.write({
'permId' : dataset.permId,
- 'statusText': 'Data for DataSet {} was successfully downloaded to {}.'.format(dataset.permId, path)
+ 'path' : path,
+ 'dataStore' : dataset.dataStore,
+ 'location' : dataset.physicalData.location,
+ 'size' : dataset.physicalData.size,
+ 'statusText': 'Data for DataSet {} was successfully downloaded to: {}.'.format(dataset.permId, path)
})
def get(self, **params):
"""Handle a request to /openbis/dataset/connection_name/permId
download the data and return a message
"""
+
try:
conn = openbis_connections[params['connection_name']]
except KeyError:
@@ -234,5 +270,77 @@ class DataSetHandler(IPythonHandler):
)
return
- results = self.download_data(conn, params['permId'])
+ results = self.download_data(conn=conn, permId=params['permId'], downloadPath=params['downloadPath'])
+
+
+class DataSetUploadHandler(IPythonHandler):
+ """Handle the requests for /openbis/dataset/connection"""
+
+
+ def upload_data(self, conn, data):
+ if not conn.is_session_active():
+ try:
+ conn.reconnect()
+ except Exception as exc:
+ self.send_error(
+ reason = 'connection to {} could not be established: {}'.format(conn.name, exc)
+ )
+ return
+
+ try:
+ sample = conn.openbis.get_sample(data.get('sampleIdentifier'))
+ except Exception as exc:
+ self.send_error(
+ reason = 'No such sample found: {}'.format(data.get('sampleIdentifier'))
+ )
+ return
+
+ filenames = []
+ for filename in data.get('files'):
+ filename = unquote(filename)
+ filenames.append(filename)
+
+ try:
+ ds = conn.openbis.new_dataset(
+ name = data.get('name'),
+ description = data.get('description'),
+ type = data.get('type'),
+ sample = sample,
+ files = filenames
+ )
+ except Exception as exc:
+ self.send_error(
+ reason = 'Error while creating the dataset: {}'.format(exc)
+ )
+ return
+
+ try:
+ ds.save()
+ except Exception as exc:
+ self.send_error(
+ reason = 'Error while saving the dataset: {}'.format(exc)
+ )
+ return
+
+
+ # return success message
+ self.write({
+ 'status': 200,
+ 'statusText': 'Jupyter Notebook was successfully uploaded to: {} with permId: {}'.format(conn.name, ds.permId)
+ })
+
+ def post(self, **params):
+ """Handle a request to /openbis/dataset/connection_name/permId
+ download the data and return a message
+ """
+
+ try:
+ conn = openbis_connections[params['connection_name']]
+ except KeyError:
+ self.send_error(
+ reason = 'connection {} was not found'.format(params['connection_name'])
+ )
+ return
+ data = self.get_json_body()
+ results = self.upload_data(conn=conn,data=data)
diff --git a/openbis-connections.yaml-example b/openbis-connections.yaml
similarity index 76%
rename from openbis-connections.yaml-example
rename to openbis-connections.yaml
index d8dfe123d941fe08f38a9d924b335ab678a30832..4e02d7ff96ecd04da4ee3b7195f6a75e416c719d 100644
--- a/openbis-connections.yaml-example
+++ b/openbis-connections.yaml
@@ -1,11 +1,11 @@
connections:
- name : local test openBIS instance
- url : localhost:8443
+ url : https://localhost:8443
verify_certificates : false
username : username
password : password
- name : productive openBIS instance
- url : openbis.example.com
+ url : https://openbis.example.com
verify_certificates : true
username : username
password : password