From ff54989b82ffd6ecdd071d4a46cbcd144e0fc0eb Mon Sep 17 00:00:00 2001
From: vermeul <swen@ethz.ch>
Date: Wed, 3 Oct 2018 23:31:33 +0200
Subject: [PATCH] updated README
---
README.md | 44 ++++
docs/howto-implement-jupyter-extension.md | 296 ++++++++++++++++++++++
2 files changed, 340 insertions(+)
create mode 100644 docs/howto-implement-jupyter-extension.md
diff --git a/README.md b/README.md
index c61fdae..b421d71 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,49 @@
# Jupyter-OpenBIS-Extension
+## How to install the extension
+
+### Install the Python module(s)
+- cd into the extension `cd jupyter-openbis-extension`
+- install the python module normally: `pip install .`
+- or, for development, install just a symbolic link: `pip install -e .`
+- this extension needs Jupyter and pyBIS in order to run, so it will load it if not yet present
+- be aware that, because pyBIS needs at least Python 3.3, your Jupyter needs to run under the
+ same version of Python.
+
+
+### Register the Jupyter Notebook Server Extension (server-side, as the name suggests)
+- in the terminal, enter the following:
+
+``
+jupyter serverextension enable --py jupyter-openbis-extension.server
+``
+
+- OR (1) register it manually:
+- open the file `~/.jupyter/jupyter_notebook_config.py`
+- add the following:
+
+```
+c.NotebookApp.server_extensions = [
+ 'jupyter-openbis-extension.server'
+]
+```
+
+- OR (2) register it in `~/.jupyter/jupyter_notebook_config.json` manually, if the automatic registration fails for some reasons:
+```
+{
+ "NotebookApp": {
+ "nbserver_extensions": {
+ "jupyter_nbextensions_configurator": true,
+ "jupyter-openbis-extension.server": true
+ }
+ }
+}
+```
+
+### Register the Jupyter Notebook Extension (client-side)
+- open the file `~/.jupyter/
+
+
## General thoughts (unordered)
diff --git a/docs/howto-implement-jupyter-extension.md b/docs/howto-implement-jupyter-extension.md
new file mode 100644
index 0000000..d1496e9
--- /dev/null
+++ b/docs/howto-implement-jupyter-extension.md
@@ -0,0 +1,296 @@
+# How to implement the Jupyter-Openbis extension
+
+## How the functionality of Jupyter notebooks is extended.
+
+There are four ways to extend a Jupyter notebook (see also [Parente's Mindtrove: Four Ways to Extend Jupyter Notebook](http://mindtrove.info/4-ways-to-extend-jupyter-notebook/) ):
+
+1. Kernels
+2. IPython Kernel Extensions
+3. Notebook Extensions
+4. Notebook Server Extensions
+
+**Kernels** are used to run your notebook in a specific environment, be it a specific language or a special running environment, e.g. on a cluster.
+
+**Kernel Extensions** allow to modify your kernel environment within a cell. They often introduce some *Magics*, e.g. `%run_with_extra_horsepower <my_function>`
+
+**Notebook Extensions** improve or alter the GUI / Webinterface in order to add new functionality, a new shortcut, a special button, etc. They are written in JavaScript. Some of them interact with the Notebook server via a *Notebook Server Extension* (see next).
+
+**Notebook Server Extensions** run as part of the Jupyter notebook server and as such, they are written in the same language as the server. Which is – depending on your installation – either Python 2.7 or Python 3.3 or greater. Notebook Server Extensions can listen to request sent by the user via a *Notebook Extension* (see previous).
+
+For our Jupyter-Openbis extension we need both Notebook Extension and Notebook Server Extension. We start with the latter.
+
+## Implementing a Notebook Server Extension
+
+The Jupyter notebook server includes a [tornado webserver](http://www.tornadoweb.org/en/stable/) which will talk to the GUI.
+
+Extensions need to implement a method called `load_jupyter_server_extension` which will provide an instance of the running tornado Application. We then easily can add new handlers, which means, we can set up new commmunication channels between the Web frontend and the backend. In our case, the channel has the name `/openbis/dataset`.
+
+Here is our script `pybis.jupyter_extension.py`:
+
+```
+from notebook.utils import url_path_join
+from notebook.base.handlers import IPythonHandler
+from pybis import Openbis
+
+o = Openbis(url='https://sprint-openbis.ethz.ch')
+o.login('username', '***password****')
+
+import json
+import os
+import urllib
+from subprocess import check_output
+from urllib.parse import urlsplit, urlunsplit
+
+
+def _jupyter_server_extension_paths():
+ return [{'module': 'gitlab_commit_push'}]
+
+
+def load_jupyter_server_extension(nb_server_app):
+ """Call when the extension is loaded.
+
+ :param nb_server_app: Handle to the Notebook webserver instance.
+ """
+ web_app = nb_server_app.web_app
+ host_pattern = '.*$'
+
+ route_pattern = url_path_join(web_app.settings['base_url'], '/openbis/dataset')
+ web_app.add_handlers(host_pattern, [(route_pattern, DataSetHandler)])
+ print("pybis loaded: {}".format(Openbis))
+
+
+class DataSetHandler(IPythonHandler):
+ """Download the requested DataSet"""
+
+ def put(self):
+ """Handle a user PUT request."""
+
+ permId = self.get_json_body()
+
+ try:
+ ds = o.get_dataset(permId)
+ except ValueError:
+ self.write({
+ 'status':
+ 404,
+ 'statusText':
+ 'DataSet {} could not be found.'.format(permId)
+ })
+ return
+
+ try:
+ destination = ds.download()
+ except Exception as e:
+ self.write({
+ 'status':
+ 404,
+ 'statusText':
+ 'DataSet {} could not be downloaded: {}'.format(permId, e)
+ })
+ return
+
+ path = os.path.join(destination, ds.permId)
+
+ self.write({
+ 'status':
+ 200,
+ 'permId': ds.permId,
+ 'statusText':
+ 'DataSet was successfully downloaded to {}.'.format(path)
+ })
+```
+
+## Register Notebook Server Extension
+
+In order to let Jupyter know that it needs to run an extension, we have to register it in a file called `~/.jupyter/jupyter_notebook_config.py` like this:
+
+```
+c.NotebookApp.server_extensions = [
+ 'pybis.jupyter_extension'
+]
+```
+See [a very basic example here](http://mindtrove.info/4-ways-to-extend-jupyter-notebook/#nb-server-exts) to demonstrate the concept.
+
+
+## Implementing a Notebook Extension
+
+As mentioned before, Notebook Extensions are written in JavaScript. Much like the Notebook Server Extensions, they need to implement a special method: `load_ipython_extension`. See [here for a basic Javascript example](http://mindtrove.info/4-ways-to-extend-jupyter-notebook/#nb-extensions).
+
+In our case, we want to offer an input box, which takes a **permId** of a Dataset and sends this information to the already implemented Notebook Server Extension via the communication channel `/openbis/dataset``. After the download succeeded, our script then receives some confirmation message, including a statusText for user feedback as well as the permId, which is stored within the metadata of the notebook itself in order to keep track of all datasets downloaded.
+
+Here is the content of the script `jupyter-openbis-extension/main.js`:
+
+```
+define([
+ "base/js/namespace",
+ "base/js/dialog",
+ "base/js/utils",
+ "jquery"
+], function(IPython, dialog, utils, $, mc) {
+ var fetchDatasetFromOpenBis = {
+ help: 'Download a openBIS dataset to your local harddrive',
+ icon: 'fa-database',
+ help_index: '',
+ handler: function (env) {
+ var p = $('<p/>').text('Please enter a DataSet identifier or permId to load from sprint-openbis.ethz.ch')
+ var input = $('<textarea rows="4" cols="72"></textarea>')
+ var div = $('<div/>')
+ var commitUrl = env.notebook.base_url + 'openbis/dataset'
+
+ div.append(p).append(input)
+
+ // get the canvas for user feedback
+ var container = $('#notebook-container')
+
+ function onOk () {
+ var notebook = IPython.notebook
+ var re = /\/notebooks(.*?)$/
+ var filepath = window.location.pathname.match(re)[1]
+ //var payload = {'filename': filepath, 'msg': input.val()}
+ var settings = {
+ url: commitUrl,
+ processData: false,
+ type: 'PUT',
+ dataType: 'json',
+ //data: JSON.stringify(payload),
+ data: JSON.stringify(input.val()),
+ contentType: 'application/json',
+ success: function (data) {
+ // display feedback to user
+ var container = $('#notebook-container')
+ var feedback = '<div class="openbis-feedback alert alert-success alert-dismissible" role="alert">' +
+ '<button type="button" class="close" data-dismiss="alert" aria-label="Close">' +
+ '<span aria-hidden="true">×</span></button>' +
+ data.statusText +
+ '</div>'
+
+ // display feedback
+ $('.openbis-feedback').remove()
+ container.prepend(feedback)
+
+ // write statusText from returned data to notebooks metadata
+ if ( typeof notebook.metadata.openbis === 'undefined') {
+ notebook.metadata.openbis = {}
+ }
+ if ( typeof notebook.metadata.openbis.permIds === 'undefined' ) {
+ notebook.metadata.openbis.permIds = {}
+ }
+ if ( data.permId ) {
+ notebook.metadata.openbis.permIds[data.permId] = data.statusText
+ }
+ },
+ error: function (data) {
+ // display feedback to user
+ var feedback =
+ '<div class="openbis-feedback alert alert-danger alert-dismissible" role="alert">' +
+ '<button type="button" class="close" data-dismiss="alert" aria-label="Close"><span aria-hidden="true">×</span></button>' +
+ '<strong>Warning!</strong> Something went wrong.<div>' + data.statusText + '</div></div>'
+
+ // display feedback
+ $('.openbis-feedback').remove()
+ container.prepend(feedback)
+ }
+ }
+ //console.log(IPython.notebook)
+
+ // display preloader during commit and push
+ var preloader = '<img class="openbis-feedback" src="https://cdnjs.cloudflare.com/ajax/libs/slick-carousel/1.5.8/ajax-loader.gif">'
+ container.prepend(preloader)
+ $('.openbis-feedback').remove()
+
+ // commit and push
+ utils.ajax(settings)
+ }
+
+ if (IPython.notebook.dirty === true) {
+ dialog.modal({
+ body: 'Please, save the notebook before sharing.',
+ title: 'Action required',
+ buttons: {
+ 'Back': {}
+ },
+ notebook: env.notebook,
+ keyboard_manager: env.notebook.keyboard_manager
+ })
+ }
+ else {
+ dialog.modal({
+ body: div,
+ title: 'Download openBIS DataSet',
+ buttons: {
+ 'Download': {
+ class: 'btn-primary btn-large',
+ click: onOk
+ },
+ 'Cancel': {}
+ },
+ notebook: env.notebook,
+ keyboard_manager: env.notebook.keyboard_manager
+ })
+ }
+ }
+ }
+
+ function _onLoad () {
+ // log to console
+ console.info('Loaded Jupyter extension: openbis-jupyter-extension')
+ // register new action
+ var actionName = IPython.keyboard_manager.actions.register(
+ fetchDatasetFromOpenBis, 'openbis-dataset-fetch', 'jupyter-gitlab')
+
+ console.info(actionName)
+ // add button for new action
+ IPython.toolbar.add_buttons_group([actionName])
+ }
+
+ return {load_ipython_extension: _onLoad}
+})
+```
+
+## Register the Notebook Extension
+
+The Jupyter configuration which indicates which (JavaScript) Notebook Extension should be loaded is stored in this file: `.jupyter/nbconfig/notebook.json`. To maintain this file, Jupyter comes with some commands which will make things a bit easier.
+
+### a) install the extension
+
+```
+jupyter nbextension install /path/to/jupyter-openbis-extension --user --symlink
+```
+
+will create a symlink of our jupyter-openbis-extension folder containing `main.js` (and maybe more files) in its destination, which is (on a Mac): `~/Library/Jupyter/nbextensions/`. This is the recommended setup if you are still developing. If you omit the `--symlink` flag, your files will be *copied* instead. If you omit the `--user` flag, your extension will be installed system-wide. There are many more flags, type
+
+```
+jupyter nbextension list
+```
+to list them all.
+
+### b) register the extension
+
+`jupyter nbextension list` lists all extensions already installed. After you installed your extension, it needs to be enabled and validated:
+
+```
+jupyter nbextension enable jupyter-openbis-extension/main
+```
+
+This is Require.js - Style: You need to omit the `.js` extension of the entry-point of your Javascript file. If you omit `/main` altogether, the validation will fail:
+
+```
+$ jupyter nbextension enable jupyter-openbis-extension
+Enabling notebook extension jupyter-openbis-extension...
+ - Validating: problems found:
+ - require? X jupyter-openbis-extension
+```
+This is because Require.js does not find its entry point script (`main.js`) automatically. To remove the error, edit `~/.jupyter/nbconfig/notebook.js` and remove the faulty line. Watch out not to leave a trailing comma `,` in the list, it will produce yet another weird error when starting Jupyter.
+
+Once the extension is correctly installed, the output of `jupyter nbextension list` should look something like this:
+
+```
+$ jupyter nbextension list
+Known nbextensions:
+ config dir: /Users/username/.jupyter/nbconfig
+ notebook section
+ jupyter-openbis-extension/main enabled
+ - Validating: OK
+```
+
+Congratulations!
\ No newline at end of file
--
GitLab