data-upload.md



Data Upload

Data upload via web UI

 
  To upload data via the web interface:
 
1.Click the Upload button in the form, as shown below.

2. Select the dataset type (e.g. Attachment).
3. Fill in the relevant fields in the form. It is advisable to always
enter a Name, because this is shown in the menu. If the name is not
provided, the dataset code is shown.
4. Drag and drop files in the Files Uploader area or browse for
files.
5. When uploading a zip file, the option to uncompress before
import will be presented in the form.
6. Save.  
 

**Note for MacOS users: **the default MacOS archiver generates hidden
folders that become visible in openBIS upon unarchive. To avoid this
there are two options:

Zip using  the following command on the command-line: zip -r
 folder-name.zip folder-name/*  -x “*.DS_Store”

Use an external archiver (e.g. Stuffit Deluxe).

Updated on March 23, 2023

Data upload via dropbox

 
Web upload of data files is only suitable for files of limited size (few
GB). To upload larger data, openBIS uses dropbox scripts that run in the
background (see
Dropboxes).
A default dropbox script is provided with the openBIS ELN-LIMS plugin,
and the dropbox folder needs to be set up by a system admin.
If this is available, users need to organise their data in a specific
way:
 
Folder 1
**         Data** (can be single files or folders)
 
Folder 1 needs to have a specific name that encodes the information
of where the data should be uploaded to openBIS.
The name of Folder 1 can be generated from the ELN interface:
 

From the page where you want to upload data, select Dataset upload
helper tool for eln-lims dropbox from the More… dropdown and
follow the instructions on screen.

 
2. Select:

The dataset type from the list of available types (mandatory);
Enter the name of your dataset (optional, but recommended);
Copy the generated name of the folder using the copy to clipboard
icon.

 
3. In your finder/explorer, create a new folder and paste the name you
copied from openBIS. Place your data in this folder.
 

4. Place this folder containing your data inside the
eln-lims-dropbox folder. openBIS continuously monitors this folder
and when data are placed here, they are moved to the final storage.
The move happens after a predefined (and customisable) inactivity period
on the eln-lims-dropbox folder.
 

Dropbox with markerfile
 
In case of uploads of data >100GB we recommend to configure the
eln-lims-dropbox-marker. The set up and configuration need to be
done by a system admin. The process of data preparation is the same as
described above, however in this case the data move to the openBIS final
storage only starts when a markerfile is placed in the
eln-lims-dropbox-marker folder. The marker file is an empty file with
this name:  **.MARKER_is_finished_<folder-to-upload-name>.
**Please note the “.” at the start of the name, which indicates that
this is a hidden file. This file should also not have any extension. For
example, if the folder to be uploaded has the following name:
 
O+BARILLAC+PROJECT_1+EXP1+RAW_DATA+test
 
The marker file should be named:
 
.MARKER_is_finished_O+BARILLAC+PROJECT_1+EXP1+RAW_DATA+test
 
 
How to create the Marker file in Windows

 
You can create the Marker file in Windows using a text editor such as
Editor. Any other text editor will  also work.

open Editor.

Save the file with a name such as
.MARKER_is_finished_O+BARILLAC+PROJECT_1+EXP1+RAW_DATA+test.

The file is automatically saved with a “.txt” extension. This needs
to be removed.
Use the Rename option to remove the extension from the file.

 
How to create the Marker file on Mac

 
If you are not familiar with the command line, you can create an empty
text file using for example the TextEdit application in a Mac. Any
other text editor will also work.

Open the TextEdit application and save an empty file with a name
such as
.MARKER_is_finished_O+BARILLAC+PROJECT_1+EXP1+RAW_DATA+test.
Save to any format.
You will get a message to say that files starting with “.” are
reserved for the system and will be hidden. Confirm that you want to
use “.”
To show these hidden files, open the Finder and press Command +
Shift + . (period).
The file you saved before has an extension, that needs to be
removed. If the extension is not shown in your Finder, go to Finder
> Preferences menu, select the Advanced tab, and check the “Show
all filename extensions” box.
Remove the extension from the file.

 
Dropbox monitor
 
It is possible to check the status of the upload via dropbox using the
Dropbox Monitor under Utilities in the main menu.