diff --git a/docs/user-documentation/advance-features/command-line-tool.md b/docs/user-documentation/advance-features/command-line-tool.md index d84df1ff1d3b932e33553e87def6421a39bb0e1d..c36b5f370866b9630dad271aed3364d910930268 100644 --- a/docs/user-documentation/advance-features/command-line-tool.md +++ b/docs/user-documentation/advance-features/command-line-tool.md @@ -619,7 +619,7 @@ PAT generation is explained in depth in `token` command section. The Big Data Link Services can be used to download files which are contained in an obis repository. The services are included in the installation folder of openBIS, under `servers/big_data_link_services`. For how to configure and run them, consult -the [README.md](https://sissource.ethz.ch/sispub/openbis/blob/master/big_data_link_server/README.md) +the [README.md](https://sissource.ethz.ch/sispub/openbis/-/blob/master/server-external-data-store/README.md) file. ## 8. Rationale for obis diff --git a/docs/user-documentation/general-users/data-export.md b/docs/user-documentation/general-users/data-export.md index f927519e3b9befe35a2f3423a293f414c5e89119..3c3619a49d339a929a028d958a59fb63c2713753 100644 --- a/docs/user-documentation/general-users/data-export.md +++ b/docs/user-documentation/general-users/data-export.md @@ -43,12 +43,15 @@ In each case, the following export options are available:  - **Make import compatible**. If selected, datasets are exported in a **data** folder and are in a format ready to be uploaded in openBIS using the default eln-lims dropbox; the metadata are exported in a **xlsx** folder which contains information in a format ready to be uploaded via the openBIS admin UI. -- **Export metadata as PDF**. Metadata are exported in a **pdf** folder that keeps the folder structure of the ELN. At each level, one pdf file for each exported entity is generated. +- **Export metadata as PDF**. Metadata are exported in a **hierarchy** folder that keeps the folder structure of the ELN. At each level, one pdf file for each exported entity is generated. -- **Export metadata as XLSX**. Metadata are exported in one **xlsx** folder. The folder contains the metadata of all exported entities and the corresponding masterdata in a **metadata.xlsx** file. If **Make import compatible** is selected, this file is suitable for re-import in openBIS. If not, the file contains some fields which are not compatible with re-imports. These fields are: PermId of entities, registrator, registration date, modifier, modification date. In addition to the metadata.xlsx file, the **xlsx** folder might contain a **scripts** folder, which contains scripts associated with types in the metadata.xlsx file, if these are present, and a **data** folder which holds the content of spreadsheet fields and large text fields that exceed the size of an Excel cell. +- **Export metadata as XLSX**. Metadata are exported in one **xlsx** folder. The folder contains the metadata of all exported entities and the corresponding masterdata in a **metadata.xlsx** file. If **Make import compatible** is selected, this file is suitable for re-import in openBIS. If not, the file contains some fields which are not compatible with re-imports. These fields are: PermId of entities, registrator, registration date, modifier, modification date. In addition to the metadata.xlsx file, the **xlsx** folder might contain additional folders: + - a **scripts** folder, which contains scripts associated with types in the metadata.xlsx file, if these are present; + - a **data** folder which holds the content of spreadsheet fields and large text fields that exceed the size of an Excel cell; + - a **miscellaneous** folder which contain images embedded in text of exported entries, if present. - **Export data**. The default maximum size of all datasets to be exported is 10GB. This can be configured by a system admin in the [AS service.properties file](../../system-documentation/configuration/optional-application-server-configuration.md). We recommend to use [sftp](../general-users/lab-notebook.md#data-access) to download large datasets. -If **Make import compatible** is selected, datasets are exported in a **data** folder in a format ready to be uploaded in openBIS using the default eln-lims dropbox. If not, the datasets are exported in a **pdf** folder that matches the ELN hierarchy. +If **Make import compatible** is selected, datasets are exported in a **data** folder in a format ready to be uploaded in openBIS using the default eln-lims dropbox. If not, the datasets are exported in a **hiearchy** folder that matches the ELN hierarchy. - **Include levels below from same space**. If selected, all hierachy levels below the selected entity and belonging to the same Space are exported. @@ -63,7 +66,6 @@ _Note: ensure that pop-ups are not disabled in your browser_. Server](../../system-documentation/configuration/optional-datastore-server-configuration.md) ->  **Known limitation of openBIS 20.10.8**: images embedded in text fields are not currently exported. This limitation will be fixed in the release after 20.10.8. @@ -91,22 +93,21 @@ The exported zip file contains 3 folders: This contains the datasets in the correct format to be uploaded via eln-lims dropbox, as shown below. - + - -**B.** **pdf** folder +**B.** **hiearchy** folder This contains folders that match the openBIS hierarchy (Space/Project/Experiment/Object). In this case 2 Space folders are present: 1. **CATERINA**: is the exported space. -2. **METHODS**: contains an Object which is parent of an Object in the space CATERINA. This was exported because the option **Include Objects and Datasets parents and children from different spaces** was selected for export. +2. **METHODS**: contains an Object which is parent of an Object in the Space CATERINA. This was exported because the option **Include Objects and Datasets parents and children from different spaces** was selected for export. - + - + Inside each folder, there is a pdf of the corresponding entity. Example: @@ -114,7 +115,7 @@ Inside each folder, there is a pdf of the corresponding entity. Example: - in the Space folder **CATERINA** there is a **CATERINA.pdf** file that contains the metadata of the Space; - in the Project folder **PROJECT_1** there is a **PROJECT_1.pdf** file that contains the metadata of the Project; - in the Experiment folder **My second experiment (PROJECT_1_EXP_1)** there is a **My second experiment (PROJECT_1_EXP_1).pdf** file with the metadata of the Experiment; -- in the Object folder **Step A (EXP4)** there is a **Step A(EXP4).pdf** file with the metadata of the Object and a **20240523100301019-55.pdf** file that contains the metadata of the dataset that belongs to this Object. +- in the Object folder **Step A (EXP4)** there is a **Step A(EXP4).pdf** file with the metadata of the Object and a **20240726094631217-68.pdf** file that contains the metadata of the dataset that belongs to this Object. **C.** **xlsx** folder. @@ -122,14 +123,17 @@ Inside each folder, there is a pdf of the corresponding entity. Example: This contains: - a **metadata.xlsx** file which has the metadata of the exported entities and the corresponding masterdata (types and properties) in the correct format to be re-imported in another openBIS instance; -- a **scripts** folder that contains evaluation plugins associated to two types defined in the metadata.xlsx file; -- a **data** folder that contains the information stored in the spreadsheet field of one of the Objects in this Space. +- a **scripts** folder that contains evaluation plugins associated to two types defined in the metadata.xlsx file. This folder is present only if the exported types have plugins associated with them. +- a **data** folder that contains the information stored in the spreadsheet field of one of the Objects in this Space. This folder is present only if the exported entities contain information in spreadsheet or if there are text fields with more than 32,767 characters (this is the limit of the Excel cells). +- a **miscellaneous** folder that contains images that are embedded in text fields of the exported entities. This folder is present only if exported entities contain images embedded in text. - + - + + + ### **2. Non import-compatible export of a Space selecting all options** @@ -138,23 +142,24 @@ We export the same Space as described in Example 1, with all options selected, b   -In this case the exported zip file contains only 2 folders: **pdf** and **xlsx**. Data are exported inside the pdf folder, instead of being in a separate **data** folder. +In this case the exported zip file contains only 2 folders: **hierarchy** and **xlsx**. Data are exported inside the **hierachy** folder, instead of being in a separate **data** folder. -**A.** **pdf** folder +**A.** **hierarchy** folder This contains the same folder structure as described above. In addition, in this case, inside the Object **Step A (EXP4)** folder there is a **data** folder that contains the dataset belonging to this Object, as shown below. The metadata of the dataset is provided as a metadata.json file inside the data folder and as pdf file inside the Object folder (**Step A (EXP4)**). - + **B.** **xlsx** folder This contains the same files and folders as described in Example 1 (see below). The only difference in this case is that the metadata.xlsx is not import-compatible. It contains some fields which are not compatible with openBIS re-import, as explained above. - + - + + ## Export to Zenodo diff --git a/docs/user-documentation/general-users/img/IC-20109-export-data-folder-v2.png b/docs/user-documentation/general-users/img/IC-20109-export-data-folder-v2.png new file mode 100644 index 0000000000000000000000000000000000000000..3133f3a5ab1f6817866c80a0eef6f19233b320a4 Binary files /dev/null and b/docs/user-documentation/general-users/img/IC-20109-export-data-folder-v2.png differ diff --git a/docs/user-documentation/general-users/img/IC-20109-export-hiearchy-2.png b/docs/user-documentation/general-users/img/IC-20109-export-hiearchy-2.png new file mode 100644 index 0000000000000000000000000000000000000000..fad60db1a02f72ddc73c90dc9d3cc90cf94f236c Binary files /dev/null and b/docs/user-documentation/general-users/img/IC-20109-export-hiearchy-2.png differ diff --git a/docs/user-documentation/general-users/img/IC-20109-export-hierarchy-1.png b/docs/user-documentation/general-users/img/IC-20109-export-hierarchy-1.png new file mode 100644 index 0000000000000000000000000000000000000000..90445459a9441ca9e6fe8efc85f511d50a9d5c54 Binary files /dev/null and b/docs/user-documentation/general-users/img/IC-20109-export-hierarchy-1.png differ diff --git a/docs/user-documentation/general-users/img/IC-20109-export-miscellaneous.png b/docs/user-documentation/general-users/img/IC-20109-export-miscellaneous.png new file mode 100644 index 0000000000000000000000000000000000000000..afbd6820e52565618d211ce21bae75eeb4bd2f54 Binary files /dev/null and b/docs/user-documentation/general-users/img/IC-20109-export-miscellaneous.png differ diff --git a/docs/user-documentation/general-users/img/IC-20109-export-xlsx-data.png b/docs/user-documentation/general-users/img/IC-20109-export-xlsx-data.png new file mode 100644 index 0000000000000000000000000000000000000000..48f0f0670ceb989a108e9b57eaba12a37e9afb01 Binary files /dev/null and b/docs/user-documentation/general-users/img/IC-20109-export-xlsx-data.png differ diff --git a/docs/user-documentation/general-users/img/IC-20109-export-xlsx-scripts.png b/docs/user-documentation/general-users/img/IC-20109-export-xlsx-scripts.png new file mode 100644 index 0000000000000000000000000000000000000000..895e09680781d885d082991c05e8f2ef84e42e6f Binary files /dev/null and b/docs/user-documentation/general-users/img/IC-20109-export-xlsx-scripts.png differ diff --git a/docs/user-documentation/general-users/img/NIC-20109-hiearchy-folder.png b/docs/user-documentation/general-users/img/NIC-20109-hiearchy-folder.png new file mode 100644 index 0000000000000000000000000000000000000000..b733d5680d8542964fd4573c63e067a23287e4ee Binary files /dev/null and b/docs/user-documentation/general-users/img/NIC-20109-hiearchy-folder.png differ diff --git a/docs/user-documentation/general-users/img/NIC-20109-miscellaneous.png b/docs/user-documentation/general-users/img/NIC-20109-miscellaneous.png new file mode 100644 index 0000000000000000000000000000000000000000..86ac6ac425f46fc329569ecd43c43b85da9d3835 Binary files /dev/null and b/docs/user-documentation/general-users/img/NIC-20109-miscellaneous.png differ diff --git a/docs/user-documentation/general-users/img/NIC-20109-xlsx-data.png b/docs/user-documentation/general-users/img/NIC-20109-xlsx-data.png new file mode 100644 index 0000000000000000000000000000000000000000..d9963811b86b703c161b7e6caf570136486a5b4d Binary files /dev/null and b/docs/user-documentation/general-users/img/NIC-20109-xlsx-data.png differ diff --git a/docs/user-documentation/general-users/img/NIC-20109-xlsx-scripts.png b/docs/user-documentation/general-users/img/NIC-20109-xlsx-scripts.png new file mode 100644 index 0000000000000000000000000000000000000000..a7c7d6b10e33bbb5c72b17f1bc3ae61fb7eddf32 Binary files /dev/null and b/docs/user-documentation/general-users/img/NIC-20109-xlsx-scripts.png differ