Skip to content
Snippets Groups Projects
Commit 8a9ac686 authored by Marco Del Tufo's avatar Marco Del Tufo
Browse files

Update archive-datasets.md

parent 3c0ba6be
No related branches found
No related tags found
1 merge request!40SSDM-13578 : 2PT : Database and V3 Implementation - include the new AFS "free"...
...@@ -7,69 +7,37 @@ Archiving Datasets ...@@ -7,69 +7,37 @@ Archiving Datasets
Archiving can be triggered by doing the following steps: Archiving can be triggered by doing the following steps:
- go to an experiment/collection or an object. - go to an experiment/collection or an object.
- switch to the tab "Data Sets". There will be in ther lower right - switch to the tab "Data Sets". There will be in ther lower right corner the button 'Archiving'.
corner the button 'Archiving'. - click on the button and choose either 'Copy to Archive' or 'Move to Archive'.
- click on the button and choose either 'Copy to Archive' or 'Move to - if you did not select any data set all data sets will be archived. If you have selected some data sets you can choose if you want to archive only them or all the data sets accessible in the table.
Archive'.
- if you did not select any data set all data sets will be archived. Because archiving does not happens immediately the status (called 'Archiving Status' in data set tables) of the data sets will be changed to BACKUP\_PENDING or ARCHIVE\_PENDING.
If you have selected some data sets you can choose if you want to
archive only them or all the data sets accessible in the table. To make archived data sets available again repeat the steps, but choose 'Unarchive'.
Because archiving does not happens immediately the status (called If you want to disallow archiving, choose 'Lock'. Remember that you can do this only for available data sets. The 'Archiving Status' will change to 'AVAILABLE (LOCKED)'. To make archiving possible again, choose 'Unlock'.
'Archiving Status' in data set tables) of the data sets will be changed
to BACKUP\_PENDING or ARCHIVE\_PENDING.
To make archived data sets available again repeat the steps, but choose
'Unarchive'.
If you want to disallow archiving, choose 'Lock'. Remember that you can
do this only for available data sets. The 'Archiving Status' will change
to 'AVAILABLE (LOCKED)'. To make archiving possible again, choose
'Unlock'.
### ELN-LIMS ### ELN-LIMS
Instead of triggering archiving only requesting archiving is possible. Instead of triggering archiving only requesting archiving is possible.
The maintenance task The maintenance task [ArchivingByRequestTask](https://openbis.readthedocs.io/en/latest/system-admin-documentation/advanced-features/maintenance-tasks.html#archivingbyrequesttask) is required. It triggers the actual archiving.
[ArchivingByRequestTask](/display/openBISDoc2010/Maintenance+Tasks#MaintenanceTasks-ArchivingByRequestTask)
is required. It triggers the actual archiving.
## Automatic archiving ## Automatic archiving
Archiving can be automated by the Auto Archiver. This is a [maintenance Archiving can be automated by the Auto Archiver. This is a [maintenance task](https://openbis.readthedocs.io/en/latest/system-admin-documentation/advanced-features/maintenance-tasks.html) which triggers archiving of data sets fullfulling some conditions (e.g. not accessed since a while). Note that the auto archiver doesn't archives itself. It just automates the selection of data sets to be archived. For all configuration parameters see [AutoArchiverTask](https://openbis.readthedocs.io/en/latest/system-admin-documentation/advanced-features/maintenance-tasks.html#autoarchivertask).
task](/display/openBISDoc2010/Maintenance+Tasks) which triggers
archiving of data sets fullfulling some conditions (e.g. not accessed
since a while). Note that the auto archiver doesn't archives itself. It
just automates the selection of data sets to be archived. For all
configuration parameters see
[AutoArchiverTask](/display/openBISDoc2010/Maintenance+Tasks#MaintenanceTasks-AutoArchiverTask).
### Archiving Policies ### Archiving Policies
An archiving policy selects from the unarchived data sets candidates An archiving policy selects from the unarchived data sets candidates (which are either data sets not accessed since some days or data sets marked by a tag) the data sets to be archived. If not specified all candidates will be archived.
(which are either data sets not accessed since some days or data sets
marked by a tag) the data sets to be archived. If not specified all
candidates will be archived.
The policy can be specified by `policy.class` property. It has to be the The policy can be specified by `policy.class` property. It has to be the fully-qualified name of a Java class implementing` ch.systemsx.cisd.etlserver.IAutoArchiverPolicy`. All properties starting with `policy.` specifying the policy further.
fully-qualified name of a Java class
implementing` ch.systemsx.cisd.etlserver.IAutoArchiverPolicy`. All
properties starting with `policy.` specifying the policy further.
#### ch.systemsx.cisd.etlserver.plugins.GroupingPolicy #### ch.systemsx.cisd.etlserver.plugins.GroupingPolicy
**Description**: Policy which tries to find a group of data sets with a **Description**: Policy which tries to find a group of data sets with a total size from a specified interval. This is important in case of [Multi Data Set Archiving](/display/openBISDoc2010/Multi+data+set+archiving). Grouping can be defined by space, project, experiment, sample, data set type or a combination of those. Groups can be merged if they are too small. Several grouping keys can be specified.
total size from a specified interval. This is important in case of
[Multi Data Set
Archiving](/display/openBISDoc2010/Multi+data+set+archiving). Grouping
can be defined by space, project, experiment, sample, data set type or a
combination of those. Groups can be merged if they are too small.
Several grouping keys can be specified.
Searching for an appropriate group of data sets for auto archiving is Searching for an appropriate group of data sets for auto archiving is logged. If no group could be found an admin is notified via email (email address specified in `log.xml`). The email contains the searching log.
logged. If no group could be found an admin is notified via email (email
address specified in `log.xml`). The email contains the searching log.
**Configuration**: **Configuration**:
...@@ -84,14 +52,17 @@ address specified in `log.xml`). The email contains the searching log. ...@@ -84,14 +52,17 @@ address specified in `log.xml`). The email contains the searching log.
**plugin.properties** **plugin.properties**
class = ch.systemsx.cisd.etlserver.plugins.AutoArchiverTask ```
interval = 10 days class = ch.systemsx.cisd.etlserver.plugins.AutoArchiverTask
archive-candidate-discoverer.class = ch.systemsx.cisd.etlserver.plugins.TagArchiveCandidateDiscoverer interval = 10 days
archive-candidate-discoverer.tags = /admin-user/archive archive-candidate-discoverer.class = ch.systemsx.cisd.etlserver.plugins.TagArchiveCandidateDiscoverer
policy.class = ch.systemsx.cisd.etlserver.plugins.GroupingPolicy archive-candidate-discoverer.tags = /admin-user/archive
policy.minimal-archive-size = 30000000000 policy.class = ch.systemsx.cisd.etlserver.plugins.GroupingPolicy
policy.maximal-archive-size = 150000000000 policy.minimal-archive-size = 30000000000
policy.grouping-keys = Space#DataSetType, Experiment#Sample:merge policy.maximal-archive-size = 150000000000
policy.grouping-keys = Space#DataSetType, Experiment#Sample:merge
```
In this example the candidates are unarchived data sets which have been In this example the candidates are unarchived data sets which have been
tag by the user `admin-user` with the tag `archive`. The policy tries to tag by the user `admin-user` with the tag `archive`. The policy tries to
...@@ -103,4 +74,4 @@ samples are assigned to `no_sample`). If no matching groups are found ...@@ -103,4 +74,4 @@ samples are assigned to `no_sample`). If no matching groups are found
and at least two groups are below the minimum the policy tries to merge and at least two groups are below the minimum the policy tries to merge
groups to a bigger group until the bigger group match the size groups to a bigger group until the bigger group match the size
condition. If no group can be found an email will be sent describing in condition. If no group can be found an email will be sent describing in
detail the several steps of finding a matching group. detail the several steps of finding a matching group.
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment