Deutsches Elektronen-Synchrotron DESY
A Research Centre of the Helmholtz Association

IT

Central Data Storage at DESY

Currently this documentation is primarily intended for the office environments, but will be complemented with further information about data storage within scientific environments.

DESY offers various possibilities for central storage of business data. This type of central data storage should be preferred, as this usually involves a backup and the possibility of data restore. Further information about the possibilities of backup and restore can be found in the concerning chapter of this document.

This is usually not the case, when using local media such as hard disks, desktop and notebook computers, external hard drives, DVDs, and USB - sticks, etc.

Below these various possibilities for central data storage at DESY are listed and the cases for which they are primarily intended are outlined.

Further Information

You can find more detailed information about some of the above mentioned possibilities of central data management at DESY at https://it.desy.de/dienste/speicherdienste.

If your needs should not or only suboptimal be covered or you are in doubt how you or your group should proceed, please make use of our consulting service and contact us via UCO (uco@desy.de, Tel: 5005) - we will find an adequate solution.

Overview about Data Access

In the following table the possibilities from which operating system you are able to access which data storage are outlined. As far as available the "+" symbol links to a corresponding guidance.

  • ( + ) Access is possible
  • ( o ) Access is possible but not supported by DESY IT
  • ( - ) Access is not possible
     
  CIFS* AFS dCache Sync & Share
Windows + + + *** +
Linux + ** + + +
macOS + + + +
Android o - - +
iOS - - - +

* Primarily intended for the access to the Windows network drives H:, N: and S:
** For technical reasons the access to the S: drive is only possible with restrictions.
*** Read only permitted

User Home Directory

Purpose

The so-called "Home Directory" – also called H: drive on DESY Windows machines, /afs/desy.de/user/<InitialLetterOfAccount>/<account> on Linux machines - is intended for the storage of official, personal data of the account owner. Therefore, only the account owner can write and read there. Excluded from this are solely the subdirectories /<group> or /public, intended for exchange, to which only group members or DESY users have read access.

Please note that due to the more extensive concept of AFS the public directory in the AFS file system is globally readable.

Note: On Ubuntu Desktop PCs the default home directory is located on the local hard drive (therefore normally no backup is available here!) at /home/<account>. The AFS home directory have to be addressed explicitly with the above mentioned AFS path.

Backup

All data in the home directory is stored in the DESY backup and can usually be restored by the user himself / herself. Otherwise, data can be restored via request from the responsible group administrator.

Quota

In particular, the following data should not be included in the Home Directory:

  • Data that is permanently relevant to the whole group
  • Mass data such as simulation and measurement data
  • Non - business related music, photo and video files
  • Easy to retrieve data (via Internet)

 

Because of these reasons, disk space in the Home directory is limited to initially 8 GB. If necessary, the quota of the Home Directories can be increased to a limited extent, if requested by the responsible group administrator.

Restore Data in AFS Home

For information about backing up and restoring data in the AFS Home directory, please visit the following Web site: Restoring data in the AFS Home Directory

Restore data in Windows Home

For information about backing up and restoring data in the Windows Home directory, please visit the following Web site: Restore data in Windows Home Directory

Group Directory

Purpose

The group directory - N: drive on DESY Windows machines - is intended for group-wide relevant data. The organization of the group drive is responsibility of the so-called group administrator according to the structures required in the respective group.

Backup

All data in the group directory is stored in the DESY backup and can be restored if necessary.

Quota

The disk space in the group drive is comparatively dynamic and can be increased up to several 100 GB if there is justified demand.

XXL Directory

Purpose

The home directory can be extended by a so-called XXL directory, in which large amounts of temporary, personal data can be stored.

Backup

Data in the XXL directory is not saved daily, a weekly backup is attempted.

Quota

Storage capacity in XXL directories can be increased up to 200 GB depending on the circumstances.

DESY Sync & Share

Purpose

DESY-IT runs a storage service based on Nextcloud and dCache "DESY Sync & Share". The service is designed to synchronize data between central storage, mobile and stationary devices, and share data with colleagues within and outside DESY.

Backup

There is no regular TSM Backup of the data which is stored on DESY Sync & Share. If data is deleted it is stored in the "Trash" folder for 180 days. After this time period the data is definetly deleted. Of course IT has a desaster recovery procedure for the service which enables a full recovery of the system.

Quota

There is no quota for storing data on the DESY Sync & Share service. Reporting on the storage usage is planned.

dCache

Purpose

The dCache is the data storage and management system specifically for measurement, simulation and analysis data.

Backup

Data can be created on disc storage with one or more replicas, depending on the requirements. It is also possible to save replicas of data on tape.

Quota

Data storage up to PB-area is possible.

Central Computing Clusters at DESY

BIRD/NAF

Recommendations about using data storage on the central computing clusters BIRD and NAF can be found on the following web sites:
https://naf-wiki.desy.de/NafStorage
https://naf-wiki.desy.de/DustLifecycle

Maxwell

Recommendations for central data management using the Maxwell Cluster can be found on the following website (login with DESY account and password required):
https://confluence.desy.de/pages/viewpage.action?pageId=24807333

UCO Hamburg

UCO Hamburg
Phone: +49 (0)40 8998 5005
E-Mail: UCO Hamburg
Location: 2b / 131d
Link: https://it.desy.de/services/uco
Logo der Helmholtz Gemeinschaft

© 2020, Deutsches Elektronen-Synchrotron DESY