Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updating backup and purging information / simplification #56

Merged
merged 1 commit into from
Apr 19, 2024
+17 −41
Merged

Updating backup and purging information / simplification #56

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 7 additions & 27 deletions docs/data/backups.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,28 @@
a regular basis. **Ultimately, it is your responsibility to
protect yourself from data loss.**

ULHPC has 3 different backup targets, with different rotation policies and physical locations.

The backups are only accessible by HPC staff, for disaster recovery purposes.
The backups are only accessible by HPC staff, for disaster recovery purposes only.

More precisions can be requested via a support request.

## User directories on the ULHPC clusters
## Directories on the ULHPC clusters infrastructure

For computation purposes, ULHPC users can use multiple storages: home, scratch and projects. Note however that the HPC Platform does not have the infrastructure to backup all of them, see details below.

| Directory | Path | Backup location | Frequency | Retention |
|------------------|----------------|-----------------|-----------|--------------------------------------------------------------|
| home directories | `$HOME` | CDC, Belval | Weekly | last 7 backups, at least one per month for the last 2 months |
| projects | `$PROJECTWORK` | CDC, Belval | Weekly | one backup per week of the backup directory (`$PROJECT/backup/`) |
| home directories | `$HOME` | not backed up | |
| scratch | `$SCRATCH` | not backed up | | |
| projects | `$PROJECTWORK` | CDC, Belval | Weekly | one backup per week of the backup directory ONLY (`$PROJECT/backup/`) |


## Isilon project directories snapshots
## Directories on the SIU Isilon infrastructure

Projects stored on the Isilon filesystem are *snapshotted* weekly, the snapshots are kept for 10 days.

!!! danger
Snapshots are **not a real backup**. It does not protect you against a system failure, it will only permit to recover some files in case of accidental deletion


Each project directory, in `/mnt/isilon/projects/` contains a hidden sub-directory `.snapshot`:

* `.snapshot` is invisible to `ls`, `ls -a`, `find` and similar
Expand All @@ -37,19 +36,10 @@ Each project directory, in `/mnt/isilon/projects/` contains a hidden sub-directo
* files can *only* be copied *out* of a snapshot


## Virtual machines

| Source | Backup location | Frequency | Retention |
|------------------------|-----------------|-----------|----------------------------------------------------|
| Gitlab infrastructure | CDC, Belval | Weekly | last 5 weekly snapshots |
| Iris infrastructure | CDC, Belval | Weekly | last 5 weekly snapshots |


## Services

| Name | Backup location | Frequency | Retention |
|------------------------------|-----------------|-----------|-----------------------------------------------------------|
| gitlab.uni.lu | CDC, Belval | Daily | last 7 daily backups, one per month for the last 6 months |
| hpc.uni.lu (pad, privatebin) | CDC, Belval | Daily | last 7 daily backups, one per month for the last 6 months |


Expand All @@ -61,16 +51,6 @@ with pathnames and timestamps of the missing data.

Such restore requests may take a few days to complete.

## Purging

!!! note
See [Filesystem Quotas and Purging](../filesystems/quotas.md) for detailed information about inode,
space quotas, and file system purge policies.

!!! warning
`$SCRATCH` directories are **not** backed up


## Backup Tools

In practice, the ULHPC backup infrastructure is fully puppetized and make use of several tools facilitating the operations:
Expand Down
24 changes: 10 additions & 14 deletions docs/filesystems/quotas.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# Quotas and Purging
# Quotas

## Overview

| Directory | Default space quota | Default inode quota | Purge time |
| ---------------------- | ------------------- | ------------------- | ---------- |
| `$HOME` | 500 GB | 1 M | - |
| `$SCRATCH` | 10 TB | 1 M | 60 days |
| `/work/projects/...` | 1 TB | 1 M | - |
| `/mnt/isilon/projects/...` | 1.14 PB globally | - | - |
| Directory | Default space quota | Default inode quota |
| ---------------------- | ------------------- | ------------------- |
| `$HOME` | 500 GB | 1 M |
| `$SCRATCH` | 10 TB | 1 M |
| `/work/projects/...` | 1 TB | 1 M |
| `/mnt/isilon/projects/...` | 1.14 PB globally | - |

## Quotas

!!! warning
When a quota is reached writes to that directory will fail.

!!! note
On Isilon everyone shares one global quota. Unfortunately it is not possible to see the quota status on the cluster.
On Isilon everyone shares one global quota and the HPC Platform team sets up project quotas. Unfortunately it is not possible to see the quota status on the cluster.

### Current usage

Expand Down Expand Up @@ -60,9 +60,9 @@ For a more graphical approach, use `ncdu`, with the `c` option to display the ag

### Increases

If you or your project needs additional space or inodes for your scratch or project directory you may request it via [ServiceNow](https://hpc.uni.lu/support/) (HPC → Storage & projects → Extend quota).
If your project needs additional space or inodes for a specific project directory you may request it via [ServiceNow](https://hpc.uni.lu/support/) (HPC → Storage & projects → Extend quota).

Quotas on the home directory cannot be increased.
Quotas on the home directory and scratch cannot be increased.

### Troubleshooting

Expand All @@ -72,7 +72,3 @@ To avoid this issue, check out the `newgrp` command or set the `s` mode bit ("se

To transfer data with `rsync` into a project directory, please check the [data transfer documentation](/data/transfer/#transfer-from-your-local-machine-to-a-project-directory-on-the-remote-cluster).

## Purging
Files in the scratch (`$SCRATCH`, `/mnt/lscratch`) directories are purged. This means **files that have not been read** (i.e. atime has not been updated) **within the last 60 days are automatically removed every month**.

Files in `/tmp` on the compute nodes are **removed** at the end of the job.
Loading