# Disaster recovery

> Bring a lost or rebuilt server back with the configuration backup, the recovery key, account backups on S3 and the zopanel ctl dr-restore command.

Source: https://zopanel.net/docs/disaster-recovery  
Updated: 2026-10-07

Account backups bring back websites, databases and mail. To rebuild a whole server you also need what account backups do not hold: the panel's settings, packages, administrator and reseller logins and its keys. ZoPanel keeps these in an encrypted **configuration backup** (`.zpb`). This page explains what is kept where, and the steps for each kind of failure.

Test the "server lost" procedure on a spare VPS at least once a quarter.

## What is kept, and where

| What | Where | How long |
| --- | --- | --- |
| Panel database (accounts, sites, DNS, mail, settings) | `/var/lib/zopanel/db-backups/zopanel-YYYYMMDD.db` | 7 days |
| Panel database before each update | `/var/lib/zopanel/pre-update/` | Last 3 updates |
| Configuration backup (`.zpb`) | S3, under `<prefix>/<hostname>/_panel/` (daily, when remote backups are on), or downloaded by hand | As many as account backups |
| Account backups (files, databases, mailboxes) | `~/backups` on the server, and S3 when remote backups are on | **Settings → General** and **Settings → Remote backups** |
| Incremental backups (restic) | Local repository or S3 | Incremental backup retention |
| Deleted accounts | `/home/.zp-terminated/<user>-<time>/` and `/var/vmail/.zp-terminated/<user>-<time>/` | **Keep deleted accounts (days)**, 7 by default |

## The configuration backup

A configuration backup contains:

- the settings: S3, SMTP relay, notifications and so on,
- packages, administrator and reseller logins, and external database servers,
- a full copy of the panel database,
- all of `/etc/zopanel`, including the restic repository password, DKIM and DNS keys, the ACME account and app environments.

Machine-specific settings, such as the panel domain and fleet tokens, are not carried over.

The file is encrypted with XChaCha20-Poly1305 using the **recovery key**. With remote backups on, a new copy is uploaded to S3 every day. You can also download one at any time from **Backups → Disaster recovery → Download configuration (.zpb)**.

## The recovery key

The recovery key opens the configuration backup and every encrypted copy on S3. To see it, click **Backups → Disaster recovery → Show key**. The same card shows when the configuration was last pushed to S3.

**Keep the key outside the server**, in a password manager or on paper in a safe. If the server is lost and you do not have the key, the backups on S3 cannot be decrypted, and nobody can recover them for you.

## First checks: the panel does not start

Before you rebuild anything, check whether the server can be repaired:

```bash
zopanel ctl doctor                        # services, agent, database, disks, memory, certificate, clock
journalctl -u zopanel -u zopanel-agent -n 200
zopanel ctl support-bundle                # logs and configuration for support, without passwords or keys
```

## A bad update

Updates roll back by themselves, both the binary and the database, when the new version is not healthy within 3 minutes. To roll back by hand later:

```bash
zopanel rollback          # previous version, current database
zopanel rollback --db     # previous version and the database saved before the update
```

## Damaged panel database

Restore one of the daily copies:

```bash
systemctl stop zopanel
cp /var/lib/zopanel/db-backups/zopanel-<day>.db /var/lib/zopanel/zopanel.db
rm -f /var/lib/zopanel/zopanel.db-wal /var/lib/zopanel/zopanel.db-shm
chown zopanel:zopanel /var/lib/zopanel/zopanel.db
systemctl start zopanel
```

Any changes made since that day, such as new accounts or settings, have to be made again.

## Server lost: restore on a new server

You need the S3 settings, the recovery key and a new server running the same OS family.

1. **Install ZoPanel** on the new server with `install.sh`. Use the same hostname if you can.
2. **Connect the same storage.** In **Settings → Remote backups**, enter the same endpoint, bucket, folder prefix and keys as on the old server, then **Save**.
3. **Restore the configuration.** Open **Backups → Disaster recovery → Browse S3**, open the old server's folder (its hostname), then `_panel`, and click **Restore configuration** on the newest `.zpb` file. Enter the recovery key. You can also upload a downloaded `.zpb` file with **Restore configuration**.
   - Settings are overwritten. Packages, administrator and reseller logins, and external database servers are added unless one with the same name already exists.
   - The old server's recovery key and incremental backup password come back, so its backups can be read on this server.
4. **Restore the accounts.** In **Browse S3**, open the old server's folder, then each account, and click **Restore** on its newest backup. Missing accounts are created as they were, with their websites, databases, mailboxes and DNS.
5. **Re-issue the panel certificate.** Set the panel domain under **Settings → General → Panel domain & SSL**.
6. **Switch DNS.** Point the domains, or the old IP address, to the new server. Website certificates are issued again automatically once the domains resolve to it.

## Same server rebuilt (OS reinstalled, `/home` kept)

When the operating system was reinstalled but the disk that holds `/home` survived, restore the whole panel state from the command line instead:

```bash
# after installing ZoPanel again
zopanel ctl dr-restore zopanel-config-<date>.zpb --key <RECOVERY-KEY> --dry-run   # check the file
zopanel ctl dr-restore zopanel-config-<date>.zpb --key <RECOVERY-KEY>
zopanel ctl rebuild
```

- `--dry-run` only decrypts and checks the file. It shows the server it came from, the ZoPanel version and the date.
- The key can also be passed in the `ZOPANEL_RECOVERY_KEY` environment variable, which keeps it out of your shell history.
- `dr-restore` stops ZoPanel, replaces the whole panel database and `/etc/zopanel`, and starts ZoPanel again. The previous files are kept as `*.before-restore-<time>`.
- `zopanel ctl rebuild` has the panel apply every account and website again (nginx, PHP, limits) from the restored database.

If some accounts' data was not on the surviving disk, restore those accounts from S3 as in step 4 above.

## Account deleted by mistake

Deleted accounts are kept for the number of days set in **Settings → General → Keep deleted accounts (days)**: 7 by default, at most 90, and 0 deletes them at once. During that time the data is still on the server:

```bash
ls /home/.zp-terminated/        # <user>-<time>/home, databases/*.sql.gz, info.json
ls /var/vmail/.zp-terminated/   # mailboxes
```

To recover:

1. Create the account again with the same username and package.
2. Copy the website files back, working as the account.
3. Import the database dumps from **Databases → Import / restore**.
4. Create the mailboxes again, copy the Maildir folders back and run `chown -R vmail:vmail` on them.

## After a security incident

- Check **Security Center**: activity log integrity, malware and 2FA.
- Send the activity log to a remote syslog server (**Settings → General → Send the activity log to**), so an intruder cannot erase the record.