first commit
This commit is contained in:
commit
50936dea58
1 changed files with 441 additions and 0 deletions
441
README.md
Normal file
441
README.md
Normal file
|
|
@ -0,0 +1,441 @@
|
||||||
|
# zuzu-system-backup
|
||||||
|
|
||||||
|
Python-based backup tool for Linux hosts that pushes **snapshots** and **live mirrors** to a remote NAS over SSH/rsync.
|
||||||
|
|
||||||
|
- Snapshots are compressed archives (tar + optional zstd) created on the host, then uploaded.
|
||||||
|
- Mirrors are “single-copy” rsync trees kept in sync with `--delete`.
|
||||||
|
- Configuration is per-host, in YAML.
|
||||||
|
- Scheduling is done via systemd timers.
|
||||||
|
|
||||||
|
All hostnames, users, and paths in this README use **fictional example values** so it’s safe to publish as-is. Replace them with your own.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. High-level architecture
|
||||||
|
|
||||||
|
### 1.1 Components
|
||||||
|
|
||||||
|
- **Host script**: `zuzu-system-backup.py`
|
||||||
|
- Runs on each Linux host.
|
||||||
|
- Reads per-host `backup.yaml`.
|
||||||
|
- Builds local temp trees, compresses them, and uploads archives.
|
||||||
|
- Manages retention (max snapshot count).
|
||||||
|
- **Remote backup share** (NAS / backup server):
|
||||||
|
- Exposed via SSH (and optionally a mounted share).
|
||||||
|
- Receives snapshot archives and sync mirrors.
|
||||||
|
|
||||||
|
### 1.2 Snapshot vs mirror
|
||||||
|
|
||||||
|
The script supports two complementary backup modes:
|
||||||
|
|
||||||
|
1. **Snapshots** (versioned)
|
||||||
|
- Controlled by:
|
||||||
|
- `system.include_paths`
|
||||||
|
- `user.include_dirs`
|
||||||
|
- `user.include_files`
|
||||||
|
- On each run:
|
||||||
|
- rsync selected paths into a **local temp tree** per category.
|
||||||
|
- compress each category into a single `*.tar` or `*.tar.zst` archive.
|
||||||
|
- upload archives to a timestamped directory on the backup share.
|
||||||
|
|
||||||
|
2. **Single-copy mirrors** (no history)
|
||||||
|
- Controlled by `single_copy_mappings`.
|
||||||
|
- Each mapping is `"LOCAL_SRC|REMOTE_DEST"`.
|
||||||
|
- rsync with `--delete` keeps the remote tree in sync with local.
|
||||||
|
- Intended for large data (databases, model stores, etc.).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Data-flow diagrams
|
||||||
|
|
||||||
|
### 2.1 Backup flow on a single host
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
subgraph Host["Linux host (example: orion.example.net)"]
|
||||||
|
A[Snapshot sources\nsystem + user] --> B[Local temp trees\nsystem / user-dirs / user-files]
|
||||||
|
B --> C[Tar plus optional zstd\ncreate archives]
|
||||||
|
D[Mirror sources\nsingle_copy_mappings] --> E[rsync --delete]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph NAS["Backup share (example: backup-nas.local)"]
|
||||||
|
C --> F[Snapshots directory\n/daily/system-orion/<timestamp>/]
|
||||||
|
E --> G[Sync mirrors\n/sync/system-orion-*]
|
||||||
|
end
|
||||||
|
|
||||||
|
````
|
||||||
|
|
||||||
|
* Snapshots: archives are compressed **locally** and only the archives are sent.
|
||||||
|
* Mirrors: rsync goes directly from host paths to remote paths, with `--delete`.
|
||||||
|
|
||||||
|
### 2.2 Control flow inside the script
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
Start([systemd timer or manual run])
|
||||||
|
--> LCFG[Load backup.yaml]
|
||||||
|
--> PREP[Compute paths and compression root]
|
||||||
|
--> SNAPROOT[Ensure remote snapshots root exists]
|
||||||
|
--> SNAPNAME[Generate snapshot name\nYYYY-MM-DD_HH-MM-SS]
|
||||||
|
--> MKDIR_REMOTE[Create remote snapshot directory]
|
||||||
|
|
||||||
|
MKDIR_REMOTE --> TMPDIR[Create local temp root]
|
||||||
|
|
||||||
|
TMPDIR --> RSYNC_SYS[rsync system.include_paths\ninto temp/system]
|
||||||
|
TMPDIR --> RSYNC_UDIR[rsync user.include_dirs\ninto temp/user-dirs]
|
||||||
|
TMPDIR --> RSYNC_UFILE[rsync user.include_files\ninto temp/user-files]
|
||||||
|
|
||||||
|
RSYNC_SYS --> COMP_SYS[Compress system category\ncreate system archive]
|
||||||
|
RSYNC_UDIR --> COMP_UDIR[Compress user-dirs category\ncreate user-dirs archive]
|
||||||
|
RSYNC_UFILE --> COMP_UFILE[Compress user-files category\ncreate user-files archive]
|
||||||
|
|
||||||
|
COMP_SYS --> UPLOAD_SYS[Upload archives to remote snapshot directory]
|
||||||
|
COMP_UDIR --> UPLOAD_UDIR
|
||||||
|
COMP_UFILE --> UPLOAD_UFILE
|
||||||
|
|
||||||
|
UPLOAD_UFILE --> MIRRORS[Process mirrors\nsingle_copy_mappings with rsync --delete]
|
||||||
|
MIRRORS --> PRUNE[Prune old snapshots\nkeep at most N]
|
||||||
|
PRUNE --> CLEANUP[Remove temp tree and excludes file]
|
||||||
|
CLEANUP --> Done([Exit])
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Backup share layout
|
||||||
|
|
||||||
|
This project assumes a structured backup share on the NAS or backup server, for example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/srv/backup/
|
||||||
|
automated/
|
||||||
|
daily/
|
||||||
|
system-orion/
|
||||||
|
2025-01-01_03-00-01/
|
||||||
|
system.tar.zst
|
||||||
|
user-dirs.tar.zst
|
||||||
|
user-files.tar.zst
|
||||||
|
2025-01-02_03-00-01/
|
||||||
|
...
|
||||||
|
system-pegasus/
|
||||||
|
system-hera/
|
||||||
|
|
||||||
|
sync/
|
||||||
|
system-orion-databases/
|
||||||
|
... live rsync mirror ...
|
||||||
|
system-orion-models/
|
||||||
|
...
|
||||||
|
system-pegasus-luns/
|
||||||
|
...
|
||||||
|
|
||||||
|
manual/
|
||||||
|
installer-isos/
|
||||||
|
http-bookmarks/
|
||||||
|
license-keys/
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Typical patterns:
|
||||||
|
|
||||||
|
* **automated/daily/system-<host>/**
|
||||||
|
Date-based snapshot directories, each containing only a few archives.
|
||||||
|
* **automated/sync/system-<host>-<purpose>/**
|
||||||
|
One dir per mirror, updated in place.
|
||||||
|
* **manual/**
|
||||||
|
Hand-managed backups, not touched by this tool.
|
||||||
|
|
||||||
|
You can adapt the root (`/srv/backup`) and naming (`system-orion`, etc.) to match your environment.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Features
|
||||||
|
|
||||||
|
* Python 3 script, no shell gymnastics.
|
||||||
|
* YAML configuration, per-host.
|
||||||
|
* Snapshot categories:
|
||||||
|
|
||||||
|
* `system.include_paths` (root-owned config and service dirs)
|
||||||
|
* `user.include_dirs` (full user trees, relative to `user.home`)
|
||||||
|
* `user.include_files` (one-off important files)
|
||||||
|
* Compression modes:
|
||||||
|
|
||||||
|
* `high` (zstd `-19`)
|
||||||
|
* `light` (zstd `-3`)
|
||||||
|
* `none` (plain `.tar`)
|
||||||
|
* Local compression only:
|
||||||
|
|
||||||
|
* No dependency on the NAS having zstd or GNU tar.
|
||||||
|
* Single-copy mirrors:
|
||||||
|
|
||||||
|
* Declarative `"LOCAL_SRC|REMOTE_DEST"` mappings.
|
||||||
|
* Retention:
|
||||||
|
|
||||||
|
* Keep at most `retention.snapshots` snapshot directories per host.
|
||||||
|
* Systemd integration:
|
||||||
|
|
||||||
|
* Ones-shot service + timer per host.
|
||||||
|
* Logging:
|
||||||
|
|
||||||
|
* Structured, timestamped logs via `journalctl`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Dependencies
|
||||||
|
|
||||||
|
On each host:
|
||||||
|
|
||||||
|
* `python` (3.x)
|
||||||
|
* `python-yaml` (PyYAML)
|
||||||
|
* `rsync`
|
||||||
|
* `openssh`
|
||||||
|
* `tar`
|
||||||
|
* `zstd` (optional but strongly recommended if using `compression.mode: high|light`)
|
||||||
|
|
||||||
|
Example (Arch-based host):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo pacman -S python python-yaml rsync openssh tar zstd
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Configuration (`backup.yaml`)
|
||||||
|
|
||||||
|
Each host has its own `backup.yaml` next to the script.
|
||||||
|
|
||||||
|
### 6.1 Schema overview
|
||||||
|
|
||||||
|
* `remote`: where to send backups
|
||||||
|
* `retention`: how many snapshots to keep
|
||||||
|
* `compression`: how to compress snapshot archives
|
||||||
|
* `rsync`: extra rsync flags
|
||||||
|
* `system`: system-level include paths
|
||||||
|
* `user`: user home and per-user include paths/files
|
||||||
|
* `exclude_patterns`: rsync-style excludes
|
||||||
|
* `single_copy_mappings`: one-way mirrors (no history)
|
||||||
|
|
||||||
|
### 6.2 Example `backup.yaml` (for host `orion`)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
remote:
|
||||||
|
user: backupuser
|
||||||
|
host: backup-nas.local
|
||||||
|
port: 22
|
||||||
|
key: /home/backupuser/.ssh/id_ed25519-orion
|
||||||
|
base: /srv/backup/automated
|
||||||
|
host_dir: system-orion
|
||||||
|
|
||||||
|
retention:
|
||||||
|
# Max number of snapshot directories to keep on NAS
|
||||||
|
snapshots: 7
|
||||||
|
|
||||||
|
compression:
|
||||||
|
# high | light | none
|
||||||
|
mode: high
|
||||||
|
# Optional: where local temp trees and archives live
|
||||||
|
path: /srv/tmp/backups
|
||||||
|
|
||||||
|
rsync:
|
||||||
|
extra_opts:
|
||||||
|
- --numeric-ids
|
||||||
|
- --info=progress2
|
||||||
|
- --protect-args
|
||||||
|
|
||||||
|
system:
|
||||||
|
include_paths:
|
||||||
|
- /etc/nftables.conf
|
||||||
|
- /etc/snapper/configs
|
||||||
|
- /etc/NetworkManager/system-connections
|
||||||
|
- /etc/chromium/policies/managed
|
||||||
|
- /etc/fstab
|
||||||
|
- /etc/systemd/system/*.mount
|
||||||
|
- /etc/systemd/system/*.automount
|
||||||
|
- /etc/nut/nut.conf
|
||||||
|
- /etc/nut/upsmon.conf
|
||||||
|
|
||||||
|
user:
|
||||||
|
home: /home/devuser
|
||||||
|
|
||||||
|
include_dirs:
|
||||||
|
- .ssh
|
||||||
|
- .gnupg
|
||||||
|
- .local/share/wallpapers
|
||||||
|
- projects
|
||||||
|
- pkgbuilds
|
||||||
|
- venvs
|
||||||
|
|
||||||
|
include_files:
|
||||||
|
- .config/chromium/Default/Preferences
|
||||||
|
- .config/chromium/Default/Bookmarks
|
||||||
|
- .config/vlc/vlcrc
|
||||||
|
- .gitconfig
|
||||||
|
- .bashrc
|
||||||
|
- .bash_profile
|
||||||
|
- .local/share/user-places.xbel
|
||||||
|
|
||||||
|
exclude_patterns:
|
||||||
|
# Caches (generic)
|
||||||
|
- "**/Cache/**"
|
||||||
|
- "**/GPUCache/**"
|
||||||
|
- "**/shadercache/**"
|
||||||
|
- "**/ShaderCache/**"
|
||||||
|
- "**/Code Cache/**"
|
||||||
|
|
||||||
|
# SSH ControlMaster sockets
|
||||||
|
- "${USER_HOME}/.ssh/ctl-*"
|
||||||
|
- "**/.ssh/ctl-*"
|
||||||
|
|
||||||
|
# JetBrains bulk (plugins + Toolbox app bundles)
|
||||||
|
- "${USER_HOME}/.local/share/JetBrains/**/plugins/**"
|
||||||
|
- "${USER_HOME}/.local/share/JetBrains/Toolbox/apps/**"
|
||||||
|
- "${USER_HOME}/.cache/JetBrains/**"
|
||||||
|
|
||||||
|
# Chromium bulk (we include only specific files above)
|
||||||
|
- "${USER_HOME}/.config/chromium/**"
|
||||||
|
|
||||||
|
single_copy_mappings:
|
||||||
|
# Example mirrors:
|
||||||
|
- "/srv/data/postgres|/srv/backup/automated/sync/system-orion-postgres"
|
||||||
|
- "/srv/data/models|/srv/backup/automated/sync/system-orion-models"
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
|
||||||
|
* `user.include_dirs` and `user.include_files` are **relative to `user.home`** unless they start with `/`.
|
||||||
|
* `${USER_HOME}` and `${HOME}` in `exclude_patterns` are expanded to `user.home` by the script.
|
||||||
|
* `single_copy_mappings` paths are *not* expanded; use absolute paths.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Script usage
|
||||||
|
|
||||||
|
### 7.1 Manual run
|
||||||
|
|
||||||
|
From the directory where the script lives, or via its full path:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo /usr/local/sbin/zuzu-system-backup/zuzu-system-backup.py
|
||||||
|
```
|
||||||
|
|
||||||
|
(or whatever path you install it to)
|
||||||
|
|
||||||
|
Logs go to stderr; under systemd, they land in `journalctl`.
|
||||||
|
|
||||||
|
### 7.2 Systemd service & timer
|
||||||
|
|
||||||
|
Example service:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
# /etc/systemd/system/host-backup.service
|
||||||
|
[Unit]
|
||||||
|
Description=Host backup to NAS via zuzu-system-backup
|
||||||
|
Wants=network-online.target
|
||||||
|
After=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
ExecStart=/usr/local/sbin/zuzu-system-backup/zuzu-system-backup.py
|
||||||
|
Nice=10
|
||||||
|
IOSchedulingClass=best-effort
|
||||||
|
IOSchedulingPriority=7
|
||||||
|
```
|
||||||
|
|
||||||
|
Example timer:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
# /etc/systemd/system/host-backup.timer
|
||||||
|
[Unit]
|
||||||
|
Description=Nightly host backup to NAS
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnCalendar=*-*-* 03:15:00
|
||||||
|
RandomizedDelaySec=20min
|
||||||
|
Persistent=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
|
```
|
||||||
|
|
||||||
|
Enable and start:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable --now host-backup.timer
|
||||||
|
```
|
||||||
|
|
||||||
|
Check status:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl list-timers 'host-backup*'
|
||||||
|
journalctl -u host-backup.service -n 50
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Retention policy
|
||||||
|
|
||||||
|
Retention is implemented as “keep at most **N snapshots**” for each host:
|
||||||
|
|
||||||
|
* `retention.snapshots: 7` → keep the newest 7 snapshot directories under `REMOTE_BASE/host_dir/snapshots`.
|
||||||
|
* Snapshot directory names are timestamps: `YYYY-MM-DD_HH-MM-SS`.
|
||||||
|
* Older snapshot dirs are deleted entirely (`rm -rf` on the NAS via SSH).
|
||||||
|
|
||||||
|
No time math on `mtime`, just count-based retention by sorted timestamp name; simple and predictable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Restore basics
|
||||||
|
|
||||||
|
### 9.1 Restoring from a snapshot archive
|
||||||
|
|
||||||
|
On the NAS or after copying archives locally:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Example: restore system snapshot for 2025-01-02 from host "orion"
|
||||||
|
cd /restore/target
|
||||||
|
|
||||||
|
# If compressed
|
||||||
|
zstd -d /srv/backup/automated/daily/system-orion/2025-01-02_03-00-01/system.tar.zst -o system.tar
|
||||||
|
tar -xf system.tar
|
||||||
|
|
||||||
|
# If compression.mode was "none"
|
||||||
|
tar -xf /srv/backup/automated/daily/system-orion/2025-01-02_03-00-01/system.tar
|
||||||
|
```
|
||||||
|
|
||||||
|
Repeat for `user-dirs.tar(.zst)` and `user-files.tar(.zst)` as needed.
|
||||||
|
|
||||||
|
### 9.2 Restoring from a mirror
|
||||||
|
|
||||||
|
Mirrors are just rsynced trees; you can restore them with rsync or cp:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# rsync mirror back to host
|
||||||
|
rsync -aHAX --numeric-ids \
|
||||||
|
backupuser@backup-nas.local:/srv/backup/automated/sync/system-orion-postgres/ \
|
||||||
|
/srv/data/postgres/
|
||||||
|
```
|
||||||
|
|
||||||
|
Always test restores on a non-production target first.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Safety notes
|
||||||
|
|
||||||
|
* **Mirrors are destructive:**
|
||||||
|
|
||||||
|
* `single_copy_mappings` use rsync `--delete`.
|
||||||
|
* Deletes on the host will remove files on the backup side in the next run.
|
||||||
|
* **Snapshots are immutable per run:**
|
||||||
|
|
||||||
|
* Each run creates a new directory, writes archives, and then retention may remove older snapshot dirs.
|
||||||
|
* **Local compression uses space:**
|
||||||
|
|
||||||
|
* `compression.path` should point at a filesystem with enough free space to hold a full snapshot’s uncompressed temp trees **plus** the compressed archives.
|
||||||
|
* **Permissions:**
|
||||||
|
|
||||||
|
* The script expects to be run as root (or with enough privileges) to read system paths and user homes.
|
||||||
|
* **SSH keys:**
|
||||||
|
|
||||||
|
* Use dedicated SSH keys per host with restricted accounts on the NAS where possible.
|
||||||
|
|
||||||
|
---
|
||||||
Loading…
Reference in a new issue