Ch 26: How to create and manage backups

In this chapter, we will explore how to create and manage backups. It is important to develop the habit of making regular backups to protect your data on your Linux System. Some of the tools include rsync, tar, and rclone. Learn to automate and secure your files against loss.

Summary

In this chapter, we will explore how to create and manage backups to protect your data on your Linux system. It is essential to develop the habit of making regular backups, and this guide will introduce you to tools like rsync, tar, and rclone. You’ll also learn how to automate and secure your backups against data loss.

Learning Objectives: Understand backup types, use Linux backup tools, automate backups, and troubleshoot issues.

Why Backups Matter

Backups protect against data loss from hardware failures, human errors, or cyberattacks. For Linux users, they ensure quick recovery of personal files, scripts, or server data, offering peace of mind.

Early in my Linux journey, I lost a week’s worth of scripts when a hard drive crashed. Without a backup, I had to rewrite everything from scratch. A simple rsync command could’ve saved me! This taught me the value of regular backups, a lesson I now share with you.

Types of Backups

• Full Backups: Copy all data (e.g., tar archives).
• Incremental Backups: Copy only changed files (e.g., rsync).
• Differential Backups: Copy changes since the last full backup.

Use Cases:
– Home users: Full backups for photos, documents.
– Servers: Incremental backups for logs, databases.

Linux Backup Tools

Linux offers several tools to create and manage backups. Below, we’ll explore the most popular ones.

rsync

rsync efficiently syncs files locally or remotely, supporting incremental transfers, compression, and permission preservation.

tar

tar creates compressed archives (e.g., .tar.gz), making it simple and widely supported for full backups.

For more on archiving and compression, see related post: Archive and Compress Files in Linux.

Other Tools

• dd: Disk imaging for full system backups.
• restic: Modern, encrypted backups with deduplication.

Setting Up Backups

Let’s dive into how to use these tools to create and manage backups.

Using rsync

Sync a local directory:

$ rsync -avz /home/user/docs /backup/docs

Remote backup (e.g., to a server):

$ rsync -avz /home/user/docs user@remote:/backup/docs

Using tar

Create a compressed archive:

$ tar -czf backup.tar.gz /home/user/docs

Extract the archive:

$ tar -xzf backup.tar.gz

Using rclone

rclone is a powerful command-line tool for syncing files to cloud storage (e.g., Google Drive, Dropbox), making it ideal for backups.

Installing rclone

$ sudo apt install rclone

Configuring rclone

Set up a remote storage provider (e.g., Google Drive):

$ rclone config

Follow the prompts to authenticate and name the remote (e.g., mygdrive).

Backing Up with rclone

Sync a local directory to cloud storage:

$ rclone sync /home/user/docs mygdrive:/backups/docs --progress

Flags: --progress shows transfer status.

Restoring Files

Restore files from cloud storage:

$ rclone sync mygdrive:/backups/docs /home/user/restored_docs --progress

Example: Cloud Backup

Back up /home/user/docs to Google Drive, excluding temporary files:

$ rclone sync /home/user/docs mygdrive:/backups/docs --progress --exclude "*.tmp"

Troubleshooting rclone

• Authentication Errors: Re-run rclone config to refresh tokens.
• Network Issues: Check connectivity with ping google.com.
• Log Output: Add --log-file=/var/log/rclone.log for debugging.

Example: Debug a failed sync:

$ rclone sync /home/user/docs mygdrive:/backups/docs --dry-run --log-file=/var/log/rclone.log
$ tail /var/log/rclone.log

Best Practices

• Test syncs with --dry-run to avoid accidental overwrites.
• Schedule backups with cron (see Chapter 24).
• Encrypt sensitive data with rclone crypt.
• Use AI to generate rclone commands or parse logs for errors.

Automating Backups with cron

Schedule a daily rsync backup at midnight (see Chapter 24 for more on cron):

0 0 * * * /usr/bin/rsync -avz /home/user/docs /backup/docs >> /var/log/backup.log 2>&1

Tip: Test cron jobs with short intervals first.

Best Practices

• 3-2-1 Rule: 3 copies, 2 local media, 1 offsite.
• Test restores: $ tar -xzf backup.tar.gz.
• Encrypt sensitive data in duplicati.
• Monitor logs for errors.
• Use AI to generate backup scripts or verify cron schedules.

Practical Examples

Local Backup

Sync /var/www to an external drive:

$ rsync -avz /var/www /mnt/external/www

Remote Backup

Backup to a server:

$ rsync -avz /home/user/docs user@192.168.1.100:/backup/docs

Cloud Backup

Use rclone to back up to Google Drive (see above).

Restoring Data

Recover from tar:

$ tar -xzf backup.tar.gz -C /restore

Troubleshooting Backups

• Permission Errors: Check user access with ls -l.
• Network Failures: Test connectivity with ping.
• Corrupt Archives: Verify with $ tar -tzf backup.tar.gz.

Example: Debug rsync:

$ rsync -avz --dry-run /home/user/docs /backup/docs
$ tail -f /var/log/backup.log

---

How to Create and Manage Backups with Rsnapshot

Rsnapshot is a command-line tool that leverages rsync and hard links to create incremental backups, making it efficient for both personal and server use. Below, we’ll explore how to create and manage backups using rsnapshot.

Background

Traditional full backups consume significant disk space. Rsnapshot addresses this by creating incremental backups: the first backup is a full copy, and subsequent backups store only changes, using hard links to reference unchanged files. This results in a series of point-in-time snapshots while minimizing storage usage.

Installation on Debian

$ sudo apt update
$ sudo apt install rsnapshot

Basic Commands

• rsnapshot configtest: Test configuration file.
• rsnapshot daily: Run daily backup.
• rsnapshot weekly: Run weekly backup.
• rsnapshot monthly: Run monthly backup.
• rsnapshot hourly: Run hourly backup.

Configuration Sample (/etc/rsnapshot.conf)

config_version        1.2
snapshot_root         /var/cache/rsnapshot/
interval              daily   6
interval              weekly  4
backup                /home/           localhost/
backup                /etc/            localhost/
#exclude               /home/*/.cache/

Key Configuration Notes:

• snapshot_root: Base directory for snapshots.
• interval: Backup frequency and retention (e.g., keep 6 daily backups).
• backup: Source directory and destination (e.g., localhost/ for local backups).
• exclude: Files/directories to exclude (e.g., /home/*/.cache/).

Use Cases

Backup a Project Folder

Add to rsnapshot.conf:

backup  /home/user/myproject/  localhost/myproject/

Run rsnapshot daily to create a snapshot.

Backup a Single File

Add to rsnapshot.conf:

backup  /etc/nginx/nginx.conf  localhost/nginx.conf

Run rsnapshot daily.

Incremental Backups

Subsequent runs only copy changed files, using hard links for unchanged data.

Excluding Files

Add to rsnapshot.conf:

backup                /home/user/myproject/  localhost/myproject/
exclude               /home/user/myproject/.git/

Scheduling (Cron)

Schedule daily backups at 2:00 AM via crontab -e:

0 2 * * *   root    /usr/bin/rsnapshot daily

Important Considerations

• Secure SSH for remote backups.
• Monitor disk space.
• Test backups regularly.
• Implement error handling.

Tips for Rsnapshot

• Regularly review and update your rsnapshot configuration.
• Test your backups to ensure they’re working correctly.
• Consider using a remote backup destination for added security.

Rsnapshot offers a reliable solution for creating and managing backups with careful configuration and monitoring.

---

Understanding Encryption for Backups

Encryption is crucial for securing sensitive data in backups. Below, we cover core concepts and tools for encrypting files.

Concepts

• Encryption: Converting plaintext into ciphertext.
• Plaintext: Unencrypted data.
• Ciphertext: Encrypted data.
• Key: Secret string used to encrypt/decrypt data.
• Symmetric Encryption: Same key for encryption/decryption (e.g., AES).
• Asymmetric Encryption: Public/private key pair (e.g., RSA).

Encrypting Files with OpenSSL

Encrypt a file using AES-256-CBC:

$ openssl enc -aes-256-cbc -in example.txt -out example.enc

Enter a password when prompted.

Encrypting Files with GPG

Encrypt a file with GPG:

$ gpg -c example.txt

Decrypt with:

$ gpg example.txt.gpg

Enter the password to decrypt.

Practice Exercises

1. Encrypt hello.txt using OpenSSL (AES-256-CBC).
2. Decrypt the encrypted file.
3. Encrypt secret.txt using GPG.
4. Decrypt the encrypted file.

---

How to Create and Manage Backups Using Rclone Crypt

Rclone Crypt provides end-to-end encryption for your backups, ensuring data security when stored in the cloud. This section will guide you through creating and managing backups with encryption.

Configuration

First, configure rclone for your cloud provider (e.g., Google Drive):

$ rclone config

Follow the prompts to set up a remote (e.g., mygdrive).

Rclone Crypt Configuration Example

Add a crypt remote to encrypt data:

[rclone-crypt]
type = crypt
remote = mygdrive:/Crypt
filename_encryption = standard
directory_name_encryption = false
password = your_secret_password

Encrypting Files

Encrypt a file:

$ rclone crypt -v encrypt example.txt

Uploading Encrypted Files

Upload the encrypted file to the cloud:

$ rclone copy example.txt.crypt rclone-crypt:/

Downloading Encrypted Files

Download the encrypted file:

$ rclone copy rclone-crypt:/example.txt.crypt .

Decrypting Files

Decrypt the file:

$ rclone crypt -v decrypt example.txt.crypt

Full Folder Encryption

Encrypt an entire folder (e.g., FolderA):

$ rclone crypt -v encrypt FolderA

Upload the encrypted folder:

$ rclone copy FolderA.gcrypt rclone-crypt:/

Download and decrypt in another folder (e.g., FolderB):

$ rclone copy rclone-crypt:/FolderA.gcrypt FolderB
$ rclone crypt -v decrypt FolderB

Tips and Variations

• Use rclone sync instead of rclone copy for ongoing backups.
• Encrypt multiple files: rclone crypt -v encrypt *
• Decrypt multiple files: rclone crypt -v decrypt *

Practice Exercises

1. Set up rclone crypt for your cloud provider.
2. Encrypt a file and upload it.
3. Download and decrypt the file.
4. Encrypt and upload an entire folder.
5. Retrieve and decrypt the folder.

---

Practice Time

Test your skills:

1. Create a full backup with tar.
2. Set up an incremental backup with rsync.
3. Schedule a daily backup using cron.
4. Restore a file and verify integrity.

Try This: Run rsync -avz /home/user/docs /backup and share your success on X with #LinuxCommandLine!

Backup Command Reference

Command	Description
rsync -avz	Syncs files with compression.
tar -czf	Creates compressed archive.
tar -xzf	Extracts archive.

Glossary of Terms

Term	Description
Backups	Data copies for recovery.
rsync	File synchronization tool.
tar	Archiving tool.
Incremental Backup	Copies only changed files.

Reference: See Linux Manpages for details.

Conclusion

You’ve learned how to create and manage backups in Linux using rsync, tar, and rclone. Automate with cron and secure your data with encryption. Practice these skills to safeguard your Linux system. For more on file transfer, see our post: File Transfer in Linux. Next, secure your network with iptables.

---

Previous: Chapter 25 | Next: Chapter 27