Backup with rsync and scp

1. Why is backup important?

Today we’re going to chat about one of the most important tasks for anyone working in IT and programming—data backup. If you’ve ever lost important files due to a drive failure or an error, you know how critical this is. And if you haven’t—well, that’s why we’re here, to make sure it doesn’t happen!

Imagine this: you’ve been grinding on a project for months, and then one day your hard drive decides to go on strike. Yeah, it’s as frustrating as forgetting a semicolon at the end of an SQL query, only a thousand times worse. Systems can fail, people make mistakes, and files get lost. Backup is how we prepare for those nasty surprises ahead of time.

In Linux, we’ve got a few powerful tools for backups. Today, we’ll get to know two of them: rsync and scp.

---

2. The rsync command: a fast and smart tool for copying data

rsync is one of the most popular tools for copying and syncing data. Its main strength is speed and efficiency. Instead of copying everything, rsync only copies modified files. This makes it perfect for regular backups of large amounts of data.

Syntax of rsync

Here’s the basic format of the command:

rsync [options] source destination

Where:

• source — the path to the file or folder you want to copy.
• destination — where you want the data delivered.

Main options and their functions

Here are some commonly used options:

• -a (archive): includes copying all metadata (permissions, timestamps, etc.).
• -v (verbose): enables detailed output, so you can see what’s happening.
• --progress: shows the progress of copying (super handy when copying large files).
• --delete: removes files on the destination side that no longer exist on the source side. Useful for full synchronization.
• -z (compress): compresses data before transferring, speeding things up (especially important for network operations).

Example: copying local data

Let’s create a directory /backup to store backups.

mkdir /backup

Copy all the contents of the directory /home to /backup:

rsync -av /home /backup

Breaking down this example:

• -a preserves the structure and permissions of the files.
• -v gives info about the process.

If you run the command again, it will only copy files that are new or have changed.

Example: remote backup

Now let’s say you want to send a backup to a remote server. For this, rsync supports using SSH.

rsync -av /backup user@remote_server:/remote_backup

Where:

• user — the username on the remote server.
• remote_server — the address of your remote server (e.g., IP address or domain name).
• /remote_backup — the path on the server where you want the data saved.

If you have an SSH key for authorization, the transfer will be even easier (and safer).

---

3. The scp Command: Simple and Efficient

scp (Secure Copy Protocol) is a tool for copying files between a local and a remote machine. It's a little simpler than rsync and works for cases where full synchronization isn't needed.

scp Syntax

Here’s what the basic format of the command looks like:

scp [options] source destination

Where:

• source — the path to the file you want to copy.
• destination — the address where it should be delivered.

Main Options and Their Functions

Some useful options:

• -r: recursive copying for directories.
• -C: compresses files before transferring.
• -P: specifies the SSH port if it's different from the default (22).

Example: Copying a File to a Remote Machine

You want to transfer files from your local computer to a server. Let’s say the file is named document.txt.

scp document.txt user@remote_server:/remote_folder

If the file is located in the folder /home/user/documents, the command will look like this:

scp /home/user/documents/document.txt user@remote_server:/remote_folder

Now the file will be available on the remote machine in the folder /remote_folder.

Example: Copying Files from the Server to a Local Machine

The process works in reverse:

scp user@remote_server:/remote_folder/document.txt /home/user/documents

In this case, the document.txt file will end up in your local folder.

---

4. Differences Between rsync and scp

Even though both commands let you copy files, they have their own features.

So, if you just need a quick file transfer, go with scp. If we're talking regular syncs and large data volumes, it's rsync all the way.

---

5. Real Example: Setting Up a Backup

Here's a practical task: you want to set up a complete backup process for your project to a remote server.

1. Let's say your project files are located in the folder /home/user/project. You want them to be backed up on the server backup.server.com in the folder /backups/project.

2. To make the copying faster, we'll use rsync.

Setting Up the Backup Script

Create a file called backup.sh:

#!/bin/bash

SOURCE="/home/user/project"
DESTINATION="user@backup.server.com:/backups/project"

# Perform backup using rsync
rsync -av --delete "$SOURCE" "$DESTINATION"

# Output a completion message
echo "Backup completed!"

Make the file executable:

chmod +x backup.sh

Now you can run this script manually:

./backup.sh

Or add it to cron for automatic execution (we'll talk about cron later in the course).

---

6. Common mistakes and how to avoid them

1. Incorrect paths in the command line. If you specify the wrong path to files, rsync or scp won't copy anything. Use commands like ls or pwd to make sure the paths are correct.

2. Missing authorization key. If you're using SSH without a configured key, you'll constantly be prompted for a password. Set up SSH keys to make the process smoother.

3. Interrupted backup. If the network is unstable, rsync will still complete the transfer the next time you run it. However, with scp, you'll have to start over.

With rsync and scp, you can confidently copy and sync your data both on your local machine and remote servers. These tools are your trusty go-to when it comes to safeguarding important files.