Quickstart Guide for Restic and Backblaze B2 Cloud Storage Quickstart Guide for Restic and Backblaze B2 Cloud Storage

Quickstart Guide for Restic and Backblaze B2 Cloud Storage

Permanently deleted user Permanently deleted user

Introduction

Restic is an open-source backup tool that works with local storage, NAS devices, or cloud storage (Backblaze B2). It uses snapshots, block-level deduplication, along with complex indexing to allow for quick restores. All this while taking minimal storage space.  One restic repository can be used to contain the snapshots from one host, or multiple, as the host information is stored in the snapshot index also.

 

This guide will show you how to configure restic to backup to B2 Cloud Storage, using our S3 API.  When you create your bucket, you will see detailed information, including your endpoint URL (ex. s3.us-west-002.backblazeb2.com).  Please make a note of it as you will need it when defining the RESTIC_REPOSITORY variable below.

 

Installing Restic

Restic can be installed from source code or binaries which can be downloaded from GitHub, or installed using the a local package manager on most operating systems.  While Restic is available on Windows, some features are not available.  

 

This article will focus on the Linux and Mac versions.  

 

Redhat/CentOS

sudo yum install restic

Ubuntu/Debian

sudo apt install restic

Mac

brew install restic

Note about fuse: fuse is needed to be able to mount the snapshot backups, which is one of the restore options. fuse comes installed with many linux distributions by default. If yours does not, then install it to enable this restore option.

 

On Mac, use OSXFuse 3.10.6 if you're running Catalina (10.15), otherwise, you can use the current version of OSXFuse (now called MacFuse) if you are on BigSur. 

 

After installation, you can make sure restic is on the latest version by running this command:

restic self-update

 

Configuring Restic

To simplify the use of restic, it’s best to define the restic environment variables you will need in a file, such as /etc/restic-env.  This will eliminate the need to pass every parameter each time we run restic.  

export AWS_ACCESS_KEY_ID=<B2_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<B2_ApplicationKey>
export RESTIC_REPOSITORY="s3:s3.us-west-002.backblazeb2.com/s3restic2023"
export RESTIC_PASSWORD_FILE=/etc/restic-password

Then /etc/restic-password could contain 1 line with the password

mYsEcureP@$$word

Secure the restic files so only root or a user you create can see the files

chown root:root /etc/restic-env
chown root:root /etc/restic-password
chmod 700 /etc/restic-env
chmod 700 /etc/restic-password

 

Before we run any Restic commands we need to load the environment variables with this command: This command can be added to your login profile (ex. ~.bashrc) so the restic variables are always defined.

source /etc/restic-env

 

Initialize the repository (repo)

source /etc/restic-env
restic -r s3:s3.us-west-002.backblazeb2.com/s3restic2023 init

created restic repository 1f669fd85e at s3:s3.us-west-002.backblazeb2.com/s3restic2023

Please note that knowledge of your password is required to access the repository. Losing your password means that your data is

irrecoverably lost.

 

Backing up with Restic

To backup the /etc directory

restic -r s3:s3.us-west-002.backblazeb2.com/s3restic2023 backup /etc
repository 1f669fd8 opened (version 2, compression level auto)
no parent snapshot found, will read all files

 

Example Output. 

Files:         872 new,     0 changed,     0 unmodified
Dirs:          257 new,     0 changed,     0 unmodified

Added to the repository: 3.369 MiB (1.097 MiB stored

processed 872 files, 2.788 MiB in 0:04
snapshot 20ee6d7b saved

 

Since we have defined the RESTIC_REPOSITORY variable, we actually don’t need to add -r s3:s3.us-west-002.backblazeb2.com/s3restic2023 to our restic commands.

 

The second backup only backs up changed files, since it uses block-level deduplication.  No changed files yet, so no files backed up.

restic backup /etc
repository 1f669fd8 opened (version 2, compression level auto)
using parent snapshot 20ee6d7b

Files:           0 new,     0 changed,   872 unmodified
Dirs:            0 new,     0 changed,   257 unmodified
Added to the repository: 0 B   (0 B   stored)

processed 872 files, 2.788 MiB in 0:02
snapshot 17bd5648 saved

 

Backing up with tags is a useful way to identify your snapshots, identify which hosts they are from, and later prune unneeded snapshots.  In this example, I am backing up a Windows SMB share on my Linux machine.

 

restic --tag Windows backup /mnt/WindowsData
repository 1f669fd8 opened (version 2, compression level auto)
no parent snapshot found, will read all files

Files:         388 new,     0 changed,     0 unmodified
Dirs:           20 new,     0 changed,     0 unmodified
Added to the repository: 38.595 GiB (36.520 GiB stored)

processed 388 files, 42.157 GiB in 13:43
snapshot e230caa6 saved

 

 

List all Restic Backups

To see a list of all the snapshots  you have done, with their data and time stamps, as well as optional tags, use the following command:

restic snapshots
repository 1f669fd8 opened (version 2, compression level auto)
ID        Time                 Host        Tags        Paths
-----------------------------------------------------------------------
20ee6d7b  2023-07-19 05:59:35  restic-s3               /etc
17bd5648  2023-07-19 06:00:34  restic-s3               /etc
e230caa6  2023-07-19 06:07:16  restic-s3   Windows     /mnt/WindowsData
-----------------------------------------------------------------------
3 snapshots

 

 

Restoring from a Restic Backup stored in B2 

 

Method #1: Restoring a snapshot to a directory

To restore a snapshot to a directory, you need to supply the snapshot id, and specify the target directory.  Restic will restore all files from the backup, with their full paths, starting under that directory.

restic restore 20ee6d7b  --target /tmp/restore
repository 1f669fd8 opened (version 2, compression level auto)
restoring <Snapshot 20ee6d7b of [/etc] at 2023-07-19 05:58:07.145565492 -0700 PDT by root@restic-s3> to /tmp/restore

 

Method #2: Mount and browse the snapshot

Another method to restore files, is to mount the snapshot database, browse to the backup you want, and copy files from the mount point to any destination you like.  To do this, we first need to create a mount point for the restic snapshots.

mkdir /mnt/restic

 

Then we can mount the snapshots and browse them.  When you run the mount command,  you will need to start another ssh or terminal session, or you can run the mount command in the background by appending the & sign.

restic mount /mnt/restic &
ls

hosts ids  snapshots  tags

 

You can browse backups by the host they were backed up from, the snapshot id, the date/time stamp, or the tags. Go into the directory of the Windows Snapshot using the ID

cd /mnt/restic/ids/e230caa6
ls

mnt

cd mnt/WindowsData/
pwd

/mnt/restic/ids/e230caa6/mnt/WindowsData

 

To copy a powershell script, install-choco.ps1, to tmp, we simply enter a normal copy command.

cp install-choco.ps1 /tmp

 

Running Restic in Docker

To install the official Restic docker container, use this command:

docker pull restic/restic:latest

 

Prepare for your first Restic backup

Create an environment file, such as /etc/restic_env, and define the following Restic variables.

  • RESTIC_DATA
    • Your data directory in the Docker image that will be mapped to your local directory you want to back up
  • RESTIC_REPOSITORY=s3:s3.us-west-002.backblazeb2.com/s3restic2023
  • RESTIC_PASSWORD
    • Your Restic repository password
  • B2_ACCOUNT_ID
  • B2_ACCOUNT_KEY

 

Example:

RESTIC_DATA=/data
RESTIC_REPOSITORY=s3:s3.us-west-002.backblazeb2.com/s3restic2023
RESTIC_PASSWORD=mYsEcureP@$$word
AWS_ACCESS_KEY_ID=<B2_KEY_ID>
AWS_SECRET_ACCESS_KEY=<B2_ApplicationKey>

 

Creating a local data directory

 

Create a local data directory to store your data to be backed up, or use an existing directory in the docker command below.  Just substitute  “$HOME/restic-data” with whatever directory you want to back up.

mkdir $HOME/restic-data

 

Docker Flags Used Explained

--rm          Automatically remove the container when it exits

-t            Allocate a pseudo-TTY

-i            Keep STDIN open even if not attached

--entrypoint  Overwrite the default ENTRYPOINT of the image

--env-file   Define the environment variable file to use

-v            This will mount the local directory to a directory in docker.

 

Initialize the Restic repository

Before the first backup, the repository (repo) needs to be initialized to create the structure needed for all the backup files.

docker run --rm -ti  --env-file=/etc/restic-env   \

-v $HOME/restic-data:/data restic/restic init

 

Backup local data to the repo

To back up the local $HOME/restic-data, use the following command .

docker run --rm -ti --env-file=/etc/restic-env  \

-v $HOME/restic-data:/data     restic/restic backup /data



repository 15827d78 opened successfully, password is correct

created new cache in /root/.cache/restic

no parent snapshot found, will read all files




Files:     165 new, 0 changed, 0 unmodified

Dirs:       46 new, 0 changed, 0 unmodified

Added to the repo: 6.346 GiB

 

List files in the latest backup

To list all files in the latest backup, use the below command.  This can be a lengthy list, so use with caution.

docker run --rm -ti    --env-file=/etc/restic-env   \

-v $HOME/restic-data:/data     restic/restic ls -l latest

 

List all snapshots

To see a list of all the snapshots  you have done, with their data and time stamps, as well as optional tags, use this command:

docker run --rm -ti  --env-file=/etc/restic-env \

-v $HOME/restic-data:/data  restic/restic  snapshots

 

Restore a backup

To restore a snapshot to a directory, you need to supply the snapshot id and specify the target directory.  Restic will restore all files from the backup, with their full paths, starting under that directory.

docker run --rm -ti  --env-file=/etc/restic-env \

-v $HOME/restic-data:/data -v $HOME/restic-restore:/restore  \

restic/restic  restore 5d1487e5 --target /restore




repository 15827d78 opened successfully, password is correct

created new cache in /root/.cache/restic

restoring <Snapshot 5d1487e5 of [/data] at 2021-07-01 15:11:37.568367623 +0000 UTC by root@e1701517be05> to /restore

 

You can also tell Restic to restore the latest backup using this command:

docker run --rm -ti  --env-file=/etc/restic-env \

-v $HOME/restic-data:/data -v $HOME/restic-restore:/restore  \

restic/restic  restore latest --target /restore



Check the health of the repository

Checking the health of your Restic repository periodically is a good idea. From the Manual “The "check" command tests the repository for errors and reports any errors it finds. It can also be used to read all data and therefore simulate a restore.

By default, the "check" command will always load all data directly from the repository and not use a local cache.”.

docker run --rm -ti  --env-file=/etc/restic-env  \

-v $HOME/restic-data:/data  restic/restic check

 

Additional Resources

Video: How to use Restic backups

Goes more in-depth on Restic options, creating schedules, retention periods, etc.

https://youtu.be/5DjNjqLuLSs

 

List of all Restic environment variables

https://restic.readthedocs.io/en/latest/040_backup.html?highlight=variables#environment-variables