How to configure Backblaze B2 with Restic on Linux

Restic is a multi-platform command line backup software program that is designed to be fast, efficient, and secure. Restic supports a variety of backends for storing backups, including a local server, SFTP server, HTTP Rest server, and a number of cloud storage providers, including Backblaze B2.

Files are uploaded to a B2 bucket as de-duplicated, encrypted chunks. Each time a backup runs, only changed data is backed up. On each backup run, a snapshot is created enabling easy restores to a specific date or time.

The location where the backup data is stored is assumed to not be a trusted environment, such as a shared space where others could possibly access the backups. To guarantee the confidentiality and integrity of the backed up data, Restic uses cryptography to protect it.

Restic is open source and free software and licensed under the BSD 2-Clause License and actively developed on GitHub.

Getting Started with Restic and Backblaze B2

This guide is intended to get you started with Restic and begin using the basic commands to do a full backup and restore of a specific folder to Backblaze B2.

There are many more options than we cover here and opportunities to streamline and automate the process through shell scripts and other means. You can find more information on using Restic at https://restic.readthedocs.io.

Installation

Being a relatively new program, Restic is not yet widely available from Linux distribution repositories. It is becoming more popular, so likely that will change. You should check whether it is possible to install Restic using your standard distribution package management tools, such as apt, yum, and dnf.

In any case, you can download a number of the latest pre-compiled binaries from the Restic release page.

Our examples use the Ubuntu Linux distribution. On that platform we needed to download the binary from the release page and install it.

As we will be installing Restic on Ubuntu 16.04, we will select the AMD64 pre-compiled binary.

Screen_Shot_2018-08-21_at_10.32.06_AM.png

Download it to your system.

Screen_Shot_2018-08-21_at_10.37.13_AM.png

Once in our downloads folder, we can extract the binary using the Archive Manager.

The binary file can be placed anywhere in the path for the current system or user. In this example, it is placed in /home/roderick/bin. Update the binary to make it executable by the user with the chmod command, such as:

$ chmod u+x restic_0.9.2_linux_amd64

(using the exact name of the binary you downloaded and extracted).

Change the name of the executable binary to restic.

$ mv restic_0.9.2_linux_amd64 restic

Verify that Restic is available by typing “restic version” in the terminal.

Screen_Shot_2018-08-21_at_10.57.21_AM.png

Create Backblaze account with B2 enabled

If you don’t have a Backblaze account, you need to create a Backblaze account and enable B2. If you already have a Backblaze account, login, visit "My Settings" and enable "B2 Cloud Storage" on your account under "Enabled Products.” 

screen5.png

Second, create an Private bucket in your B2 account. (Or, you can do this directly from Restic. See Initialize the Restic Repository below.)

screen6.png

Note that the bucket name must be unique across all of B2, not just your account.

Third, retrieve your account id and application key. You will need to specify these in Restic to perform backups and restores. 

Performing a backup of your home directory

Restic can use environment variables to make it easier to use the command line. Use the account ID and application key you obtained from your B2 account. If you wish, you can set these as environment variables so you don't have to enter them with every command. You can export them in your .bashrc file or other place of your choosing so they’re available whenever you type a restic command. 

$ export B2_ACCOUNT_ID="<MY_ACCOUNT_ID>"
$ export B2_ACCOUNT_KEY="<MY_SECRET_ACCOUNT_KEY>"

Note:
In case you want to use Backblaze Application keys, replace <MY_ACCOUNT_ID>
and <MY_SECRET_ACCOUNT_KEY> with <applicationKeyId> and <applicationKey>
respectively. See Application Keys for more information.

You also can specify the name of the B2 bucket in an environment variable. In these examples, the bucket is called "my-restic-backup". As it is located on B2, it has a b2: prefix.

$ export RESTIC_REPOSITORY="b2:my-restic-backup"

If you do that, you don't need to enter the "-r <repo string>" parameter every time you enter a Restic command. In the examples below, we'll assume the account ID and application key are in environment variables, but we'll specify the -r repository parameter each time for clarity.

Each time you enter a Restic command, you will be asked for your repository password. You will need to enter that or store the password in a file and retrieve it with the -p parameter. Or, you can use an environment variable to store the name of the password file and it will be used automatically.

$ export RESTIC_PASSWORD_FILE="restic-pw.txt"

Initialize the Restic repository

Initialize the repository stored at Backblaze B2 with the bucket name of your choice. If the bucket does not exist yet, it will be created.

$ restic -r b2:my-restic-backup init

1.png

You will be asked to create a repository password.

To back up your home directory using the home environment variable ($HOME or ~) and the restic environment variables set as above, you would enter the following:

$ restic -r b2:my-restic-backup backup ~

2.png

The contents of a directory at a specific point in time is called a “snapshot” in Restic. The specific snapshot just created is identified by a sequence of hexadecimal characters.

To view the snapshot you now have stored on B2, enter

$ restic -r b2:my-restic-backup snapshots

3.png

If you run the backup command again, Restic will create another snapshot of your data, but this time it will be faster due to de-duplication. When you back up the same directory again (maybe with new or changed files) Restic will find the old snapshot in the repo and by default only reads those files that are new or have been modified since the last snapshot. This is decided based on the modify date of the file in the file system.

For subsequent backups (snapshots), you would enter the same command.

$ restic -r b2:my-restic-backup backup ~

4.png

 You now have two snapshots in your repository.

$ restic -r b2:my-restic-backup snapshots

5.png

Restoring your backup

To restore your backup, you specify which snapshot you wish to restore and where you want it to be put with the restore and the -t parameters.

$ restic -r b2:my-restic-backup restore  9a83a611 -t /tmp/restic-restore 

6.png

(Open image above in new window/tab to view.)

The snapshot will be restored to your system at /tmp/restic-restore.

You can save to and restore from different systems. The repository in the example above was saved from an Ubuntu Linux system and a snapshot from that repository was restored to a Debian Linux system.

On Debian:

$ restic -r b2:my-restic-backup snapshots

7.png

$ restic -r b2:my-restic-backup restore 9a83a611 -t /tmp/restic-restore

8.png

(Open image above in new window/tab to view.)

Note: The number of concurrent connections to the B2 service can be set with the

-o b2.connections=10

switch. By default, at most five parallel connections are established.

To Learn More

There's a lot you can do with Restic, including adding tags, mounting a repository locally, and scripting. To learn more, you can read the documentation at https://restic.readthedocs.io.

 

 

Have more questions? Submit a request

28 Comments

  • 1
    Avatar
    rnicksic

    Restic Release Page link is broken. Missing ":" after https https//github.com/restic/restic/releases/tag/v0.7.3

  • 0
    Avatar
    roderick

    Thanks. Fixed.

  • 0
    Avatar
    eplant

    Your guide is helpful, but why are the snapshots I create via the command line restic backup not visible online under the "snapshots" section? I was hoping that online I could see the filenames, directory structure, modification times, etc.

    Thanks for your assistance.

  • 0
    Avatar
    roderick

    The data is encrypted on B2 and not directly available. You'll have to use restic to view the information you are looking for. 

  • 0
    Avatar
    eplant

    OK, thank you. I thought that may be the case. 

  • 0
    Avatar
    upssnowman

    Unfortunately restic doesn't work with B2 :-(

    I have gotten restic to work on Google Cloud, Azure, and various sftp servers but it does not work with B2 unfortunately

  • 0
    Avatar
    rnicksic

    @upssnowman, I recently uploaded almost 500GB of files to B2 via restic. I set env variables in the head of my backup script, e.g.

    #!/bin/bash

    export B2_ACCOUNT_ID="<account_id>"

    export B2_ACCOUNT_KEY="<account_key>"

    export RESTIC_REPOSITORY="b2:<bucket_name>"

    export RESTIC_PASSWORD="<password>"

    RESTIC=/usr/local/sbin/restic

    # Initialize the repo, comment out or remove after first run or you will re-initialize it and lose your backup

    $RESTIC init 

     

    # Backup files

    $RESTIC backup <path1>

    $RESTIC backup <path2>

    ...

     

    I hope that helps. I am very pleased with B2, especially the low cost.

  • 0
    Avatar
    roderick

    upssnowman Yes, restic works with B2. That's what this article is about. What's the problem you're seeing with restic and B2?

  • 0
    Avatar
    upssnowman

    I was finally able to get it to work but it is extremely slow. It is only uploading out a rate of a few kilobytes per second. It took over 30 minutes to copy a 5 MB file. It's not my connection because if I use duplicity to the same account it is really quick

     

  • 0
    Avatar
    roderick

    Which version of restic you use will make a difference, as will which binary. On their help page a user reports different results with different ISPs. https://github.com/restic/restic/issues/1383

  • 0
    Avatar
    upssnowman

    restic 0.7.3
    compiled with go1.9 on linux/amd64

     

  • 0
    Avatar
    eplant

    I was able to get restic working nicely on my Xubuntu system at home.
    Uploaded many GB over several days, usually at 200-250kbps over my garden variety Comcast cable internet. A little slow, but definitely tolerable.

  • 0
    Avatar
    upssnowman

    It's not my connection or B2 itself. It's a combination of B2 and restic. Definitely not ready for prime time. From my side to B2 using duplicity it's super fast. B2 backups only struggle with restic. It takes over 1 minute to sync on single small text file with restic. However restic is blindly fast from my side to Google Cloud and AWS. And for the record, my internet connection is a rock solid 300/20

     

    Edited by upssnowman
  • 0
    Avatar
    roderick

    FYI, restic 0.8.0 fixes a bug that should speed up transfers with B2.

    See https://github.com/restic/restic/releases/tag/v0.8.0

    “A bug was discovered in the library we’re using to access Backblaze, it now reuses already established TCP connections which should be a lot faster and not cause network failures any more. #1291 #1301” 

  • 0
    Avatar
    andy

    I have started to get the following error with ubuntu server running restic 0.8.1;

    create backend at b2:myrepo failed: b2.NewClient: b2_authorize_account: 401: Invalid authorization token

    Is there any documentation on how to fix this? I presume I need to re-auth my connection?

    Thanks

  • 0
    Avatar
    roderick

    Andy, you could try to re-auth, but I suggest you ask restic about the error message. They know best about their product.

  • 0
    Avatar
    andy

    Hi Roderick,

    I get a similar response if I run this which is not using restic code at all;

    curl https://api.backblazeb2.com/b2api/v1/b2_authorize_account -u "ACCOUNT_ID:myaccountid"

    {
    "code": "bad_auth_token",
    "message": "Invalid authorization token",
    "status": 401
    }

    Appreciate your help.

  • 0
    Avatar
    roderick

    Andy, I suggest you contact our support folks about that. https://help.backblaze.com/hc/en-us/requests/new

  • 1
    Avatar
    backblaze

    I assume having bucket versioning enabled when using Restic is not necessary, correct? I can see it protecting against accidentally re-initializing a repository and removing the contents per restic, but aside from that restic snapshots would provide all the versioning functionality at the higher level, correct?

  • 0
    Avatar
    roderick

    Bucket versioning would be on top of what restic is already doing, so, yes, it's probably not necessary. It doesn't hurt to have the extra protection, however.

  • 1
    Avatar
    cojorgensen

    Great guide.

    Might I suggest a typo fix...

    currently:

    $ export B2_ACCOUNT_ID =""
    $ export B2_ACCOUNT_KEY=""

    There is a space at the end of B2_ACCOUNT_ID but before the = which I never catch, and always causes my first profile builds to fail when I'm setting up (fresh) from this guide.

    Just needs to be deleted. Tiny thing.

    (I didn't know where else to put this, please delete the comment if fixed)

    Thank you!

    Edited by cojorgensen
  • 0
    Avatar
    roderick

    cojorgensen - Thanks for the catch. Fixed.

  • 0
    Avatar
    deepnadig

    I'm using restic version 0.8.3 on Bionic. I'm getting a bucket create failed error:

    failed: NewBucket: b2_create_bucket: 400: Bucket name is already in use

    However, I can't see the bucket in the bucket list. Any help appreciated!

  • 0
    Avatar
    thechiragaggarwal

     

    Thanks for the guide, I've successfully configured my backups. However, I backed up 100+ mb of data but there was no change in bucket size even after waiting.

     

    Moreover, can I not see the files backed up via restic in UI? Do I need to tell it to transfer the data to b2?

  • 0
    Avatar
    roderick

    thechiragaggarwal - If you've successfully uploaded your files to B2, you'll see a number of folders: data, index, keys, and snapshot as well as a config file. If you don't see these, there likely is something wrong with your configuration. Remember, your bucket name needs to be unique across all of B2, not just your account.

  • 0
    Avatar
    roderick

    deepnadig - Your bucket wasn't created because you chose a bucket name already in use (see the error message returned). Your bucket name needs to be unique across all of B2, not just your account.

  • 0
    Avatar
    socios

    Hi, i am having a problem following this guide. I do all the steps but when i get to initialize the bucket

    When i do the 

    $ restic -r b2:testagiraudo init

    it just hang in there, no error, no warning, nothing, it just freeze until i interrupt it.

    Can you help me?

  • 0
    Avatar
    roderick

    socios: We recently updated our Application Keys API, which changed the procedure. See the note above for Application Keys.

Please sign in to leave a comment.
Powered by Zendesk