How to configure Backblaze B2 with Duplicity on Linux

Millions of Linux installations around the world include duplicity, an open source backup tool which supports encryption, compression, data retention and many other powerful features. Duplicity can store backup data in many destinations, including Backblaze B2. This guide will help you get setup and give you the commands to do a full backup and restore of a specific folder.

Installation

While duplicity is pre-installed on many Linux systems, you need version v0.7.12 or newer, which has been available since March 2017. 

The easiest way to upgrade on Ubuntu, Debian and other Linux systems that support apt is via the terminal:

$ duplicity --version
duplicity 0.7.06
$ sudo add-apt-repository ppa:duplicity-team/ppa $ sudo apt-get update $ sudo apt-get --only-upgrade install duplicity
$ duplicity --version
duplicity 0.7.13.1

Create Backblaze account with B2 enabled

First, you have to create a Backblaze account and enable B2. If you already have a Backblaze account, login, visit "My Settings" and enable "B2 Cloud Storage" on your account under "Enabled Products" 

My_Settings.png

Second, create an Private bucket in your B2 account.

B2_Cloud_Storage_Buckets.png

Third, retrieve your account id and application key. You will need to specify these in duplicity to perform backups and restores. 

Performing a backup of your home directory

While duplicity is a sophisticated tool with many features, getting started is very easy. For example, if you want to create a backup of your home directory, use the following command, replacing your Backblaze B2 account id, application key and bucket name appropriately:

$ duplicity ~ b2://[account id]:[application key]@[B2 bucket name]

On first run, duplicity will create an encryption key for you and run a full backup. After the backup is complete, duplicity will give you stats about how much was backed up and stored. Each subsequent run of duplicity will only do an incremental backup, which will be considerably faster.

--------------[ Backup Statistics ]--------------
StartTime 1503518136.97 (Wed Aug 23 12:55:36 2017)
EndTime 1503518593.19 (Wed Aug 23 13:03:13 2017)
ElapsedTime 456.22 (7 minutes 36.22 seconds)
SourceFiles 456
SourceFileSize 2157364972 (2.01 GB)
NewFiles 456
NewFileSize 2157364972 (2.01 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 456
RawDeltaSize 2156938988 (2.01 GB)
TotalDestinationSizeChange 1449185876 (1.35 GB)
Errors 0
------------------------------------------------- 

Restoring a folder from your duplicity backup

To restore a folder from your backup, use the following command:

$ duplicity restore --file-to-restore [folder name from backup] b2://[account id]:[application key]@[B2 bucket name] [restore path]

The [folder from backup] is the folder's name in your duplicity backup. The [restore folder] is the folder on your filesystem where you want this folder to be restored.

Just specify the encryption passphrase you set above, and the folder will be restored.

What else can I do?

We got great inspiration from Logan Marchione's post on how to setup Duplicity and B2. It includes instructions on setup up your own GPG key, and a script you can run daily to do full backups every 30 days. 

Other resources include Ubuntu's howto page for Duplicity and the duplicity man page.

I want a GUI. Where can I get a GUI? 

There is a popular GUI for duplicity, Déjà Dup. While it does not support Backblaze B2 today, there has been some interest to implement this. If you'd like to see B2 support, make sure you share your interest.

Another open source backup tool that supports Backblaze B2 and has a web based GUI is Duplicati. There are commercial tools with GUIs too: Cloudberry, qBackup, and Duplicacy.

 

Special thanks to Matthew Bentley for adding the Backblaze B2 support to duplicity.

 

 

Have more questions? Submit a request

12 Comments

  • 0
    Avatar
    logan

    Hey, thanks for the mention! Great article!

  • 1
    Avatar
    hanno

    Per-bucket API keys please. Thank you :)

  • 0
    Avatar
    daddy

    Duplicity support was exactly the reason I've started using backblaze!

  • 0
    Avatar
    mike.hewitt

    Hopefully I will get past the stage of getting the files to download.  For example I receive:

    Failed to fetch http://ppa.launchpad.net/duplicity-team/ppa/ubuntu/dists/jessie/main/binary-amd64/Packages 404 Not Found

    W: Failed to fetch http://ppa.launchpad.net/duplicity-team/ppa/ubuntu/dists/jessie/main/binary-i386/Packages 404 Not Found

     

     

  • 0
    Avatar
  • 0
    Avatar
    ron.siesta

    I have had no luck with the restore. The backup seems to work fine. If I do a backup with the following command.:

    duplicity /etc b2://myaccount:mykey@duplicity-davis/etc

    I have a bucket called duplicity-davis, it has a folder in it of etc, in the folder is the encrypted file dataset.

    I now want to restore that folder /etc. All the combinations I have tried all result in

    etc not found in archive - no files restored.

    Can someone please give me the syntax to restore the /etc folder to the current directory?

    duplicity restore --file-to-restore /etc b2://myaccount:mykey@duplicity-davis/etc ./

    is one format I have tried.

     

     

     

     

     

     

     

     

     

     

  • 0
    Avatar
    ron.siesta

    I guess I totally misunderstood the example command to restore above, to do what I wanted I used

    duplicity restore b2://myaccount:mykey@duplicity-davis/etc ./etc

     

  • 0
    Avatar
    greg

    Using these instructions, I ran into a problem backing up a large directory:

    Attempt 1 failed. URLError: <urlopen error [Errno 104] Connection reset by peer>
    Attempt 2 failed. URLError: <urlopen error [Errno 104] Connection reset by peer>
    Attempt 3 failed. URLError: <urlopen error [Errno 104] Connection reset by peer>
    Attempt 4 failed. URLError: <urlopen error [Errno 104] Connection reset by peer>


    Apparently the sig file was larger than the 5 GB limit imposed by Backblaze, so the backup failed. To fix this, I added "--max-blocksize=20480" to the backup command:

    duplicity --max-blocksize=20480 ~ b2://[account id]:[application key]@[B2 bucket name]
    Edited by greg
  • 0
    Avatar
    ivan-backblaze

    I wasted a week trying to use the duplicity B2 support to backup some company laptops.  As of October 2017, this is not ready for any kind of serious use.  It didn't show progress data (not working in the b2 backend yet, even though multiple patches are in the project bugtracker), it didn't finish backing up either laptop on the first try, it couldn't resume uploading, and it filled my entire disk, aborting the backup.

    Maybe this can be a reasonable solution in the future when these problems are fixed, but right now this was a HUGE waste of my time.  I'd ask for my money back from Backblaze, but it was only a few bucks.  I write here to warn other Linux users not to waste their valuable time.

  • 0
    Avatar
    ron.siesta

    Works great for me, I have not had any issues I am using it on Ubuntu 16.04 with duplicity 0.7.14. I back my home, root, etc on multiple machines and have done a restore. I am sure there are still issues with certain combinations but works great for me. My largest backup is about 30G. I have not found a compatible version in the normal distributions channels yet, I had to add the PPA.

     

    ppa:duplicity-team/ppa

  • 0
    Avatar
    ivan-backblaze

    Doesn't work at all for me, on Ubuntu 16.04 with the PPA.  I needed to backup considerably more than 30G, and I needed to do it on laptops that were more than 50% full.  The promise of this working at all wasted a week of my time.   Didn't work, didn't show progress, didn't resume an aborted upload.  Can't even upload at wire speed from a basic SOHO internet connection.  Patches in the duplicity bugtracker for these things seem to be stalled and ignored.  I wound up overnighting a pair of external drives instead.

     

  • 0
    Avatar
    ron.siesta

    My backup is pretty simple and minor so probably doesn't test the system. Here is a short proof of concept script I wrote, makes backups based on host name of the array of paths. Does regular full backups and deletes old backup. Most of the machines I run this on are less then 1G of data on those paths, but they are servers not user machines.

     

    #!/bin/bash
    export PASSPHRASE=mypassphrase
    acctid_appkey=myaccount_id:my_application_key
    pathsave=("/etc" "/root" "/home")

    for i in "${pathsave[@]}"
    do
    echo "backup of : $HOSTNAME:$i"
    /usr/bin/duplicity -v4 --gpg-options="--cipher-algo AES256" --exclude **/.cache --full-if-older-than 1M $i b2://$acctid_appkey@duplicity-$HOSTNAME$i
    /usr/bin/duplicity -v4 --gpg-options="--cipher-algo AES256" remove-older-than 1Y --force b2://$acctid_appkey@duplicity-$HOSTNAME$i
    done

    unset PASSPHRASE

     

     

     

    Edited by ron.siesta
Please sign in to leave a comment.
Powered by Zendesk