How do I use the b2 sync command?

 

The B2 sync command is meant to be used to keep a local directory synced with a B2 destination. You can upload files from this directory and then either sync changes only, add new files, create file versions, or even hold onto versions for a specified number of days.

 

The B2 sync command is used within the B2 Command Line tool. If you haven't installed the B2 Command Line tool, you can do so here.

 

The basic parameter structure for the b2 sync call is as follows:

b2 sync [--delete] [--keepDays N] [--skipNewer] [--replaceNewer] \

        [--compareVersions <option>] [--threads N] [--noProgress] \

        [--excludeRegex <regex>] <source file location> <B2 bucket destination>

 

 

To initially upload a source directory to a B2 bucket destination: 

Call "b2 sync <source file location> <B2 bucket destination>"

The source file location must be a local file path, and the B2 bucket destination must be a B2 bucket path. Use "b2://<bucketName>/<prefix>" for B2 paths, e.g."b2://my-bucket-name/a/path/prefix/".

File uploads are done in parallel in multiple threads 

The default number of threads is 10.  Progress is displayed on the console unless '--noProgress' is specified.  A list of actions taken is always printed. There is not a set amount of threads that will work best in every scenario or for every user. We recommend that you start with the default 10 threads and watch you system activity. If you are maxing out your CPU, RAM, disk or network usage, decrease the number of threads. If you aren't, try to keep increasing the number of threads until you see performance dip. It is a trial and error process to find the right number of threads for your use case.

 

Now that you have your directory uploaded to B2, let's say you make changes to the directory files locally (i.e update your files and keep the same names). Here are some different use cases you may run into

 

 

To upload local changes to all files and keep previous versions:

Simply call "b2 sync <source file location> <B2 bucket destination>" once more. Your older versions of the files will remain, and you will upload new copies on top of the originals. Please note, you will be charged for storing multiple versions of files: if your original file is 5 GB and your updated file of the same name is 5.3 GB, you will be charged for 10.3 GB of storage.

 

You will only upload new file versions for files that have been updated 

If you have files that are the same and haven't changed, you will not upload a new file version. Files are considered to be the same if they have the same name and modification time. However, this behavior can be changed using the --compareVersions option.  Possible values are:

      'none':    Comparison using the file name only

      'modTime': Comparison using the modification time (default)

      'size':    Comparison using the file size

You can specify --excludeRegex to selectively ignore files that match the given pattern. Ignored files will not copy during the sync operation. The pattern is a regular expression that is tested against the full path of each file.

 

You can upload changes to all files and keep previous versions for set amount of days

For example, to make the destination match the source, but retain previous versions for 30 days, call       "b2 sync --keepDays 30 --replaceNewer <source file location> <B2 bucket destination>". You'll still upload new file versions, however the older versions will now set to be removed in 30 days.

 

 

For files that are present in the B2 bucket compared to the source:

When a destination file is present that is not in the source, the default is to leave it there.  Specifying "--delete" means to delete destination files that are not in the source. 

Files at the source that have a newer modification time are always copied to the destination.  If the destination file is newer, the default is to report an error and stop.  But with "--skipNewer" set, those files will just be skipped.  With "--replaceNewer" set, the old file from the source will replace the newer one in the destination.

 

 

If you run into any issues or have any suggesting/feature requests while using the b2 sync, please submit them to our Github page.

Have more questions? Submit a request

0 Comments

Article is closed for comments.
Powered by Zendesk