Guide to using Backblaze Fireball with Iconik Internet Storage Gateway (ISG)

Udara Gunawardena February 27, 2023 19:56

This is a guide on how to take advantage of a Backblaze Fireball with an Iconik Internet Storage Gateway (ISG) setup.

Steps assuming you HAVE NOT set up an Internet Storage Gateway (ISG)

Configure a new ISG storage in Iconik
Make sure your ISG storage root is the same root that you'd like Iconik to manage after the fact
Configure new B2 cloud storage endpoint
This should be the same bucket that the fireball is linked and will be ingested to
Copy your data to the fireball
Make sure the data that is copied to your fireball has the same relative pathing to your storage on the things you want Iconik to manage
Make sure that your ISG root and your B2 bucket root are pointing to the same place in your storage config
For example, if your local ISG is mounted at /Volumes/MySAN/iconik/ISG, you can copy any tree into the fireball, but you want to make sure the bucket defined as storage is set to this same root. So if you copied /Volumes/MySAN to / on the Fireball, you would want your storage root on the Fireball bucket set to /iconik/ISG. It is important that the files on the ISG are seen in the same relative place in the Fireball ingested bucket

Set the ISG setting as follows

auto-generate-filename-for-remote-upload = false

Enable Upload on the ISG and select the B2 cloud storage
Make sure the upload directory is the same as where the root of the fireball is on the cloud storage
Start the Internet Storage Gateway.
It will now try to upload all files; it will notice the files are already in B2 thus causing it to not perform any upload. Once it has completed indexing/generating proxies, you can move to the next step
If you do not let it index everything first, you will pay egress to transcode the proxies from B2
(Optional) Scan the B2 bucket that was tied to the fireball in Iconik
This will trigger transcoding in the cloud if the files are found first on B2

If this worked properly, you will see the assets in Iconik now have Format links to both the ISG

Steps assuming you HAVE set up an Internet Storage Gateway (ISG)

Configure new B2 cloud storage endpoint
This should be the same bucket that the fireball is linked and will be ingested to
Copy your data to the fireball
Contact Iconik support to retrieve the script for checksum generation
Run this script against the fireball drive
This will create checksums on all the files on the fireball
Scan the B2 bucket that was tied to the fireball in Iconik
Assuming proper steps were followed, files will now be linked to the correct assets

How can we help?