18
loading...
Backing up Appwrite to Backblaze
Recently, Appwrite has released an article on how to backup and restore all data on your production server. You can read it here:
This article got me thinking 🤔 Sure, you can backup most MySQL databases into a simple
backup.sql file, but what if you have multiple massive projects inside Appwrite? Your disk storage may or may not be able to hold a backup of all Appwrite volumes. Additionally, you most likely need to backup periodically and need multiple versions of the backup. So, what are your options?You should go with storage providers not only because they have a retention system already in place, also due to their experience in the field as they most likely care about security a lot and use raids to ensure your data won't get lost.
I wrote this article as I was implementing this on my own production server, and I will show you everything you need to know about backing up Appwrite into Backblaze.
First things first, what is Backblaze? Backblaze is one of the most, if not most, used commercial platform to backup your data for the lowest price possible. They provide solutions to backup your personal computer or your business data. The service we will be using is B2 Cloud Storage as this gives our flexibility just like an actual hard drive. You don't have to worry about pricing yet because their free plan gives you 10GB of free space. To learn more about this product and its pricing, you can take a look at: https://www.backblaze.com/b2/cloud-storage.html
"Over one exabyte of data storage under management, and counting." - Backblaze website
After creating an account and logging in (make sure to confirm your email), Backblaze may ask you to set up 2FA using your phone number for additional security.
To use Backblaze from our server, we will need:
Backblaze uses buckets to store your data. In our case, we will only need one bucket as we will only back up two or three files at most (MySQL database and Appwrite Docker volumes). To create a bucket, navigate to
Bucket and press Create a Bucket. Give it some name such as appwrite and click Create a Bucket again. You should see a newly created bucket on your list. Make sure to mark your Bucket Name as we will need this later.If you enter
App Keys, you can Add a New Application Key. For simplicity, we will ignore that, and we will use our master key to have all permissions. Simply click Generate a New Master Application Key, and this should give you keyID and applicationKey.Wohoo 🎉 we have all secrets we need; let's continue!
To make sure everything is working, let's try some simple action on B2:
We need to have B2 CLI installed on our machine, but this can be easily avoided thanks to a Docker image
mtronix/b2-cli:0.0.1. This is the only image I could find that was actually working.To list all files in our bucket, we run:
docker run \
--rm \
mtronix/b2-cli:0.0.1 \
bash -c \
"b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 list-file-names <BUCKET_NAME>"The output should be an empty list of files:
{
"files": [],
"nextFileName": null
}Let's upload a file 😎
We need to have a file... I downloaded a cute cat image and saved it as
We need to have a file... I downloaded a cute cat image and saved it as
cat.png. To upload the file, I run:docker run \
--rm \
-v $PWD:/b2 \
mtronix/b2-cli:0.0.1 \
bash -c \
"b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> cat.png cat.png"The successful result should look something like:
{
"action": "upload",
"fileId": "4_z78da5cd2a05db73574a90515_f11841831f8e91ca6_d20210717_m092603_c002_v0001140_t0056",
"fileName": "cat.png",
"size": 136021,
"uploadTimestamp": 1626513963000
}Finally, run the
list-file-names command from above again, and you should see your file in the array of files:{
"files": [
{
"accountId": "8ac20d754955",
"action": "upload",
"bucketId": "78da5cd2a05db73574a90515",
"contentLength": 136021,
...Backblaze let you do all of this function manually on their website. Exercise for you 😲 Remove our test file from the bucket. Do it however you want - you can remove it using B2 CLI or manually on the website.
To create an Appwrite MariaDB backup, you run this command:
docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u "$MYSQL_USER" -p "$MYSQL_ROOT_PASSWORD"' > ./dump.sql(Source: Appwrite article mentioned earlier)
To backup this export into Backblaze, I created a script that:
dump.sql
dump.sql to Backblazedump.sql to free up the space on the machineThis is what the script looks like:
docker-compose exec \
mariadb \
sh -c \
'exec mysqldump --all-databases --add-drop-database -u"$MYSQL_USER" -p"$MYSQL_ROOT_PASSWORD"' > ./dump.sql ; \
docker run \
--rm \
-v $PWD:/b2 \
mtronix/b2-cli:0.0.1 \
bash -c \
"b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> dump.sql dump.sql" ; \
rm dump.sqlEach time you run this script, it will backup your MariaDB database into Backblaze. Backblaze can store multiple versions of a single file; just make sure to configure retention on a bucket, for example, 7 days.
Once again, let's get our base command from Appwrite's article:
Before running these commands is it highly recommended to shut down your Appwrite instance to ensure you get a complete backup.
mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/uploads && tar cvf /backup/uploads.tar ."mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/functions && tar cvf /backup/functions.tar ."These two commands will backup two different volumes. The first one back up all files in Appwrite storage while the second one backup your function tags. Let's start with backing up uploads.
Just like with MariaDB, we will:
backup/uploads.tar
The full command to do this is:
mkdir -p backup && docker run \
--rm \
--volumes-from "$(docker-compose ps -q appwrite)" \
-v $PWD/backup:/backup \
ubuntu \
bash -c \
"cd /storage/uploads && tar cvf /backup/uploads.tar ." ; \
docker run \
--rm \
-v $PWD:/b2 \
mtronix/b2-cli:0.0.1 \
bash -c \
"b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> backup/uploads.tar upload_backup.tar" ; \
rm backup/uploads.tarThis should create a file called
upload_backup.tar in your Backblaze bucket.Exercise time 💪 You will need to backup Appwrite functions too! Using all knowledge from this article, you should be able to prepare a command that will do it.
If you are here only to copy&paste commands, here is the command:
If you are here only to copy&paste commands, here is the command:
mkdir -p backup && docker run \
--rm \
--volumes-from "$(docker-compose ps -q appwrite)" \
-v $PWD/backup:/backup \
ubuntu \
bash -c \
"cd /storage/functions && tar cvf /backup/functions.tar ." ; \
docker run \
--rm \
-v $PWD:/b2 \
mtronix/b2-cli:0.0.1 \
bash -c \
"b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> backup/functions.tar upload_functions.tar" ; \
rm backup/functions.tar
That's it! I hope this article helped you backing up your Appwrite server. If you have any questions, feel free to join Appwrite's Discord server and chat with their amazing community: https://appwrite.io/discord
18