Taking data protection more seriously, one step at a time.
I will be the first to admit that my backup strategy needs work. I have been chasing the 3-2-1 holy grail of back up strategies for a while now. Two main roadblocks keep coming up: time, and money. Time to configure the backups, test them, test a restore, and setup notifications to ensure that I am aware of any hiccups or failures. Money because well, everything costs money. Hard drives for local storage, and remote storage as well. Whether it is a rented box somewhere like Hetzner, or hard drives at a family member’s home. But like I said, I have been slowly working on improving that.
My current strategy is still mostly lacking the 1 offsite copy, though I have selectively achieved it. Perfect is the enemy of good, and I have started building backups around that philosophy. I kept getting paralysed because I didn’t have all the 3-2-1 components figured out. It got to the point where I had no backups because I didn’t have everything in place. Then I decided to just start even if it’s imperfect. Having no backups is far worse than having functional backups that aren’t following all best-practices.
My current backups
As it stands, I have a NAS with Proxmox Backup Server on it for my Proxmox backups. Those run every 2-3 days and backup whole VMs and containers. This is great, most of my services are not what I would describe as critical, and the data often does not frequently change much on 80% of my services. However, there are a few things like Nextcloud and Immich that should absolutely be backed up every night. For this I currently run restic, and backup to a REST server also running on my NAS.
The Plan™️
So, it sounds like I have a pretty decent solution so far, why would I change it? Well, ease of use for the most part. I’d say that Nextcloud and Immich are absolutely critical to backup. I want to make this as easy as possible for myself. I am going to be migrating from my current Nextcloud stack with an image from LSIO, and managing all the containers myself to the Nextcloid AIO image. AIO includes BorgBackup in it, so I am going to start using Borg for backups. I can pretty much migrate over my backup scripts from restic to Borg pretty easily since it is just bash.
I have goals set out below for my backup solution:
- Must be configurable via Ansible.
- Must be on my TrueNAS server in a dataset I can then backup offsite.
- Notifications for backup failures.
Number 1 is pretty easy with BorgBackup, there is even a page in their documentation with guidance and an example for configuring hosts via Ansible. The challenge will come with generating a script to backup all the various services and all the pre/post actions required to take safe and complete backups of the data from all my containers.
Number 2 is also pretty straightforward. I am running BorgBackup in a container on TrueNAS SCALE, and then I attached my BorgBackup dataset into my user’s home directory, this is where the Borg repos will live.
Number 3 is also something I am fairly certain I solved with UptimeKuma for my restic backups. I have a service that expects a push from the backup scripts ever 24 hours + 15 minutes. The restic script only pings if the restic backup completes successfully as reported by the exit code. The next challenge I need to solve is automating the creation of these watchers so I don’t have to create them by hand for every service I wish to back up.
Next Steps & Conclusion
My goals ultimately aim to reduce the friction for setting up backups. I will have follow-up posts with details from my setup as I work on it. My first steps will be to automate connecting and setting up repos with my BorgBackup service and my VPS, then working out how to best approach automating backup scripts and notifications. I will leave my currently working backups intact until I am comfortable switching them over. Don’t fix what isn’t broken, and improve upon it only when ready.