Lets take a moment to explore my homelab journey as it relates to data! One of the first services I hosted in my network was a storage server. I went with TrueNAS Scale virtualized on an XCP-NG host.
TrueNAS Scale has been an excellent experience this far. It has file server, container hosting and KVM hypervisor capabilities. I have been playing around with the file server and container environment services for a few months and have learned quite a bit. There is still a lot more to learn and figure out.
Addressing the elephant in the room, yes, the file server is a VM. This is bad practice. What makes it worse, there is a NFS share from the virtualized TrueNAS back to the hypervisor as a storage repository. When the VM reboots, the hypervisor gets a storage repo ripped out. It is not great, but it has worked well with a limited footprint. It is however high on my list to rearchitect. Having spent a year with TrueNAS, I think it deserves to be its own appliance in my network. I will be working towards deploying it bare metal and then building my hypervisor infrastructure on different hardware.
I will probably create a project page dedicated to TrueNAS as it really has served a large role in my lab, and I plan to continue to expand its responsibilities. ZFS RaidZ2 is truly a data hoarders best friend. Not only that, but it also has a docker and Kubernetes environment baked in. This lead is the second part of the story, Containers.
I like containerized applications. Portable, added security, organized. The first website I built was a containerized web application. You are reading this entry on a containerized web application. I have had the luck and privilege to play with a few different container environments thus far. Mostly Docker with a sprinkle of podman and some Kubernetes environments (OKD and k3s). Hopefully, I will have a OKD environment up and running shortly.
My container workloads are currently running in four places. Docker on a cloud Alma Linux Server, k3s and docker on my homelab TrueNAS scale VM, docker on a Fedora CoreOS server VM and docker on a Kasm VM. I plan to add an OKD cluster as time and money allows. Fedora CoreOS also deserves a project of its own, and I will probably be revisiting it more and more as I get deeper into OKD.
All these threads have coalesced into the current task at hand. Design, deploy, maintain, and persist a PostgreSQL database environment for my lab and my website(s).
This website is currently running SQLite, it will soon be migrated to Postgres. The production website database will be automatically backed up and replicated to a PreProduction and Development database on a weekly basis. The databases are containerized so there are a few different options to go with. The PostgreSQL instance in my lab is running as a container on my TrueNAS server. This simplifies backups as both the data and backup datasets can be automatically backed up and snapshotted using TrueNAS and its suite of ZFS features. The website database is also containerized but running in a cloud. For this task I will use an ansible playbook to both back up the database volume data using a helper container and also to do a database dump to a bind mount which can then be copied and imported.
The final Postgres and data storage hurdle is designing the schema for storing and displaying my projects. I aim to make it modular to easily append new sections to a project, link it to relevant blog posts and hopefully build it to be interchangeable between the different back end iterations I plan on experimenting with (Flask, JavaScript, Go, PHP and possibly Drupal) Also looking forward to getting into Kubernetes environments with distributed replicated databases to facilitate easy horizontal scaling!
Overall, quite a bit has transpired since I started working on my website, and many more things are to come. Thanks for stopping by. Hope to see you again soon!