I’m still running a 6th-generation Intel CPU (i5-6600k) on my media server, with 64GB of RAM and a Quadro P1000 for the rare 1080p transcoding needs. Windows 10 is still my OS from when it was a gaming PC and I want to switch to Linux. I’m a casual user on my personal machine, as well as with OpenWRT on my network hardware.
Here are the few features I need:
- MergerFS with a RAID option for drive redundancy. I use multiple 12TB drives right now and have my media types separated between each. I’d like to have one pool that I can be flexible with space between each share.
- Docker for *arr/media downloaders/RSS feed reader/various FOSS tools and gizmos.
- I’d like to start working with Home Assistant. Installing with WSL hasn’t worked for me, so switching to Linux seems like the best option for this.
Guides like Perfect Media Server say that Proxmox is better than a traditional distro like Debian/Ubuntu, but I’m concerned about performance on my 6600k. Will LXCs and/or a VM for Docker push my CPU to its limits? Or should I do standard Debian or even OpenMediaVault?
I’m comfortable learning Proxmox and its intricacies, especially if I can move my Windows 10 install into a VM as a failsafe while building a storage pool with new drives.
You’re not using a reverse proxy on rhel, so you’ll need to also make sure that the ports you want are available, and set up a dns record for it, and set up certbot.
On k8s, I believe istio gateways are meant to be reused across services. You’re using a reverse proxy so the ports will already be open, so no need to use firewall-cmd. What would be wrong with the Service included in the elasticsearch chart?
It’s also worth looking at the day 2 implications.
For backups you’re looking at bespoke cronjobs to either rsync your database or clone your entire 100gb disk image, compared to either using velero or backing up your underlying storage.
For updates, you need to run system updates manually on rhel, likely requiring a full reboot of the node, while in kubernetes, renovate can handle rolling updates in the background with minimal downtime. Not to mention the process required to find a new repo when rhel 11 comes out.
I am using a reverse proxy in production. I just didn’t mention it here.
I’d have to set up a DNS record for both. I’d also have to create and rotate certs for both.
We use LVM, I simply mounted a volume for /usr/share/elasticsearch. The VMWare team will handle the underlying storage.
I agree with manually dealing with the repo. I dont think I’d set up unattended upgrades for my k8s cluster either so that’s moot. Downtime is not a big deal: this is not external and I’ve got 5 nodes. I guess if I didn’t use Ansible it would be a bit more legwork but that’s about it.
Overall I think we missed each other here.
Well, my point was to explain how Kubernetes simplifies devops to the point of being simpler than most proxmox or Ansible setups. That’s especially true if you have a platform/operations team managing the cluster for you.
Some more details missed here would be that external-dns and cert-manager operators usually handle the DNS records and certs for you in k8s, you just have to specify the hostname in the HTTPRoute/VirtualService and in the Certificate. For storage, ansible probably simplifies some of this away, but LVM is likely more manual to set up and manage than pointing a PVC at a storageclass and saying “100Gi”.
Either way, I appreciate the discussion, it’s always good to compare notes on production setups. No hard feelings even in the case that we disagree on things. I’m a Red Hat Openshift consultant myself these days, working on my RHCE, so maybe we’ll cross paths some day in a Red Hat environment!
Considering I am the operations team, just goes to show how much I have left to learn. I didn’t know about the external-dns operator.
Unfortunately, my company is a bit strange with certs and won’t let me handle them myself. Something to check out at home I guess.
I agree with you about the LVM. I have been meaning to set up Rook forever but never got around to it. It might still take a while but thanks for the reminder.
Wow. That must have been some work. I don’t have these certs myself but I’m looking at the CKA and CKS (or whatever that’s called). For sure, I loved our discussion. Thanks for your help.
Yeah, I think you pick up things from all over the place as a consultant. I see lots of different environments and learn from them.
Ah yeah, external-dns operator is great! it’s maybe a bit basic at times but its super convenient to just have A/AAAA records appear for all your loadbalancer svcs and HTTPRoutes. Saves a ton of time.
That’s super unfortunate that the certs are siloed off. Maybe they can give you a NS record for a subdomain for you to use ACME on? I’ve seen that at some customers. Super important that all engineers have access to self-service certs, imo.
Rook is great! It definitely can be quite picky about hardware and balancing, as I’ve learned from trying to set it up with two nodes at home with spare hdds and ssds 😅 Very automated once it’s all set up and you understand its needs, though. NFS provisioner is also a good option for a storageclass as a first step, that’s what I used in my homelab from 2021 to 2023.
Heres my rook config:
https://codeberg.org/jlh/h5b/src/branch/main/argo/external_applications/rook-ceph-helm.yaml
https://codeberg.org/jlh/h5b/src/branch/main/argo/custom_applications/rook-ceph
Up to 3 nodes and 120TiB now and I’m about to add 4 more nodes. I probably would recommend just automatically adding disks instead of manually adding them, I’m just a bit more cautious and manual with my homelab “pets”.
I’m not very far on my RHCE yet tbh 😅 Red hat courses are a bit hard to follow 😅 But hopefully will make some progress before the summer.
The CKA and CKS certs are great! Some really good courses for those on udemy and acloudguru, there’s a good lab environment on killer.sh, and the practice exams are super useful. I definitely recommend those certs, you learn a lot and it’s a good way to demonstrate your expertise.