I've recently migrated to Incus and am enjoying it so far. Hopefully the good feelings last.
Roughly ten years ago, my homelab consisted of a dozen virtual machines running on SmartOS. I was not familiar with Illumos, and this was before it had a widely available web UI, but it was simple enough to use that these challenges didn't matter much. SmartOS was designed to boot reliably from USB flash storage, allowed me to use all my SATA ports for VM storage, and was my first "immutable" operating system. The primary focus on ZFS storage was another great quality of SmartOS.
Two moves and several years later, it was time to rebuild the lab, and I decided to go with Proxmox because it had decent ZFS support. Experience with Proxmox has been very good too. The GUI, many more virtualization features (in addition to the key ones I care about), and better hardware support through the Linux kernel have kept me on Proxmox for a long time.
Customizing my Proxmox installation always gave me anxiety. How could I defend my hypervisor from configuration drift? I wished there could be an immutable version of Proxmox.
Later on, I learned about govulcheck, which offers a novel dynamic/static analysis hybrid approach to vulnerability management. Nothing else out there does this (without teaming up with some huge company). I began to think that I should favor software solutions based on golang.
Ultimately, Incus (and IncusOS) fit this need very well. My IncusOS hosts excellent and I'm glad I can run Incus itself on most Linux distros - including NixOS!
I'll keep a small Proxmox host around for experimenting with new kernel features (Intel GVT-g / SR-IOV graphics) and old operating systems like Windows XP or anything else that needs special QEMU options.
CLI is first class in proxmox, I use the qm command for managing vms all the time. The networking is also just a file in `/etc/network/interfaces` that I modify with vim as needed.
CLI is dandy but Terraform is even nicer. I run something like ten Proxmox clusters now around the world, and an AI agent can deploy a fully terraformed VM for me, tied into monitoring, asset mgmt, EDR, etc in a few minutes...
I concur highly. Well, make sure you buy your cheap support contract, help beta-test Proxmox Datacentre Manager, and push for it. They could probably just onboard one of the open source authors as a contractor or something if enough companies actually pony up the couple of grand per year per box. At 50K a server, paying like 2k for a proxmox license is a joke
Doing things via the CLI and editing config files doesn't automatically give you reproducability or audit trail. Last time I check etckeeper doesn't work with proxmox either (due to the proxmox FUSE mount in /etc).
I get your point and this is likely also not what you mean, but I am very happy with a simple git tracking approach that I use across Proxmoxes and VMs [1].
I don't get their argument, I run GeForceNow on one of my Proxmox nodes in metal and there's no problem with it. It's a Debian underneath, after all. They didn't need to switch to NixOS to run Kodi.
Also, do they get PBS using ZFS snapshots? Do they get HA, live migration, shared storage, easy CephOS, easy snapshots, quick cloning? Do you really want to migrate a VM from one node to another using the command line when you're in some serious situation?
Sure, for a homelab this might be OK, but the UI does make things easy for a reason.and it's not a gimmick.
I agree on a lot of the points, though, I just set up a second cluster and it took over 3 work days because of how much repetitive work is needed to do so. To be able to just take a file with instructions, adjust it a bit and deploy would be so much easier.
I'm a big LXD and now Incus fan. But I went with NixOS rather than IncusOS for my latest build because I prefer the LTS linux kernel over the mainline kernel.
It's really hard to tell what the benefits of Incus over Ganeti are. The Incus site doesn't really lead with features or benefits that I can see. About the only things I could figure out are that Incus can manage containers and can manage instance creation through cloning a base volume, where ganeti (afaik) always creates volume and populates it. Well, and Incus seems to be MUCH more popular than Ganeti.
I searched the documentation but it wasn't really clear what its live migration and ZFS migration story is, but when I asked Claude to research it, it tells me that it supports live migration via ZFS snapshot replication, which is exactly what I'm looking for. I implemented a ganeti storage driver that does the same thing and am just getting ready to start testing it, but if Incus supports it I might look at moving that direction.
Yeah, one thing I've found with Nix is that documentation is generally very terse and assumes you're capable of filling the blanks yourself, even though you might not know they exist.
I find I have a lot of trouble with updating with nixos. I really like the OS but I find that when I have something like Claude Code and I need to update to the newest version, I just have trouble. It just seems like a lot of work updating versions otherwise NixOS is really awesome and great.
I am satisfied with pve and qm commands on proxmox, but I wish there was a nix answer to pfsense where my network and firewall are defined declaratively
This! pfSense/OPNSense seem to me like perfect candidates for declarative configuration. Even though, technically speaking, they are since everything is being save to a config file, which you can backup and restore. It's just less declarative then you'd like it to be. Sounds like they could easily employ Nix for handling that, though.
I just started playing with Incus this week. So far really impressed. It's one of those tools that just feels well designed and high quality. I keep expecting the abstractions to leak and so far they haven't.
> But fundamentally, Proxmox is built around clicking buttons. It is a GUI-first paradigm.
Uhh, whut? It provides a button-y interface, but you can do everything via config files and `pct` on the command line if you prefer. I know that’s not full nix-style declarative, but you don’t have to mislead to sell me on the advantages of declarative infra.
NixOS has transformed my use of Proxmox. I configure, build, and deploy everything from my nix workstation. I don't need to use the PVE gui at all. Proxmox is just a target, and I've abstracted things enough to where I can deploy the same machines to libvirt on a local machine too. Why would I need to let my agent into my PVE box? I haven't looked at incus, but if I wanted to run the full stack declaratively, nixos and LLMs are so powerful now that I would probably just say to run libvirt and ZFS on nixos natively.
> I configure, build, and deploy everything from my nix workstation. I don't need to use the PVE gui at all.
I would love to know more about how you do this, particularly the deploy part. I'm considering moving away from Ansible, but haven't had the time to dedicate to exploring a similar Nix experience.
This is what I've built [1], and I'm using it daily for things like agent sandboxing when I deploy it builds a new image completely replaces the OS disk - on upgrade it only keeps its data drive mounted to /var.
I switched over to NixOS around a month ago from Ubuntu and it's been just a dream. I expected there to be some friction with installing things that aren't already built for NixOS, but honestly it's been easier with LLM+NixOS than it was with Ubuntu.
I don't if you know this, but there was a collection for LXD. Last time I checked, it didn't seem very popular so not maintained very well but it did work. Maybe that could be used for inspiration.
I remember Stéphane worked on adding support for incus containers to opentofu which seemed more popular than using ansible to describe the resources.
This seems very cool and I will probably try it, but I think I’m missing something. I run Proxmox so that I can have multiple VMs running on my NUC. This doesn’t really solve that right? I cant spin up a windows 11 vm one weekend for a random experiment.
Incus is roughly a frontend for qemu, so you can launch an empty VM and use the ISO to install the OS. You don't have to use a preconfigured base image.
VM support has been there for quite a while now and works nicely. I think that's what they're referring to. It started out with only system containers and then gained support for VMs and now there's work to support launching OCI images directly.
Another proxmox-esque project I've been watching is Sylve, a control plane for FreeBSD that provides a web interface to jails, bhyve VMs, and containers [1]. It's new-ish, but it looks like a possible sweet spot replacement for both proxmox and TrueNAS (which was originally also built on FreeBSD before they switched to Linux IIRC), at least for my homelab-ing use case. Potentially eventually for environments at greater scale also. The company behind it is a software consultancy and Sylve is built with their actual business needs in mind, and is BSD-licensed like the OS it runs atop.
One of the developers building Sylve gave a talk last year [2].
>By moving to NixOS, my entire infrastructure is defined in text files. This means my AI agents can read, understand, and even safely modify my infrastructure
You lost me there!!
Firstly, NixOS is hype, like everything being moved to Rust and failing miserably.
Secondly, "AI ... can .... safely modify my infrastructure", OP is either being a troll or haven't seem how the whole IT world is upside down because of those very same statements.
Thirdly, "my entire infrastructure is defined in text files", you clearly never heard of Ansible.
All my Proxmox LXC containers from DNS servers, to NGINX firewall aliases feeding OPNSense firewall rules, from Forgejo hosting my repos to PostgreSQL database, from Semaphore running my Ansible playbooks on schedule to *Arr collection, everything is fully infrastructure as code, there is no GUI.
I do not log into Proxmox to deploy my stuff, I enjoy CLI and Ansible makes everything like a walk in the park. I use Proxmox CLI tool "pct" for everything, even snapshots are CLI via "vzdump" and its config file.
My take from that post and comments resume in "hype" "not understanding processes" "seeing problems where there isn't one"
I only run Linux here, even my 3D printer runs Debian Netinst Linux. I am missing something here.
> Secondly, "AI ... can .... safely modify my infrastructure", OP is either being a troll or haven't seem how the whole IT world is upside down because of those very same statements.
That's exactly why you want your infrastructure defined in version controlled files with easy revert when something gets screwed up.
> Thirdly, "my entire infrastructure is defined in text files", you clearly never heard of Ansible.
Ansible is a half-assed version of what nix or even Puppet gets you. Having a version controlled record of which shell commands you ran doesn't help that much when you're still running random uncontrolled shell commands and hoping they do what you want them to.
NixOS is more than 20 years old, and virtually all of the things that make it compelling were already present a decade ago. If it's not for you, you'll know after you give it a try. But for those for whom it clicks, it's desirable because it just provides a more enjoyable computing experience. It's not more complicated than that.
> Secondly, "AI ... can .... safely modify my infrastructure", OP is either being a troll or haven't seem how the whole IT world is upside down because of those very same statements.
I do lots of Terraform work and some Nix work with LLM agents at my job. Is it worth it to rewrite a huge amount of whatever infrastructure-as-code your LLM agent generates? Hell yes; they generate way too much code and they make lots of mistakes. Are LLM agents still useful for experimentation via infrastructure-as-code? Also yes.
> Thirdly, "my entire infrastructure is defined in text files", you clearly never heard of Ansible.
I've used Ansible and Puppet at previous jobs. They don't manage state in a comparable way to NixOS. It just ain't the vibe. Domen Kozar wrote a decent blog post about the technical differences a decade ago: https://www.domenkozar.com/2014/03/11/why-puppet-chef-ansibl...
But the real reason is that those technical differences add up to a more pleasant experience for NixOS.
In my earlier days as a NixOS user, I used to get really excited with its design and how cool it is, and the neat technical properties that fall out of that (atomic upgrades! rollbacks! (and no filesystem snapshotting needed!)). I still think those things are awesome. But at the risk of feeding into your impression that "NixOS is hype", I've learned since then that the better pitch is about the subjectivity of using it: it feels good to use because experimentation is extremely cheap, reversible, transparent, and... fun. If you know, you know.
>NixOS is more than 20 years old, and virtually all of the things that make it compelling were already present a decade ago
20 years???
NixOS mention only started this year, I have been working in IT for the past 20 years, from e-commerce to banking and airline companies, I have never ever heard of NixOS.
Not until "social media influencers" started flooding YouTube with it this year.
We must be living in a completely different world then.
> We must be living in a completely different world then.
I'm not even a real Nix old-timer, but I've been using NixOS on the job for roles in IT operations and software development for more than 10 years now.
And for a few years before that, I used Nix on a personal basis as a college student.
I'm aware of the rapid growth in user interest, of course; I was there for it!
I haven't abandoned Proxmox yet, but the take here resonates with me. I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc. I have had pretty good luck managing Proxmox clusters with the Proxmox API (https://pve.proxmox.com/pve-docs/api-viewer/index.html) or just letting the agent shell in as root (lol). I built a very simple provisioning tool called vmfactory that takes some really somple config on disk, bakes a fresh qcow image, pushes it to proxmox and then configures networking and boots it. It's extremely rudimentary but has been working well for me.
I did abandon TrueNAS, however. It really is a locked-down appliance. Good luck installing custom software on the base OS. I have a domain-joined Ubuntu/ZFS box that inherits a lot of policy from FreeIPA and/or Ansible config that is all backed by files on disk. It's been really easy to orchestrate what many would consider overkill in my homelab because literally everything is represented in a single Github repo.
I run Ansible against Proxmox hosts to ensure the config on them is what I want, and then Terraform all VMs into place on them. It's not too far off from having your own mini-EC2, minus some of the nice trappings like load balancers.
At $DAYJOB, we use the Telmate terraform proxmox provider, too. Upgrades over the years have had a few papercuts (mainly when values are shifted around), but it has overall been an immense timesaver. A terraform repo contains the 2 colocation datacenter proxmox environments plus a local lab edition of the same builds (locallab uses containers vs full VM's on the "real" hardware so that I can run almost all of the configuration on my laptop). Saltstack gets deployed during the buildouts, which picks up the OS/ application configuration after the OS is deployed.
No agentic stuff on our stack, as our security posture can't afford that currently.
As for the load-balancing, I think the later versions have supported targeting proxmox clusters vs a single node, and the newest Proxmox can do resource auto-balancing. That might get you what you need
Ah, I mean load balancing like an AWS ALB; obviously one can use an Nginx or HAproxy for that, but it's not a primitive, you need to deploy a machine and run it.
They have auto-balancing now? Damn. I wrote a balancer using the Google OR-SAT solver because there was no VMWare DRS equiv.
I will consider switching, but with like 400 VMs now it'd be a lot of work. Do you happen to know if it lets you deal with disks separately from the VM host? Used to use that flow on VMware, taint a machine but not its stateful secondary disk, and bring it back up with a fresh image and re-attach a database disk or whatever.
>I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc.
I hardle ever access Proxmox GUI, everything is fully managed via Ansible playbooks, from deploying the LXC containers to updating them, from taking snapshot to syncing stuff to my TrueNAS is done via batching.
>I did abandon TrueNAS, however. It really is a locked-down appliance. Good luck installing custom software on the base OS
The most common mistake people seem to make when it comes to NAS, TrueNAS is a NAS not a Proxmox server.
I see people running dozens if not hundreds of services on UnRaid, one fart and the whole infrastructure, if I can call it that, goes to sh1t.
>But fundamentally, Proxmox is built around clicking buttons. It is a GUI-first paradigm. While you can automate it with Terraform or Ansible, it often feels like fighting the tool. State drift is real. You change a setting in the UI to debug something, forget about it, and six months later your “infrastructure as code” is out of sync with reality.
Share the whole thing :)
Again, if you are managing everything as IaC, why touching the UI to make changes??
OP POV I just shared above tells everything that is wrong and Proxmox is not it.
I've recently migrated to Incus and am enjoying it so far. Hopefully the good feelings last.
Roughly ten years ago, my homelab consisted of a dozen virtual machines running on SmartOS. I was not familiar with Illumos, and this was before it had a widely available web UI, but it was simple enough to use that these challenges didn't matter much. SmartOS was designed to boot reliably from USB flash storage, allowed me to use all my SATA ports for VM storage, and was my first "immutable" operating system. The primary focus on ZFS storage was another great quality of SmartOS.
Two moves and several years later, it was time to rebuild the lab, and I decided to go with Proxmox because it had decent ZFS support. Experience with Proxmox has been very good too. The GUI, many more virtualization features (in addition to the key ones I care about), and better hardware support through the Linux kernel have kept me on Proxmox for a long time.
Customizing my Proxmox installation always gave me anxiety. How could I defend my hypervisor from configuration drift? I wished there could be an immutable version of Proxmox.
Later on, I learned about govulcheck, which offers a novel dynamic/static analysis hybrid approach to vulnerability management. Nothing else out there does this (without teaming up with some huge company). I began to think that I should favor software solutions based on golang.
Ultimately, Incus (and IncusOS) fit this need very well. My IncusOS hosts excellent and I'm glad I can run Incus itself on most Linux distros - including NixOS!
I'll keep a small Proxmox host around for experimenting with new kernel features (Intel GVT-g / SR-IOV graphics) and old operating systems like Windows XP or anything else that needs special QEMU options.
I get the advantage of fully declarative systems such as Nix, but everything you do in proxmox via GUI can be done via CLI: https://pve.proxmox.com/pve-docs/#_command_line_interface
CLI is first class in proxmox, I use the qm command for managing vms all the time. The networking is also just a file in `/etc/network/interfaces` that I modify with vim as needed.
CLI is dandy but Terraform is even nicer. I run something like ten Proxmox clusters now around the world, and an AI agent can deploy a fully terraformed VM for me, tied into monitoring, asset mgmt, EDR, etc in a few minutes...
I run 20+… and it sure would be nice if Proxmox had a first party terraform provider. I have a lot of stuff I have to do via REST.
I concur highly. Well, make sure you buy your cheap support contract, help beta-test Proxmox Datacentre Manager, and push for it. They could probably just onboard one of the open source authors as a contractor or something if enough companies actually pony up the couple of grand per year per box. At 50K a server, paying like 2k for a proxmox license is a joke
Doing things via the CLI and editing config files doesn't automatically give you reproducability or audit trail. Last time I check etckeeper doesn't work with proxmox either (due to the proxmox FUSE mount in /etc).
I get your point and this is likely also not what you mean, but I am very happy with a simple git tracking approach that I use across Proxmoxes and VMs [1].
[1]: https://du.nkel.dev/blog/2026-05-16_rootless_docker_virtiofs...
I don't get their argument, I run GeForceNow on one of my Proxmox nodes in metal and there's no problem with it. It's a Debian underneath, after all. They didn't need to switch to NixOS to run Kodi.
Also, do they get PBS using ZFS snapshots? Do they get HA, live migration, shared storage, easy CephOS, easy snapshots, quick cloning? Do you really want to migrate a VM from one node to another using the command line when you're in some serious situation?
Sure, for a homelab this might be OK, but the UI does make things easy for a reason.and it's not a gimmick.
I agree on a lot of the points, though, I just set up a second cluster and it took over 3 work days because of how much repetitive work is needed to do so. To be able to just take a file with instructions, adjust it a bit and deploy would be so much easier.
I'm also considering migrating from Proxmox to Incus, but I'd look into IncusOS rather than having to manage the host OS myself.
I'm a big LXD and now Incus fan. But I went with NixOS rather than IncusOS for my latest build because I prefer the LTS linux kernel over the mainline kernel.
It's really hard to tell what the benefits of Incus over Ganeti are. The Incus site doesn't really lead with features or benefits that I can see. About the only things I could figure out are that Incus can manage containers and can manage instance creation through cloning a base volume, where ganeti (afaik) always creates volume and populates it. Well, and Incus seems to be MUCH more popular than Ganeti.
I searched the documentation but it wasn't really clear what its live migration and ZFS migration story is, but when I asked Claude to research it, it tells me that it supports live migration via ZFS snapshot replication, which is exactly what I'm looking for. I implemented a ganeti storage driver that does the same thing and am just getting ready to start testing it, but if Incus supports it I might look at moving that direction.
Anyone use Incus live migration with ZFS?
Yeah, one thing I've found with Nix is that documentation is generally very terse and assumes you're capable of filling the blanks yourself, even though you might not know they exist.
Like... this... this is not great documentation (I know I know, contribute myself): https://wiki.nixos.org/wiki/Incus
Within a few lines:
> To provide non-root access to the Incus server, you will want to add your user to the incus-admin group. Don't forget to reboot.
I mean I get that they probably mean /etc/group, but going on from there plenty of examples of "just change the config to use x" or similar.
I find I have a lot of trouble with updating with nixos. I really like the OS but I find that when I have something like Claude Code and I need to update to the newest version, I just have trouble. It just seems like a lot of work updating versions otherwise NixOS is really awesome and great.
I am satisfied with pve and qm commands on proxmox, but I wish there was a nix answer to pfsense where my network and firewall are defined declaratively
This! pfSense/OPNSense seem to me like perfect candidates for declarative configuration. Even though, technically speaking, they are since everything is being save to a config file, which you can backup and restore. It's just less declarative then you'd like it to be. Sounds like they could easily employ Nix for handling that, though.
It is a scripting language but for Mikrotiks you can scp a .rsc file and
I just started playing with Incus this week. So far really impressed. It's one of those tools that just feels well designed and high quality. I keep expecting the abstractions to leak and so far they haven't.
> But fundamentally, Proxmox is built around clicking buttons. It is a GUI-first paradigm.
Uhh, whut? It provides a button-y interface, but you can do everything via config files and `pct` on the command line if you prefer. I know that’s not full nix-style declarative, but you don’t have to mislead to sell me on the advantages of declarative infra.
NixOS has transformed my use of Proxmox. I configure, build, and deploy everything from my nix workstation. I don't need to use the PVE gui at all. Proxmox is just a target, and I've abstracted things enough to where I can deploy the same machines to libvirt on a local machine too. Why would I need to let my agent into my PVE box? I haven't looked at incus, but if I wanted to run the full stack declaratively, nixos and LLMs are so powerful now that I would probably just say to run libvirt and ZFS on nixos natively.
> I configure, build, and deploy everything from my nix workstation. I don't need to use the PVE gui at all.
I would love to know more about how you do this, particularly the deploy part. I'm considering moving away from Ansible, but haven't had the time to dedicate to exploring a similar Nix experience.
This is what I've built [1], and I'm using it daily for things like agent sandboxing when I deploy it builds a new image completely replaces the OS disk - on upgrade it only keeps its data drive mounted to /var.
[1] https://github.com/EnigmaCurry/nixos-vm-template
Just use Terraform against Proxmox and don't leave mysterious things nobody else knows how to work with.
What do you mean exactly? You have a nix workstation and a physically separate PVE server, and configure the second from the first?
Yes, I don't really touch the PVE server at all after the initial install. I just build and deploy from my Fedora workstation running nix.
[1] https://github.com/EnigmaCurry/nixos-vm-template
>The Agentic Multiplier
Oh man, you buried the lede there.
I switched over to NixOS around a month ago from Ubuntu and it's been just a dream. I expected there to be some friction with installing things that aren't already built for NixOS, but honestly it's been easier with LLM+NixOS than it was with Ubuntu.
edit: Thinko
I've been using Incus to support coding agents to run wild. I've been pretty happy about it, but I wish I understood it more.
I've been using incus for a while now and actually run it on a side project in production for the better part of a year. Rock solid performance.
Very happy to see more people Nixpilled. Haven't used incus but seems like an interesting tool for when I want to leave the comforts of my dotfiles.
Incus is great. I've been trying to revive an unmaintained ansible collection to manage incus resources https://github.com/sbstp/ansible-collection-incus
I don't if you know this, but there was a collection for LXD. Last time I checked, it didn't seem very popular so not maintained very well but it did work. Maybe that could be used for inspiration.
I remember Stéphane worked on adding support for incus containers to opentofu which seemed more popular than using ansible to describe the resources.
This seems very cool and I will probably try it, but I think I’m missing something. I run Proxmox so that I can have multiple VMs running on my NUC. This doesn’t really solve that right? I cant spin up a windows 11 vm one weekend for a random experiment.
It sounds like all his containers are Linux, so that's why Incus is such a good fit for him. For your use case, yeah, proxmox is likely a better fit.
This small project makes running Windows on Incus a breeze: https://github.com/antifob/incus-windows
Incus is roughly a frontend for qemu, so you can launch an empty VM and use the ISO to install the OS. You don't have to use a preconfigured base image.
I think incus can be a frontend for qemu, but it's primary mode of operating is to run containers. It's a fork of LXD.
VM support has been there for quite a while now and works nicely. I think that's what they're referring to. It started out with only system containers and then gained support for VMs and now there's work to support launching OCI images directly.
Another proxmox-esque project I've been watching is Sylve, a control plane for FreeBSD that provides a web interface to jails, bhyve VMs, and containers [1]. It's new-ish, but it looks like a possible sweet spot replacement for both proxmox and TrueNAS (which was originally also built on FreeBSD before they switched to Linux IIRC), at least for my homelab-ing use case. Potentially eventually for environments at greater scale also. The company behind it is a software consultancy and Sylve is built with their actual business needs in mind, and is BSD-licensed like the OS it runs atop.
One of the developers building Sylve gave a talk last year [2].
[1]: https://sylve.io/
[2]: https://youtu.be/wo4oD5UON30
How does Sylve compare to Webmin [1]?
[1]: https://docs.vultr.com/how-to-install-webmin-on-freebsd-14-0
>By moving to NixOS, my entire infrastructure is defined in text files. This means my AI agents can read, understand, and even safely modify my infrastructure
You lost me there!!
Firstly, NixOS is hype, like everything being moved to Rust and failing miserably.
Secondly, "AI ... can .... safely modify my infrastructure", OP is either being a troll or haven't seem how the whole IT world is upside down because of those very same statements.
Thirdly, "my entire infrastructure is defined in text files", you clearly never heard of Ansible.
All my Proxmox LXC containers from DNS servers, to NGINX firewall aliases feeding OPNSense firewall rules, from Forgejo hosting my repos to PostgreSQL database, from Semaphore running my Ansible playbooks on schedule to *Arr collection, everything is fully infrastructure as code, there is no GUI.
I do not log into Proxmox to deploy my stuff, I enjoy CLI and Ansible makes everything like a walk in the park. I use Proxmox CLI tool "pct" for everything, even snapshots are CLI via "vzdump" and its config file.
My take from that post and comments resume in "hype" "not understanding processes" "seeing problems where there isn't one"
I only run Linux here, even my 3D printer runs Debian Netinst Linux. I am missing something here.
> Secondly, "AI ... can .... safely modify my infrastructure", OP is either being a troll or haven't seem how the whole IT world is upside down because of those very same statements.
That's exactly why you want your infrastructure defined in version controlled files with easy revert when something gets screwed up.
> Thirdly, "my entire infrastructure is defined in text files", you clearly never heard of Ansible.
Ansible is a half-assed version of what nix or even Puppet gets you. Having a version controlled record of which shell commands you ran doesn't help that much when you're still running random uncontrolled shell commands and hoping they do what you want them to.
> Firstly, NixOS is hype
NixOS is more than 20 years old, and virtually all of the things that make it compelling were already present a decade ago. If it's not for you, you'll know after you give it a try. But for those for whom it clicks, it's desirable because it just provides a more enjoyable computing experience. It's not more complicated than that.
> Secondly, "AI ... can .... safely modify my infrastructure", OP is either being a troll or haven't seem how the whole IT world is upside down because of those very same statements.
I do lots of Terraform work and some Nix work with LLM agents at my job. Is it worth it to rewrite a huge amount of whatever infrastructure-as-code your LLM agent generates? Hell yes; they generate way too much code and they make lots of mistakes. Are LLM agents still useful for experimentation via infrastructure-as-code? Also yes.
> Thirdly, "my entire infrastructure is defined in text files", you clearly never heard of Ansible.
I've used Ansible and Puppet at previous jobs. They don't manage state in a comparable way to NixOS. It just ain't the vibe. Domen Kozar wrote a decent blog post about the technical differences a decade ago: https://www.domenkozar.com/2014/03/11/why-puppet-chef-ansibl...
But the real reason is that those technical differences add up to a more pleasant experience for NixOS.
In my earlier days as a NixOS user, I used to get really excited with its design and how cool it is, and the neat technical properties that fall out of that (atomic upgrades! rollbacks! (and no filesystem snapshotting needed!)). I still think those things are awesome. But at the risk of feeding into your impression that "NixOS is hype", I've learned since then that the better pitch is about the subjectivity of using it: it feels good to use because experimentation is extremely cheap, reversible, transparent, and... fun. If you know, you know.
>NixOS is more than 20 years old, and virtually all of the things that make it compelling were already present a decade ago
20 years???
NixOS mention only started this year, I have been working in IT for the past 20 years, from e-commerce to banking and airline companies, I have never ever heard of NixOS.
Not until "social media influencers" started flooding YouTube with it this year.
We must be living in a completely different world then.
> We must be living in a completely different world then.
I'm not even a real Nix old-timer, but I've been using NixOS on the job for roles in IT operations and software development for more than 10 years now.
And for a few years before that, I used Nix on a personal basis as a college student.
I'm aware of the rapid growth in user interest, of course; I was there for it!
I haven't abandoned Proxmox yet, but the take here resonates with me. I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc. I have had pretty good luck managing Proxmox clusters with the Proxmox API (https://pve.proxmox.com/pve-docs/api-viewer/index.html) or just letting the agent shell in as root (lol). I built a very simple provisioning tool called vmfactory that takes some really somple config on disk, bakes a fresh qcow image, pushes it to proxmox and then configures networking and boots it. It's extremely rudimentary but has been working well for me.
I did abandon TrueNAS, however. It really is a locked-down appliance. Good luck installing custom software on the base OS. I have a domain-joined Ubuntu/ZFS box that inherits a lot of policy from FreeIPA and/or Ansible config that is all backed by files on disk. It's been really easy to orchestrate what many would consider overkill in my homelab because literally everything is represented in a single Github repo.
I yanked vmfactory out and into a standalone repo if anyone is interested: https://github.com/whalesalad/vmfactory
https://github.com/Telmate/terraform-provider-proxmox ...
I run Ansible against Proxmox hosts to ensure the config on them is what I want, and then Terraform all VMs into place on them. It's not too far off from having your own mini-EC2, minus some of the nice trappings like load balancers.
At $DAYJOB, we use the Telmate terraform proxmox provider, too. Upgrades over the years have had a few papercuts (mainly when values are shifted around), but it has overall been an immense timesaver. A terraform repo contains the 2 colocation datacenter proxmox environments plus a local lab edition of the same builds (locallab uses containers vs full VM's on the "real" hardware so that I can run almost all of the configuration on my laptop). Saltstack gets deployed during the buildouts, which picks up the OS/ application configuration after the OS is deployed.
No agentic stuff on our stack, as our security posture can't afford that currently.
As for the load-balancing, I think the later versions have supported targeting proxmox clusters vs a single node, and the newest Proxmox can do resource auto-balancing. That might get you what you need
Ah, I mean load balancing like an AWS ALB; obviously one can use an Nginx or HAproxy for that, but it's not a primitive, you need to deploy a machine and run it.
They have auto-balancing now? Damn. I wrote a balancer using the Google OR-SAT solver because there was no VMWare DRS equiv.
Switched to that one some time ago, more features and adapts faster to changes in newer proxmox versions https://github.com/bpg/terraform-provider-proxmox
I will consider switching, but with like 400 VMs now it'd be a lot of work. Do you happen to know if it lets you deal with disks separately from the VM host? Used to use that flow on VMware, taint a machine but not its stateful secondary disk, and bring it back up with a fresh image and re-attach a database disk or whatever.
>I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc.
I hardle ever access Proxmox GUI, everything is fully managed via Ansible playbooks, from deploying the LXC containers to updating them, from taking snapshot to syncing stuff to my TrueNAS is done via batching.
>I did abandon TrueNAS, however. It really is a locked-down appliance. Good luck installing custom software on the base OS
The most common mistake people seem to make when it comes to NAS, TrueNAS is a NAS not a Proxmox server.
I see people running dozens if not hundreds of services on UnRaid, one fart and the whole infrastructure, if I can call it that, goes to sh1t.
The amount of b.s. in this article is staggering.
I agree!!!
The "my entire infrastructure is defined in text files" alone tells OP never hear of Ansible.
So how are you reading this sentence?
> While you can automate it with Terraform or Ansible, it..
>But fundamentally, Proxmox is built around clicking buttons. It is a GUI-first paradigm. While you can automate it with Terraform or Ansible, it often feels like fighting the tool. State drift is real. You change a setting in the UI to debug something, forget about it, and six months later your “infrastructure as code” is out of sync with reality.
Share the whole thing :)
Again, if you are managing everything as IaC, why touching the UI to make changes??
OP POV I just shared above tells everything that is wrong and Proxmox is not it.