Diffing desired vs actual state in TF is problematic

Hey folks,

There is an interesting new feature suggestion that has been added to the GitHub issue trackers for both Terraform and OpenTofu. The command is tf plan -light, and it aims to make the plan operation quicker.

The idea is a little hacky but it's smart. The proposals suggest using the hash of the ordered arguments to each resource as a way to know when a resource's desired state has changed.

With that knowledge, you would only need to do a refresh on resources the developer wants to change. Currently, TF has to diff the full desired configuration vs actual infrastructure. The hashing approach could be a nice optimization for advanced users who are dealing the dreaded Terralith (more info linked below if you’ve never heard of this term).

Another idea to simplify the diffing process is to use git. This is similar to how Terramate solves this problem. With git, you can check the current branch's changes against the main branch. This is less complicated, available without any changes to TF, and would only support git, but for this niche of a feature, maybe that's okay?

To be honest, I'm not sure I think tf plan -light should be implemented. I’m writing this because even the suggestions highlights how painful TF can be.

This is a band-aid for the biggest shortcoming in IaC today: the fact that diffing desired vs actual infrastructure state is slow slow slow when you're managing too many resources in one state. A high resource count is, sadly, the most common code smell in IaC.

This sucks and if you've been in IaC for any amount of time, you know this pain. I would rather see smart methods to deal with this root issue than niche workarounds, but that doesn't mean this approach wouldn't solve pain and help some.

What do you think? Would you use tf plan -light?

May your state files always be small,

Matt @ Masterpoint

PS If you want to learn more about the Terralith, we have a blog post for that; check it out.