3 reasons to keep TF state files small

Hey folks,

Are you working with Infrastructure as Code (IaC) at scale, with hundreds of managed resources? Or more?

If so, be sure to keep your IaC Root Modules, i.e. “state files” small.

Here are three reasons why:

1. Minimize the blast radius. If you manage both your database and your application cluster in the same state file, then changes to resources may be entangled. For example, the application cluster may need an urgent environment variable update and the database cluster may need a major upgrade at the same time. It can be tedious and confusing to make sure that the simpler environment variable update is deployed without the database upgrade. You can get around this by targeting specific resources, but that is a manual operation. Not something you want to deal with.

2. Slow tf plan and tf apply times. The more resources you are managing in one state file, the more API requests that your infrastructure as code tool needs to do when diffing your desired versus actual state. If you start managing hundreds, thousands or even tens of thousands of resources in your root module, this diff process slows down. I’ve seen plan and apply times well over an hour. That slows down your ability to manage and operate your infrastructure and does not scale.

3. Least privilege is impossible. If you manage your network and application cluster in the same root module, you no longer have the ability to say "the network team can change only network resources and the application engineering team is limited to changing application cluster settings". That reduces your ability to implement PoLP within your organization, can lead to unwanted bugs and incidents, and weakens your security posture.

The logical conclusion of larger and larger state files is the Terralith, where you keep all TF state in a single root module. This Masterpoint article explains why this is a terrible idea.

May your state files always be small,

Matt @ Masterpoint

PS: Time flies! A few months ago, OpenTofu celebrated its year anniversary. More details about the project’s progress here.