Don't Let Terraliths Drag Your Team Down

Hey folks,

One thing that can drag down teams is giant bloated infrastructure as code (IaC) projects.

We call these terraliths. It's one of the things that we see all the time and we need to deal with all the time at our clients. We always recommend breaking up the terralith.

What is a terralith?

It's when you have too many resources in a root module in a state file. That results in slow plan and apply times. You end up with role-based access control (RBAC) issues. You have huge blast radius concerns since you cannot independently work on some set of resources without potentially affecting another set of resources.

This slows down organizations and increases operational risk.

I'll give you an example from a client. We went into an organization recently. They weren't that large but they had one root module that had 40,000 resources in it. 40,000!

That many resources caused the root module to plan so slowly that it took a half hour to plan. And half hour an hour to apply. At a certain point that root module just started to fail. It would just OOM because Terraform Cloud couldn't didn’t have enough memory to hold all 40K resources.

That is an extreme example, but it was also a part of a critical component of their business. It was how they onboarded clients. We wrote a case study with this client and it's very in-depth.

If you're not planning your IaC root modules well, you can end up getting into a position where things blow up. To avoid this, build reusable root modules with tight scopes and clean interfaces. These root modules should be repeatable.

You don't want everyone to be building their own root modules all the time. If you do, you end up with a ton of small snowflakes.

What you want to do is build root modules driven by the variables that you pass in. Then you have something that you can take to another team and say, "Hey, here's how we deploy databases. Hey, here's how we deploy a Kubernetes cluster." This shares best practices and makes sure infra is consistent.

Does this sound like something you’ve run into? If you want a step-by-step guide on breaking up terraliths, here’s our in-depth article on how to break one up. The post gives you a checklist, walks you through planning, and provides in depth migration advice.

May all your root modules plan quickly,

Matt @ Masterpoint

PS Are you interested in being on a devops-focused podcast? Reply to me with the topic you want to discuss and I’m happy to intro you to the right podcast. Or, if you want to chat about terraliths, grab some time on my calendar here.