HashiTalks 2024: Mastering Terraform Testing, a layered approach to testing complex infrastructure

HashiCorp Ambassador Mattias Fjellstrom talks about a five-tiered strategy for mastering Terraform testing, in an online community event HashiTalks 2024.

The first layer consists of both imperative and declarative validations. Imperative validation is where you run CLI commands such as terraform init, terraform validate, and terraform plan. You validate not only your HCL configuration of your resources, but also your providers and desired changes. Declarative validation leverages validation blocks within various Terraform components, like variables, outputs, resources, etc.

variable "location" {
  type = string

  validation {
    condition     = contains([
      "swedencentral",
      "westeurope",
      "northeurope"
    ], var.location)
    error_message = "Use an approved location!"
  }
}

With validation in place, Layer 2 implements the Terraform test framework, starting from Terraform v1.6.0, emphasizing the importance of testing the entire module interface.

When you write tests for your Terraform modules you want to make sure to test the full module interface. What is part of the module interface? Generally:

• Variables.

• Outputs.

• Any resource that is externally visible and of importance to your module consumers.

• Any behaviour or functionality that is exposed by the resources or data sources that are part of your module.

By default, tests within Terraform create real infrastructure, where you can override the normal testing behaviour by updating the command attribute within a run block, i.e. command = plan instructs Terraform to not create new infrastructure, which allows test authors to validate logical operations and custom conditions within their infrastructure in a process analogous to unit testing.

Most of your tests should use command = plan since these tests will be fast. When it comes to testing your validation logic many (all?) of the tests will be using the expect_failure array instead of assert blocks.

run "should_not_allow_invalid_location"{
    command = plan
    variables {
        location = "westus"
    }
    expect_failures = [
        var.location
    ]
}

Layer 3 adds integration testing, where you have a few options on how to handle dependencies. The options range from testing dependencies by creating real infrastructure, to using a helper module, to using static values or mocks.

Turn your compatibility requirements into tests, for instance if your last three major versions of a module is expected to be supported together with a third-party module, make sure that this is the case by using integration testing.

Layer 4 involves testing end-to-end scenarios with complex infrastructure and operations. The author demonstrates how helper modules are required to perform things such as setting up dependencies and query for information.

Take the testing to the extreme by identifying use-cases or scenarios that your modules are intended to support.

Layer 5 shifts left by identifying patterns in your validation code and creating policies from them. The benefit of policies is that it can be easier to administer and offers different enforcement levels. The author picks HashiCorp Sentinel policies as it is integrated with Terraform Cloud.

Introduce policy-as-code in your organization. You might do this from day 1 or you might do it whenever your organization is ready for it. Start small with a set of common policies for the whole organization, then expand the scope and the number of policies successively.