Terraform Modules

Terraform Modules

Hello Bits Lovers! Many people send me an email suggesting one article about Terraform Modules. Some people make some confusion between the modules and resources. We will understand what it is why we should use it.

Terraform is becoming more popular, and more companies and start-ups are adopting it because it improves the productivity and quality of our infrastructure and supports a high demand for your business.

Terraform and Complex Infrastructure Projects

So, before understanding what terraform modules are, let’s present some problems that may occur when our infrastructure gets bigger.

You will design increasingly complicated configurations as you handle your infrastructure with Terraform. There is no inherent limitation to the complexity for just one Terraform configuration file or project folder, so it is likely to persist in writing and editing our configuration files in one folder. However, if we do, we may experience some problems like the ones below:

Challenges

  • Finding and understanding the directory from your Terraform files gets more complex.
  • It could be dangerous to change our configuration because sometimes we can’t understand all risks or the cause-effect of new changes and what will affect other configurations.
  • Duplication will become a problem because we are not reusing the same configuration for the same resource in another project.
    • For example, when we decided to update one configuration because we found one error, we later noticed that the same error occurs in different environments, like production, Dev, and QA. So, we need to go through all those parts and perform the same changes to fix one issue.
  • You may desire to share specific parts from our configuration between teams or other projects. And is not a good approach to cutting and pasting the code blocks between different projects because it will increase the difficulty of maintaining the configuration, even if somehow you have an excellent method to versioning your code.

What is Terraform Module?

So, knowing the issues that we can face when our configuration gets more extensive, you may already understand the meaning of Terraform module and what it could help us.

Terraform modules help us group different infrastructure resources into just one unified resource. So, it implies that we can reuse them later with possible customizations without duplicating the resource blocks each time we require them, which is practical for big projects with complicated designs.

Also, we can create and customize module representatives using input variables we describe and pull information from them utilizing outputs. We can also use the pre-made modules available publicly at the Terraform Registry from creating our custom modules. Finally, the Engineers can customize inputs like our source modules.

Re-usable modules are determined utilizing the same configuration language ideas we use in root modules. Most typically, modules use:

  • Output values return variable values to the calling module, populating arguments elsewhere.
  • Input variables to receive variable values from the calling module.
  • Resources determine one or more infrastructure entities that the Module will handle.

Terraform Directory Structure

From the files structure point of view, let’s see what Terraform modules are.

To design our Module, we create a new directory and put one or more .tf files on that directory, just as we would do for a root module. Because the Terraform can read modules either from our local computer or remote repositories. If many configurations reuse a module, we may desire to set it in its version control repository, like Git.

Modules can also reach other modules utilizing a module block. Still, we suggest preserving the module tree relatively flat and using module composition as an option for a deeply-nested tree of modules. This drives the particular modules more manageable to reuse in various combinations.

The directory below is a structure of a module and root module.

.
├── README.md
├── main.tf
├── variables.tf
├── outputs.tf

However, it’s important to highlight that the files above aren’t required and don’t have a defined meaning to Terraform. We can use just one .tf file or any approach that you preferer for the structure of your project. But, of course, one standard is better.

Let’s each files goal:

main.tf – Its main file of your Module. However, you can create more files in your project, and the other files can be called any name different than the main.tf. For example, I like to create files for each kind of resource. To store a configuration for one EC2 instance, I would make a new file, webserver.tf, and define all code there.

variables.tf – On this file, the main goal is to define all variables needed for a module. The variables should be configured as arguments in the module block when others utilize our Module. Since all Terraform values must be specified, any variables that don’t have a default value will become required arguments. An error will happen in the “plan” phase if not set.

We can override the default values from the variables, providing module arguments. We use other files to define the values of the variables, usually by *.tfvars extension.

output.tf – This file contains all definitions of all output for the Module. Without those outputs, the modules wouldn’t be possible or exist because it’s the only approach we can have to do one Module to communicate to other parts of your infrastructure modules.

For example, let’s suppose that we have one Module that creates some Security Groups, and later we need to use that Security Group to attach to an EC2 or Load Balance. After making it, how do we return the Security Group ID to the Module that creates the Load Balance or EC2? So, we use the output to send information to other modules that depend on it. The outputs are variables that are visible outside the Module. 

Also, at the end of execution, the Terraform prints out all values to check the values.

Important Note 📝

We mentioned before that we define all variables values on files with extension *.tfvars. And be careful with that file; this file does not belong to the modules because it contains values specific to your environment, such as a HOST and credentials to connect to a database, so it doesn’t make sense to share it. Also, it contains sensitive information, so try to avoid committing those files, especially on a public repository like GitHub, you can add this file on the .gitignore

Files Created by Terraform

Terraform State Files

After executing your terraform, some files you will see some new files that Terraform created, like terraform.tfstate.terraform(directory), aren’t part of your Module; we don’t need to share them when you are you aren’t part of your Module sharing the module files. It means that if you are using Git to store your configuration file, you can add those files to the .gitignore file to avoid committing them by mistake.

terraform.tfstate – These files include your Terraform state and how Terraform keeps track of the connection between our configuration and our infrastructure. The Terraform creates those files when we run “terraform apply”.

Terraform Plugin Directory

.terraform – This folder includes the plugins and modules that Terraform used to provision our infrastructure. We can delete it after we deploy our configuration. Because when we execute “terraform init,” Terraform will create the folder again.

Terraform Module Example

Like we saw before, modules can talk with another module, which allows us to include the child module’s resources into the configuration straightforwardly. Modules can also be reached numerous times, either within the same configuration or in different configurations, permitting the resource code to be packaged and reused.

Let’s suppose that the directory below is our root module:

.
├── README.md
├── main.tf
├── variables.tf
├── outputs.tf

Imagine installing a new module into our root module to install one EC2 with a web server apache2. So, we need to create a folder, called modules, and then load all files from that Module inside that folder.

├── README.md
├── main.tf
├── modules
│  └── webserver
├── variables.tf
├── outputs.tf

So, here we added the directory modules. Inside it, we add another directory called webserver, which will contain the main.tf with all configurations needed for the EC2 instance.

Now that we have all files in place, we need to link our new Module to our root module.

Add a Child Module

Right now, our root module looks like the one below:

provider "aws" {
  alias = "usa1"
  region = "us-east-1"
}
terraform {
  backend "s3" {
    bucket  = "bitslovers-terraform-state"
    key     = "api/prod/terraform.tfstate"
    region  = "us-east-1"
    encrypt = true
  }
}
data "aws_vpc" "myvpc" {
  id = "vpc-2ccfbf2d"
}

Our root module doesn’t have any resources yet.

To add a new module means to include the contents of that Module inside the configuration with the required values for its input variables. We add modules within other modules utilizing module blocks like the example below:

module "ec2_instances" {
  source  = "modules/webserver"
  name           = "bitslovers-webserver"
  asg_min = 2
  asg_max = 4
  ami                    = "ami-0c4314620f811e0f4"
  instance_type          = "m5a.xlarge"
  vpc_security_group_ids = [module.vpc.default_security_group_id]
  subnet_id              = module.vpc.public_subnets[0]
  tags = {
    Created-By   = "Terraform"
    Environment = "production"
  }
}

After the module keyword, the label “webserver” is a local module name, which the calling module can utilize to refer to this module instance.

Terraform Module Source

This is for Terraform module path. We must always specify the source argument for all modules because it is a meta-argument determined by Terraform. It’s the only required argument besides the input values defined inside the Module. Its value is either the path to your local computer pointing to a directory containing the Module’s code files or a remote module source that Terraform should download and use, for example, from the Git repository. Also, remember that the value must be a literal text with no template sequences; expressions are not supported. 

We can call the same modules multiple times inside the same root module but with different input values.

One important note: After adding a new module or editing or removing modules from the root module, we must execute the “terraform init” again to install the new modules or remove them. However, we need to use “terraform init –update” to update an existing module inside the root module.

Terraform Meta Arguments

Terraform supports a couple of meta arguments that help our productivity and organization manage our configuration.

Using Count on Terraform Modules

In our previous post, we have learned how to use the terraform count, and we can use it to add new modules into our root module to create several instances of a module from a unique module block. Also, besides the count, we can use the for_each.

Terraform Module Depends On

We can also use the depends_on to create explicit dependencies between a module and the listed targets. Explicitly defining a dependency is just required when a resource or module depends on some other resource’s behavior but doesn’t access any of that resource’s data, for example, where we need to wait for one resource to be created before another.

Terraform Modules Provider

Here, it’s essential to understand how the providers work with modules. For example, you may notice that on the root module above, we defined a provider:

provider "aws" {
  alias = "usa1"
  region = "us-east-1"
}

When we define the provider configuration on the root module, it passes to all child modules. If not specified, the child module inherits all the default provider configurations from the calling module.

We can override the provider from the root module, defining inside the module block:

module "load_balance" { 
	source = "./another-module" 
	providers = { 
		aws = aws.usw2
	 }
}

When to Specify Providers

There are two principal motivations to utilize the provider’s argument:

  • Using other default provider configurations for a child module.
  • Configuring a module that needs numerous configurations of the identical provider.

Terraform Module Output

The resources described in a module are encapsulated, so the parent module cannot read their attributes directly. However, the child module can create output values to export specific values to be accessed from the parent module.

Migrate Existing Resources into Modules

Shifting resource blocks from one module into various small child modules forces the Terraform to interpret the new location as a completely different resource. Consequently, the “terraform plan” command will destroy all resource instances that point to the old location and create new resources at the new location.

We can utilize refactoring blocks to register each resource instance’s old and new locations to keep existing resources. This leads Terraform to minister existing objects at the old location as if they had been initially created at the corresponding new location.

Terraform Module Best Practices

In numerous methods, Terraform modules are equivalent to the ideas of libraries, packages, or modules available in most programming languages and deliver several of the same advantages. Just like nearly any non-trivial application, real-world Terraform configurations should nearly consistently use modules to offer the benefits noted above.

Use local modules to manage and encapsulate your code. Even if we aren’t utilizing or sharing remote modules, managing your configuration in like modules from the start will immensely decrease the burden of maintaining and correcting your configuration as your infrastructure increases in complexity.

Also, begin composing your configuration with modules in mind. Even for modestly complicated Terraform configurations controlled just for one person, you’ll see the advantages of employing modules outweigh the time it takes to utilize them correctly.

Finally, we can append our new module code to our root module. And will look like the below example:

The main.tf from the root module:

provider "aws" {
  alias = "usa1"
  region = "us-east-1"
}
terraform {
  backend "s3" {
    bucket  = "bitslovers-terraform-state"
    key     = "api/prod/terraform.tfstate"
    region  = "us-east-1"
    encrypt = true
  }
}
data "aws_vpc" "myvpc" {
  id = "vpc-2ccfbf2d"
}
module "ec2_instances" {
  source  = "modules/webserver"
  name           = "bitslovers-webserver"
  asg_min = 2
  asg_max = 4
  ami                    = "ami-0c4314620f811e0f4"
  instance_type          = "m5a.xlarge"
  vpc_security_group_ids = [module.vpc.default_security_group_id]
  subnet_id              = module.vpc.public_subnets[0]
  tags = {
    Created-By   = "Terraform"
    Environment = "production"
  }
}

Conclusion

It was an extensive article about the Terraform module. And honestly, once you know how to work with Terraform Modules, you will unlock a lot of alternatives on how to grow your infrastructure code in a very well-organized way.

Also, sometimes you don’t need to create a module from scratch. We can find any module from any cloud provider on GitHub and the Terraform Registry, and we can download and make our changes, saving a lot of time.

Boost your Terraform skills:

How to create complex expressions using Terraform Template.

What is the difference between Locals and Terraform Variables?

How to use the Terraform Data on your modules.

What are the advantages of Terraform Output?

Learn how to create multiple copies from the same resource on Terraform.

How to create a Pipeline on Gitlab to execute a Terraform code?

Leave a Comment

Your email address will not be published.