Many people suggest an article about Terraform Modules. Some people have difficulties distinguishing between the modules and resources. You will learn what it is and why you should use it in your projects.
Terraform is becoming more popular, and more companies and start-ups are adopting it because it improves the productivity and quality of our infrastructure and supports the environment that requires a high volume of changes to improve your business.
Terraform and Complex Infrastructure Projects
So, before understanding what terraform modules are, let’s show you some problems that may occur when your infrastructure gets more extensive and complex.
Usually, new projects contain a small number of resources and files. But, after some releases and improvements, more resources and files are added to the projects.
Would you convert your project, which contains only two resources?
In that scenario, you can agree that it is not needed.
But, for one project that deploys, for example, a whole new environment like VPC, Subnet, Gateway, Bastion, Security Group, and some instances?
In that example, it would create at least 70 resources. So it looks like a great idea to design modules, right?
If you disagree with me, let me point out some difficulties and challenges if you don’t choose to design modules for a big project.
Challenges
- Finding and understanding the directory from your Terraform files gets more complex.
It could be risky to change our project because sometimes, you may not understand all risks or the cause-effect beforehand when you push the new changes and what will affect other configurations.
- Duplication will become a problem: because you are not reusing the same configuration for the same resource in another project.
- You decided to update one configuration because you found one issue. Later you noticed that the same error occurs in different environments, like production, Dev, and QA. So, you will need to review all parts and make changes to fix one issue. Did you get it?
- You may desire to share specific parts of your code between teams or other projects. And is not a good approach to cutting and pasting the code blocks between different projects because it will increase the difficulty of maintaining the configuration, even if somehow you have an excellent method for versioning your code.
What is Terraform Module?
So, knowing the issues that we can face when our configuration gets more extensive, you may already understand the meaning of Terraform module and what it could help us.
Terraform modules help us group different infrastructure resources into just one unified resource. So, it implies that you can reuse them later with possible customizations without duplicating the resource blocks each time you require them, which is practical for big projects with complicated designs.
Also, you can create and customize module representatives using input variables. Also, you can describe and pull information from them utilizing outputs.
Besides your modules, you can also use the pre-made modules available publicly at the Terraform Registry to create our custom modules. Yet, you can customize inputs to fit your needs better.
Re-usable modules are determined by utilizing the same configuration language ideas in root modules. Most typically, modules use:
- Output values return variable values to the calling module, populating arguments elsewhere, especially when it needs to share values between modules.
- Input variables to receive variable values from the calling module.
- Resources determine one or more infrastructure entities that the Module will handle.
Terraform Directory Structure
From the folder and file structure point of view, let’s see what Terraform modules look like:
To design our Module, you create a new directory and put one or more .tf files on that directory, just as we would do for a root module. However, because Terraform can read modules either from our local computer or remote repositories, if many configurations reuse a module, we may desire to set it in its version control repository, like Git.
Modules can also reach other modules utilizing a module block. Still, it’s recommended to preserve the module tree relatively flat and use module composition as an option for a deeply-nested tree of modules. This drives the particular modules more manageable to reuse in various combinations.
The directory below is a structure of a a root module.
. ├── README.md ├── main.tf ├── variables.tf ├── outputs.tf
However, it’s important to highlight that the files above aren’t required and don’t have a defined meaning to Terraform.
You can use just one .tf file or any approach that you preferer for the structure of your project. But, of course, one standard is better than nothing.
So, the command line operations will read and interpret any file with extension .tf.
Let’s analyze each file:
main.tf – It’s the main file of your Module. However, you can create more files in your project, and the other files can be called any name different than the main.tf.
For example, I like to create files for each kind of resource if it depends on several configurations. So, to store a configuration for one EC2 instance, I would make a nefw file, webserver.tf, and define all code there.
variables.tf – In this file, the main goal is to define all variables needed for a module. The variables should be configured as arguments in the module block. In addition, for all Terraform, you must specify values. An error will happen in the “plan phase if not set.
For instance, you can override the default values from the variables, providing module arguments. Also, you use other files to define the values of the variables, usually by the *.tfvars extension.
output.tf – This file contains all definitions of all output for the Module. Without those outputs, the modules wouldn’t be possible or exist because you wouldn’t take advantage of one Module to communicate to other parts of your infrastructure or another different module.
For example, let’s suppose that you created a Module that creates some Security Groups, and later we need to use that Security Group to attach to an EC2 or Load Balance. After making it, how do you return the Security Group ID to the Module that creates the Load Balance or EC2? So, you can use the output block to declare and send its value to other modules that depend on it. The outputs are variables that are visible outside the Module.
Also, at the end of execution, the Terraform prints out all values from all outputs you have declared, making it easier to check the values when your Terraform finishes.
Important Note 📝
I mentioned to you before that if you define all variables values on files with extension *.tfvars.
Be careful with that file;
This file does not belong to the modules because it contains values specific to your environment, such as a HOST and credentials to connect to a database, so sharing it doesn’t make sense.
Also, it contains sensitive information, so try to avoid committing those files, especially on a public repository like GitHub; you can add this file on the .gitignore
Files Created by Terraform
Terraform State Files
After executing your terraform, you will notice some files were created automatically in your project root folder, like terraform.tfstate, and .terraform(directory), aren’t part of your Module;
Also, you don’t need to share them aren’t part of your Module. It means that if you are using Git to store your configuration file, you can also add those files to the .gitignore file to avoid committing them by mistake.
terraform.tfstate – These files include your Terraform state and how Terraform keeps track of the connection between our configuration and our infrastructure. The Terraform creates those files when we run “terraform apply”.
Terraform Plugin Directory
.terraform – This folder includes the plugins and modules that Terraform used to provision our infrastructure. We can delete it after we deploy our configuration. Because when we execute “terraform init,” Terraform will create the folder again.
Terraform Module Example
As you saw before, modules can talk with another module, allowing us to include the child module’s resources into the configuration.
Modules can be reached numerous times, either within the same or in different configurations, permitting the resource code to be packaged and reused.
Let’s suppose that the directory below is our root module (main):
. ├── README.md ├── main.tf ├── variables.tf ├── outputs.tf
Imagine installing a new module into our root module to install one EC2 with a web server apache2. So, we need to create a new folder, called modules, and load all files from that Module inside that folder.
├── README.md ├── main.tf ├── modules │ └── webserver ├── variables.tf ├── outputs.tf
So, here we added the directory modules. Inside it contains another directory called webserver, which will hold another main.tf with all configurations needed for the EC2 instance.
Easy, right? But, if you have more files for the webserver? What can you do it?
Simple: Add all files that belong to the webserver to modules/webserver.
Now that you have all files in place, you need to link the new Module to the root module.
Add a Child Module
Right now, our root module looks like the one below:
provider "aws" {
alias = "usa1"
region = "us-east-1"
}
terraform {
backend "s3" {
bucket = "bitslovers-terraform-state"
key = "api/prod/terraform.tfstate"
region = "us-east-1"
encrypt = true
}
}
data "aws_vpc" "myvpc" {
id = "vpc-2ccfbf2d"
}
Our root module doesn’t have any resources yet.
Adding a new module means including the contents of that module inside the configuration with the required values for its input variables. We add modules within other modules utilizing module blocks like the example below:
module "ec2_instances" {
source = "modules/webserver"
name = "bitslovers-webserver"
asg_min = 2
asg_max = 4
ami = "ami-0c4314620f811e0f4"
instance_type = "m5a.xlarge"
vpc_security_group_ids = [module.vpc.default_security_group_id]
subnet_id = module.vpc.public_subnets[0]
tags = {
Created-By = "Terraform"
Environment = "production"
}
}
After the module keyword, the label “webserver” is a local name which the calling module can use to refer to this module instance.
Terraform Module Source
What is the source?
You must always specify the source argument for all modules block because it is a meta-argument determined by Terraform.
It’s the only required argument besides the input values defined inside the Module.
Can you use a Git repository or a Local Directory?
Its value is either the path to your local computer pointing to a directory containing the Module’s code files or a remote module source that Terraform should download and use, for example, from the Git repository. Also, remember that the value must be a literal text with no template sequences; expressions are not supported.
Ok, but if I need to reuse or call the same module multiple times? Is it possible?
Yes, you can call the same modules multiple times inside the same root module but with different input values.
One important note: After adding a new module or editing or removing modules from the root module. Are we good to go? No. You must execute the “terraform init” again to install or remove the new modules, depending on what you did. However, you may need to use “terraform init –update” to update an existing module inside the root module.
Terraform Meta Arguments
Terraform supports several meta arguments that help your productivity and organization of your code.
Using Count on Terraform Modules
In our previous post on how to use the terraform count, you learned that it is possible to add new modules into our root module to create several instances of a module from a unique module block. Also, besides the count, you can use the for_each.
Terraform Module Depends On
Sometimes, our modules may depend on another resource or another module. So, how can you resolve it? You can also use the depends_on to create explicit dependencies between a module and the listed targets. Explicitly defining a dependency is just required when a resource or module depends on some other resource’s behavior but doesn’t access any of that resource’s data, for example, where you need to wait for one resource to be created before another.
Terraform Modules Provider
Here, it’s essential to understand how the providers work with modules. For example, you may notice that on the root module above, we defined a provider:
provider "aws" {
alias = "usa1"
region = "us-east-1"
}
Interesting fact: What happens when you declare a provider on the root module?
When you define the provider configuration on the root module, it passes to all child modules. If not specified, the child module inherits all the default provider configurations from the calling module.
You may be asking yourself, how override it?
You can override the provider from the root module, defining it inside the module block. See the example below:
module "load_balance" {
source = "./another-module"
providers = {
aws = aws.usw2
}
}
When to Specify Providers
Which scenario may require specifying an extra provider?
There are two principal motivations for utilizing the provider’s argument:
- Using other default provider configurations for a child module.
- Configuring a module that needs numerous configurations of the identical provider.
Terraform Module Output
I told you about the output block. Let me go deep a little more on the modules perspective:
The resources described in a module are encapsulated, so the parent module cannot read their attributes directly. However, the child module can create output values to export specific values to be accessed from the parent module.
Migrate Existing Resources into Modules
Shifting resource blocks from one module into various small child modules forces the Terraform to interpret the new location as a completely different resource. Consequently, the “terraform plan” command will destroy all resource instances that point to the old location and create new resources at the new location.
How to avoid it? For instance, it will be a big problem in a production environment.
You can utilize refactoring blocks to register each resource instance’s old and new locations to keep existing resources. This leads Terraform to minister existing objects at the old location as if they had been initially created at the corresponding new location.
Terraform Module Best Practices
In numerous methods, Terraform modules are equivalent to the ideas of libraries, packages, or modules available in most programming languages and deliver several of the same advantages. Like nearly any non-trivial application, real-world Terraform configurations should consistently use modules to offer the benefits noted above.
Use local modules to manage and encapsulate your code. Even if you aren’t utilizing or sharing remote modules, managing your configuration in like modules from the start will immensely decrease the burden of maintaining and correcting your configuration as your infrastructure increases in complexity.
Also, begin composing your configuration with modules in mind. Even for modestly complicated Terraform configurations controlled just for one person, you’ll see the advantages of employing modules outweigh the time it takes to utilize them correctly.
Finally, let’s our new module code to our root module:
The main.tf from the root module:
provider "aws" {
alias = "usa1"
region = "us-east-1"
}
terraform {
backend "s3" {
bucket = "bitslovers-terraform-state"
key = "api/prod/terraform.tfstate"
region = "us-east-1"
encrypt = true
}
}
data "aws_vpc" "myvpc" {
id = "vpc-2ccfbf2d"
}
module "ec2_instances" {
source = "modules/webserver"
name = "bitslovers-webserver"
asg_min = 2
asg_max = 4
ami = "ami-0c4314620f811e0f4"
instance_type = "m5a.xlarge"
vpc_security_group_ids = [module.vpc.default_security_group_id]
subnet_id = module.vpc.public_subnets[0]
tags = {
Created-By = "Terraform"
Environment = "production"
}
}
Conclusion
It was an extensive article about the Terraform Module. And honestly, once you know how to work with Terraform Modules, you will unlock a lot of alternatives on how to grow your infrastructure code in a very well-organized way.
Also, sometimes you don’t need to create a module from scratch. We can find any module from any cloud provider on GitHub and the Terraform Registry and download and make your changes, saving a lot of time, see our following example of modules: Auto Scaling Lifecycle Hooks.
