The Terraform and Ansible are Infrastructures as Code tools that help us DevOps Engineers automate complex tasks in a scalable fashion. They both reach and perform the same goal.
Few people know we can combine both technologies, giving us more flexibility to resolve complex problems related to infrastructure and provisioning resources in the Cloud, no matter which cloud provider you adopt.
But why combine both technologies?
Why Terraform and Ansible?
Terraform is an excellent tool and delivers everything we need to create infrastructure in the Cloud. We can easily configure and define any configuration, like Security, Networking, or Storage. However, there is some limitation in specific scenarios. Tip: Use Terraform Module to create your ansible scripts.
Terraform Limitation
Suppose you created a configuration that deploys several EC2 instances on AWS using Terraform. Such a goal we can easily reach with Terraform. But, when we need to go deeper and make a lot of instructions within EC2, install a couple of applications, and perform several configurations, you will notice that we don’t have too many features that could save us a lot of time.
Terraform can define instructions that could be executed within EC2, for example, relying on a specific property within an EC2 called userdata.
In some scenarios, you can use null_resource.
User Data Approach
When you create an instance in Amazon EC2, you can give user data to the server that can be utilized to execute common automated configuration duties and even execute scripts after the server starts. Also, you can provide two types of user data: shell scripts and cloud-init directives. You can also provide this data in the launch template as plain text.
This userdata can be specified when you declare an EC2 instance using the aws_instance resource block from Terraform.
And to customize userdata using Terraform, we have to rely on another feature called Terraform Template File.
But, Ansible outshines in this area to execute instructions inside an instance. You can use it in any circumstance and on any cloud provider, such as Azure, AWS, GCP, or Digital Ocean, because we can perform the instructions using SSH.
Terraform with Ansible: Pull or Push?
Before working with Terraform and Ansible, you must define one crucial step before starting.
Let’s talk about the two possibilities.
Pull
First is the pull method, where the server we are provisioning will pull the ansible-playbook from some remote storage, like S3 or FTP.
So, in this scenario, we execute the pipeline, for example, using GitLab. First, the GitLab Runner will run the Terraform; later, when the EC2 comes to live, the userdata will pass instructions to download the ansible-playbook and execute the Ansible instructions, and we are done.
A critical advantage of the pull method, we don’t need to worry about the network between the GitLab Runner and your local computer to reach the new server. Thus it’s a little tricky because when we automate the whole process, sometimes it’s pretty hard to guarantee that we will not have firewall rules blocking our computer or that the pipeline server will reach the new server.
Push
The second is the push method when we push the changes from our computer or remote server to the new server that we are provisioning using an SSH connection to transfer our playbooks and start installing and configuring our new server. But, as we mentioned before, we always need to guarantee that we will have a clear path to reach the new server.
Terraform and Ansible Example
Now, let’s see how to do it. Again, thinking in makes this a more straightforward process. First, we created a project in our Bits Lovers official repository.
The terraform and ansible example project it’s pretty simple. It was created only to show how the workflow should work.
The project will create a simple EC2 instance, and when the instance is running, the Ansible script will start executing the playbook.
User Data to call Ansible
The script below will start our Ansible script in the server startup.
webserver/user_data.sh:
#!/bin/bash -xe
# Make sure that the AMI is Amazon Linux 2
amazon-linux-extras install ansible2=2.8 -y
yum install -y zip unzip
cd /root/
aws s3 cp s3://${s3_bucket}/terraform/${zip} /root/ansible_script.zip
unzip ansible_script.zip
#Export Variable Session
export S3_BUCKET=${s3_bucket}
export environment=${environment}
ansible-playbook -i "127.0.0.1," provision.yml --connection=local
So, the script above helps us guarantee that we have Ansible and other dependencies installed. Later, we downloaded the Ansible script from our S3; here, we could store it anywhere.
Export Environment Variables
Did you notice the syntax ${}? They are variables from Terraform, and we pass those values using Terraform Template File. For all environment variables, “export” will be visible for the Ansible scripts. We can use this approach to send argument values to change the workflow from Ansible and make it more dynamic.
In the last line, from our webserver/user_data.sh, we call the ansible-playbook to execute our instructions.
Now, let’s see the Terraform side.
resource "aws_instance" "web" {
ami = var.ami
instance_type = var.instance_type
vpc_security_group_ids = ["${aws_security_group.inst.id}"]
subnet_id = "${element(random_shuffle.sni.result, 0)}"
user_data = "${data.template_file.user_data.rendered}"
key_name = "${var.key_pair}"
iam_instance_profile = "${aws_iam_instance_profile.iam_profile.name}"
root_block_device {
volume_type = "gp3"
volume_size = "50"
}
lifecycle {
create_before_destroy = true
}
tags = {
environment = "${var.environment}"
}
}
data "template_file" "user_data" {
template = "${file("${path.module}/user_data.sh")}"
vars = {
zip = "bits-${data.archive_file.ansible_script.output_md5}.zip"
s3_bucket = "${var.s3_bucket}"
environment = "${var.environment}"
}
}
The Terraform Data above is the location from Terraform where we replace the variable values from our webserver/user_data.sh. So, first, we define and retrieve the values from Terraform, and later, you give those values to Ansible through the user_data.sh script.
Create the User Data using Terraform
And to use the user_data on your EC2, you need to specify the property “user_data” on the “aws_instance” in our case above is: data.template_file.user_data.rendered.
The project “Terraform and Ansible” example will have within the root one folder called “ansible_script .” That contains all the instructions that we need to execute inside the EC2. So, it means that we need to send it to the instance. So, let’s see how we do that.
data "archive_file" "ansible_script" {
type = "zip"
source_dir = "${path.module}/ansible_script/"
output_path = "ansible_script.zip"
}
resource "aws_s3_bucket_object" "ansible_script" {
bucket = "${var.s3_bucket}"
key = "terraform/bits-${data.archive_file.ansible_script.output_md5}.zip"
source = "ansible_script.zip"
etag = "${data.archive_file.ansible_script.output_md5}"
}
First, we wrap all files in a single Zip file using the archive_file data block from Terraform. Then, later, we can upload this Zip file as an S3 Object. As you noticed, we are using the variable “zip” to store the file name (prefix) from our Zip file inside the user_data. So, when the instance startup, we are sure it will download the correct file because we also use md5 in its name.
Before continuing, here is one last tip about this integration between Terraform and Ansible. The variables in the user_data file need to be exported on Ansible on your provision.yml file. For example:
---
- name: Bits Server
hosts: all
become: true
become_method: sudo
vars:
temp_dir: /tmp/devops
environment: "{{ lookup('env','ENVIRONMENT') }}"
Inside the vars property, we need to execute a lookup on all environment variables from the server, look for a specific variable and store it as a variable on Ansible.
If you would like to execute that example, remember to change all Terraform Variables values from the file terraform.tfvars, to match with your resources from your AWS account.
Using Provisioners on Terraform with Ansible
One second alternative is using Terraform with remote provisioners. Follow the example above.
resource "aws_instance" "web" {
provisioner "remote-exec" {
# Install Python for Ansible Or use userdata.sh for it.
inline = ["cat /etc/os-release || true",
"cat /etc/system-release || true",
]
connection {
type = "ssh"
user = "ec2-user"
host = "${self.private_ip}"
private_key = "${file(var.ssh_key_private)}"
}
}
provisioner "local-exec" {
command = "ansible-playbook -u ec2-user -i '${self.private_ip},' --private-key ${var.ssh_key_private} -T 600 \"${path.module}/ansible_script/provision.yml\""
}
}
The disadvantage of the example above it’s that you need to secure the SSH private key or password. Besides, to worry about the connectivity. But it’s good to be aware that it is possible.
Conclusion
Terraform and Ansible it’s a fantastic combinations to automate your infrastructure. And because you can keep them on a Git repository, it becomes easier to keep tracking and create a new environment once necessary.
It’s not worth comparing Ansible vs. Terraform. Both deliver good features.
