Terraform and Ansible

Terraform and Ansible

The Terraform and Ansible are Infrastructures as Code tools that help us DevOps Engineers automate complex tasks in a scalable fashion. They both reach and perform the same goal.

Few people know we can combine both technologies, giving us more flexibility to resolve complex problems related to infrastructure and provisioning resources in the Cloud, no matter which cloud provider you adopt.

But why combine both technologies?

Why Terraform and Ansible?

Terraform is an excellent tool and delivers everything we need to create infrastructure in the Cloud. We can easily configure and define any configuration, like Security, Networking, or Storage. However, there is some limitation in specific scenarios.

Terraform Limitation

Suppose you created a configuration that deploys several EC2 instances on AWS using Terraform. Such a goal we can easily reach with Terraform. But, when we need to go deeper and make a lot of instructions within EC2, install a couple of applications and perform several configurations, you will notice that we don’t have too many features that could save us a lot of time.

Terraform can define instruction that could be executed within EC2, for example, relying on a specific property within an EC2, called userdata

User Data Approach

When you create an instance in Amazon EC2, you have the possibility of giving user data to the server that can be utilized to execute common automated configuration duties and even execute scripts after the server starts. Also, you can provide two types of user data: shell scripts and cloud-init directives. You can also provide this data into the launch template as plain text.

This userdata can be specified when you declare an EC2 instance using the aws_instance resource block from Terraform. 

And to customize userdata using Terraform, we have to rely on another feature called Terraform Template File.

 But, Ansible outshines in this area to execute instructions inside an instance. You can use it in any circumstances and on any cloud provider, such as Azure, AWS, GCP, or Digital Ocean because we can perform the instructions using SSH.

Terraform with Ansible: Pull or Push?

Before working with Terraform and Ansible, you must define one crucial step before starting.

Let’s talk about the two possibilities. 

Pull

First is the pull method, where the server we are provisioning will pull the ansible-playbook from some remote storage, like S3 or FTP. 

So, in this scenario, we execute the pipeline, for example, using GitLab. First, the GitLab Runner will run the Terraform; later, when the EC2 comes to live, the userdata will pass instructions to download the ansible-playbook and execute the Ansible instructions, and we are done.

A critical advantage of the pull method, we don’t need to worry about the network between the GitLab Runner or your local computer to reach the new server. Thus it’s a little tricky because when we automate the whole process, sometimes it’s pretty hard to guarantee that we will not have firewall rules blocking our computer or that the pipeline server will reach the new server.

Push

The second is the push method when we push the changes from our computer or remote server to the new server that we are provisioning using an SSH connection to transfer our playbooks and start installing and configuring our new server. But, as we mentioned before, we always need to guarantee that we will have a clear path to reach the new server.

Terraform and Ansible Example

Now, let’s see how to do it. Again, thinking in makes this a more straightforward process. First, we created a project in our Bits Lovers official repository.

The terraform and ansible example project it’s pretty simple. It was created only to show how the workflow should work.

The project will create a simple EC2 instance, and when the instance is running, the Ansible script will start executing the playbook.

User Data to call Ansible

The script below will start our Ansible script in the server startup.

webserver/user_data.sh:

#!/bin/bash -xe

# Make sure that the AMI is Amazon Linux 2
amazon-linux-extras install ansible2=2.8 -y
yum install -y zip unzip

cd /root/
aws s3 cp s3://${s3_bucket}/terraform/${zip} /root/ansible_script.zip

unzip ansible_script.zip

#Export Variable Session
export S3_BUCKET=${s3_bucket}
export environment=${environment}

ansible-playbook -i "127.0.0.1," provision.yml --connection=local

So, the script above helps us guarantee that we have Ansible and other dependencies installed. Later, we downloaded the Ansible script from our S3; here, we could store it anywhere.

Export Environment Variables

Did you notice the syntax ${}? They are variables from Terraform, and we pass those values using Terraform Template File. All environment variables we use “export” will be visible for the Ansible scripts. We can use this approach to send argument values to change the workflow from Ansible and make it more dynamic.

In the last line, from our webserver/user_data.sh, we call the ansible-playbook to execute our instructions.

Now, let’s see the Terraform side. 

resource "aws_instance" "web" {
  ami                    = var.ami
  instance_type          = var.instance_type
  vpc_security_group_ids = ["${aws_security_group.inst.id}"]
  subnet_id              = "${element(random_shuffle.sni.result, 0)}"
  user_data              = "${data.template_file.user_data.rendered}"
  key_name               = "${var.key_pair}"
  iam_instance_profile   = "${aws_iam_instance_profile.iam_profile.name}"

  root_block_device {
    volume_type = "gp3"
    volume_size = "50"
  }

  lifecycle {
    create_before_destroy = true
  }

  tags = {
    environment = "${var.environment}"
  }
}


data "template_file" "user_data" {
  template = "${file("${path.module}/user_data.sh")}"
  vars = {
    zip                       = "bits-${data.archive_file.ansible_script.output_md5}.zip"
    s3_bucket                 = "${var.s3_bucket}"
    environment               = "${var.environment}"
  }
}

The Terraform Data above is the location from Terraform where we replace the variable values from our webserver/user_data.sh. So, first, we define and retrieve the values from Terraform, and later, you give those values to Ansible through the user_data.sh script.

Create the User Data using Terraform

And to use the user_data on your EC2, you need to specify the property “user_data” on the “aws_instance” in our case above is: data.template_file.user_data.rendered.

The project “Terraform and Ansible” example will have within the root one folder called “ansible_script .” That contains all the instructions that we need to execute inside the EC2. So, it means that we need to send it to the instance. So, let’s see how we do that.

data "archive_file" "ansible_script" {
  type        = "zip"
  source_dir  = "${path.module}/ansible_script/"
  output_path = "ansible_script.zip"
}

resource "aws_s3_bucket_object" "ansible_script" {
  bucket = "${var.s3_bucket}"
  key    = "terraform/bits-${data.archive_file.ansible_script.output_md5}.zip"
  source = "ansible_script.zip"
  etag   = "${data.archive_file.ansible_script.output_md5}"
}

First, we wrap all files in a single Zip file, using the archive_file data block from Terraform. Then, later, we can upload this Zip file as an S3 Object. As you noticed, we are using the variable “zip” to store the file name (prefix) from our Zip file inside the user_data. So, when the instance startup, we are sure it will download the correct file because we also use md5 on its name.

Before continuing, one last tip about this integration between Terraform and Ansible. The variables in the user_data file need to be exported on Ansible on your provision.yml file. For example:

---
- name: Bits Server
  hosts: all
  become: true
  become_method: sudo
  vars:
    temp_dir: /tmp/devops
    environment: "{{ lookup('env','ENVIRONMENT') }}"

Inside the vars property, we need to execute a lookup on all environment variables from the server, look for a specific variable and store it as a variable on Ansible.

If you would like to execute that example, remember to change all Terraform Variables values from the file terraform.tfvars, to match with your resources from your AWS account. 

Using Provisioners on Terraform with Ansible

One second alternative is using Terraform with remote provisioners. Follow the example above.

resource "aws_instance" "web" {

  provisioner "remote-exec" {
    # Install Python for Ansible Or use userdata.sh for it.
    inline = ["cat /etc/os-release || true",
      "cat /etc/system-release || true",
    ]
  
    connection {
      type        = "ssh"
      user        = "ec2-user"
      host        = "${self.private_ip}"
      private_key = "${file(var.ssh_key_private)}"
      
    }
  }
  
  provisioner "local-exec" {
    command = "ansible-playbook -u ec2-user -i '${self.private_ip},' --private-key ${var.ssh_key_private} -T 600 \"${path.module}/ansible_script/provision.yml\""
  }
}

The disadvantage of the example above it’s that you need to secure the SSH private key or password. Besides, to worry about the connectivity. But, it’s good to be aware that it is possible.

Conclusion

Terraform and Ansible it’s a fantastic combination to automate your infrastructure. And because you can keep them on a Git repository, it becomes easier to keep tracking and create a new environment once necessary.

It’s not worth comparing the Ansible vs. Terraform. Both deliver good features.

Leave a Comment

Your email address will not be published.