Terraform State [Complete Guide]

Written by Bits Lovers on 13 Feb 2022

Terraform needs to track state about your infrastructure. This state tells Terraform how your configuration maps to real resources already running in the cloud, stores metadata about those resources, and speeds things up when you’re managing large infrastructure. If you’re using workspaces, check out Terraform Workspace.

Terraform State File

Where is Terraform State Stored?

By default, Terraform stores state in a local file called terraform.tfstate at the root of your project. For teams, storing state remotely works better. S3 with DynamoDB is the common choice on AWS.

Terraform uses this local state file to plan and apply changes to your infrastructure. Before any operation, Terraform refreshes the state to reflect what’s actually running in the cloud.

The core purpose of state is to link resources in your configuration to actual objects in your cloud provider. When Terraform creates a resource, it records that object in the state. Later changes to your configuration tell Terraform what to update or destroy based on this mapping.

Terraform Update State Manually

The state file is plain JSON, but you shouldn’t edit it directly. Terraform provides the terraform state command for making changes safely.

The command-line output plays well with Unix tools like grep and awk. Using the CLI also protects you from accidentally corrupting the state format. Terraform maintains backward compatibility with the state format across versions.

Terraform aims for a one-to-one mapping between your configuration and real cloud resources. Terraform should be the only thing creating or destroying these resources. If you need to import existing resources with terraform import, or tell Terraform to forget something with terraform state rm, you need to manage that one-to-one relationship yourself.

Terraform State File Format

State snapshots are JSON. Newer Terraform versions can read state from older versions. The format may change in future releases, so don’t build tools that parse it directly unless you’re prepared to maintain them.

Terraform provides better integration points for external tools. The terraform output -json flag gives you a clean view of root module outputs. The terraform show -json command outputs the full state snapshot and saved plan files, including the original state when the plan was created.

For automation, a common pattern is running terraform apply followed immediately by capturing the output for your CI/CD pipeline. Store that snapshot as an artifact so other tools can read the state without calling Terraform directly.

Terraform State File Example

A real state file can get large. Here’s a simplified example:

{
  "version": 4,
  "terraform_version": "1.10.0",
  "serial": 3,
  "lineage": "364d8449-e325-c78f-132a-c1c5791fec40",
  "outputs": {},
  "resources": [
    {
      "module": "module.webserver",
      "mode": "data",
      "type": "archive_file",
      "name": "data-source",
      "provider": "provider.archive",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "id": "1d0f71c7661280bb00571c024a501161e96b4bce",
            "output_base64sha256": "8r+/wgwvAMsDn4iimiRzY9z9GZwsaTh7gNlLSwa/E60=",
            "output_md5": "fb7feb1d7731375bb0f17d18a42e111f",
            "output_path": "data-source.zip",
            "output_sha": "1d0f71c7664180bb00571c024a501169e96b4bce",
            "output_size": 4796,
            "source_dir": "modules/webserver/data-source/",
            "type": "zip"
          }
        }
      ]
    },
    {
      "mode": "data",
      "type": "aws_vpc",
      "name": "myvpc",
      "provider": "provider.aws"
    }
  ]
}

Terraform State File Best Practices

1. Always backup your Terraform state file. If you lose it, you lose the ability to manage your existing cloud resources.

2. Don’t commit state to version control. State often contains sensitive data like passwords and connection strings. Plus, running Terraform against stale state from your repo is a fast way to break things.

3. Use remote state with a backend. For teams, this is non-negotiable. AWS S3 with DynamoDB locking is the standard approach. HashiCorp also supports Azure Blob, GCS, Consul, and more.

4. Create separate state for each environment. Mixing dev and prod in the same state is an accident waiting to happen. State corruption is hard to fix.

5. Never edit state files manually. Use terraform state mv to rename resources, terraform state rm to remove tracking, and terraform import to bring in existing resources. These commands handle the bookkeeping correctly.

Terraform State Backup

Commands that modify state automatically create a backup first. Use the -backup flag to control where backups go.

Read-only commands like terraform state show and terraform state list don’t create backups.

You can’t disable automatic backups for state-modifying operations. If you need to clean up backup files, do it manually.

Terraform State Remote

Remote State

All Terraform commands work with state, whether it’s local or remote. Remote state just means the file lives in S3 or another backend instead of your filesystem.

Operations on remote state take longer since data transfers over the network. Backups still go to disk automatically. The CLI commands work the same way.

Terraform State Backend using S3

Here’s how to configure S3 as your backend:

terraform {
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "prod/terraform.tfstate"
    region = "us-west-1"
    encrypt = true
    dynamodb_table = "terraform-state-lock"
  }
}

Enable S3 bucket versioning so you can recover from accidental deletions. If you’re using DynamoDB for locking, know that this approach is deprecated. The use_lockfile argument enables native S3-based locking instead, which is the recommended path forward for new configurations.

Terraform State Command

The terraform state command group manages state directly. As your infrastructure grows, you’ll hit cases where you need to modify state without destroying resources.

This is a nested command structure. You get subcommands like list, mv, rm, show, pull, and push.

Terraform State Management

Terraform State Lock

Some backends lock state during operations that modify it. This prevents two people from running terraform apply at the same time and corrupting the state.

Terraform handles locking automatically. You usually don’t see any message about it.

Terraform State Force Unlock

If a Terraform command gets interrupted (you Ctrl+C it, or your laptop dies), the lock can get stuck. When you try to run Terraform again, you’ll see an error like this:

╷
│ Error: Error acquiring the state lock
│
│ Error message: ConditionalCheckFailedException: The conditional request failed
│ Lock Info:
│   ID:        c1024f1b-b615-05cf-f512-e49ed2852284
│   Path:      <...>
│   Operation: OperationTypePlan
│   Who:       <...>
│   Version:   1.0.3
│   Created:   2022-02-28 12:14:31.112341 +0000 UTC
│
│ Terraform acquires a state lock to protect from simultaneous writes.
│ Resolve the issue above and try again.

To unlock manually:

terraform force-unlock <LOCK_ID>

In this case:

terraform force-unlock c1024f1b-b615-05cf-f512-e49ed2852284

If the normal unlock fails, wait a few minutes. AWS DynamoDB locks have a default timeout. If that doesn’t work, find and kill the stuck Terraform process on your machine.

Terraform State Refresh

The terraform refresh command updates state to match real cloud resources. This command is deprecated. Use terraform apply -refresh-only instead if you need to sync state without making changes.

Move Terraform State

Rename resources or move them between modules:

terraform state mv

This moves an item to a new address. You can rename a resource, move it into or out of a module, or transfer it to a different state file entirely. It’s useful for refactoring configurations without destroying infrastructure.

State modifications create a backup automatically.

Removing Objects from Terraform State

Remove Terraform’s tracking for a resource that still exists in the cloud:

terraform state rm

This tells Terraform to forget the object without destroying it. Check what exists in state first with terraform state list.

You can’t use wildcards with terraform state rm. Pipe the output through grep and xargs instead:

terraform state list | grep "module.webserver" | xargs terraform state rm

Remove a single object:

terraform state rm module.webserver.aws_instance.app

Show Resource Attributes from Terraform State

Inspect a specific resource:

terraform state show

This shows all attributes for one resource. Useful for checking if your configuration actually created what you expected.

terraform state show module.webserver.aws_instance.app

The output includes the instance ID, IP addresses, tags, and everything else Terraform knows about that resource.

Push Local State to Remote

Overwrite remote state with a local file:

terraform state push

This overwrites the remote state. Without the -force flag, Terraform protects you from overwriting newer state with older state (checks lineage and serial).

terraform state push [options] PATH

Pull State to Local

Download state from the backend:

terraform state pull

This outputs the current state to stdout. Pipe it to a file or jq to inspect values:

terraform state pull > terraform.tfstate

The output matches your locally-installed Terraform version.

Conclusion

Terraform state is foundational. If you’re learning Terraform, understanding state is the key to everything else. State is what makes Terraform know what it manages, what changed, and what needs updating.

The terraform state subcommands give you safe ways to modify state without editing JSON directly. Get comfortable with mv, rm, list, and show. You’ll need them as your infrastructure grows.