146

I need to deploy a list of GCP compute instances. How do I loop for_each through the "vms" in a list of objects like this:

    "gcp_zone": "us-central1-a",
    "image_name": "centos-cloud/centos-7",
    "vms": [
      {
        "hostname": "test1-srfe",
        "cpu": 1,
        "ram": 4,
        "hdd": 15,
        "log_drive": 300,
        "template": "Template-New",
        "service_types": [
          "sql",
          "db01",
          "db02"
        ]
      },
      {
        "hostname": "test1-second",
        "cpu": 1,
        "ram": 4,
        "hdd": 15,
        "template": "APPs-Template",
        "service_types": [
          "configs"
        ]
      }
    ]    
}
3
  • Terraform uses a declarative language, consequently there are no if, for or while statements. You can simulate these however, to some extent, in some cases thanks to the countproperty. Read this excellent article about how to reproduce conditions or loops with Terraform: blog.gruntwork.io/…
    – arvymetal
    Oct 28, 2019 at 16:26
  • 3
    Beginning in Terraform 0.12.6 resources can use for_each blocks: 0.12.6 Changelog. For what it's worth, if/else conditionals have been in Terraform for a couple of versions now: Terraform docs: Conditional Expressions Oct 29, 2019 at 0:14
  • Here a blog post containing all possible for loops explanation of for loops in terraform - stackoverflow.com/questions/58594506/…
    – Rahul Wagh
    Jul 6 at 7:48

9 Answers 9

211

I work a lot with iterators in Terraform, they always gave me bad headaches. Therefore I identified five of the most common iterator patterns (code examples are given below), which helped me construct a lot of nice modules (source).

  1. Using for_each on a list of strings
  2. Using for_each on a list of objects
  3. Using for_each to combine two lists
  4. Using for_each in a nested block
  5. Using for_each as a conditional

Using for_each and a list of strings is the easiest to understand, you can always use the toset() function. When working with a list of objects you need to convert it to a map where the key is a unique value. The alternative is to put a map inside your Terraform configuration. Personally, I think it looks cleaner to have a list of objects instead of a map in your configuration. The key usually doesn't have a purpose other than to identify unique items in a map, which can thus be constructed dynamically. I also use iterators to conditionally deploy a resource or resource block, especially when constructing more complex modules.

1. Using for_each on a list of strings

locals {
  ip_addresses = ["10.0.0.1", "10.0.0.2"]
}

resource "example" "example" {
  for_each   = toset(local.ip_addresses)
  ip_address = each.key
}

2. Using for_each on a list of objects

locals {
  virtual_machines = [
    {
      ip_address = "10.0.0.1"
      name       = "vm-1"
    },
    {
      ip_address = "10.0.0.1"
      name       = "vm-2"
    }
  ]
}    

resource "example" "example" {
  for_each   = {
    for index, vm in local.virtual_machines:
    vm.name => vm # Perfect, since VM names also need to be unique
    # OR: index => vm (unique but not perfect, since index will change frequently)
    # OR: uuid() => vm (do NOT do this! gets recreated everytime)
  }
  name       = each.value.name
  ip_address = each.value.ip_address
}

3. Using for_each to combine two lists

locals {
  domains = [
    https://example.com
    https://stackoverflow.com
  ]
  paths = [
    /one
    /two
    /three
  ]
}
    
resource "example" "example" {
  # Loop over both lists and flatten the result
  urls = flatten([
    for domain in local.domains : [
      for path in local.paths : {
        domain = domain
        path   = path
      }
    ]
  ]))
}

4. Using for_each on a nested block

# Using the optional() keyword makes fields null if not present
variable "routes" {
  type = list(
    name   = string
    path   = string
    config = optional(object({
      cache_enabled = bool
      https_only    = bool
    }))
  default = []
}

resource "example" "example" {
  name = ...
  
  dynamic "route" {
    for_each = {
      for route in var.routes :
      route.name => route
    }
    content {
      # Note: <top_level_block>.value.<object_key>
      name = route.value.name
    }
    
    dynamic "configuration" {
      # Note: <top_level_block>.value.<optional_object_key>
      for_each = route.value.config != null ? [1] : []
      content {
        cache_enabled = route.value.config.cache_enabled
        https_only    = route.value.config.https_only
      }
    }
  }

5. Using for_each as a conditional

variable "deploy_example" {
  type        = bool
  description = "Indicates whether to deploy something."
  default     = true
}

# Using count and a conditional, for_each is also possible here.
# See the next solution using a for_each with a conditional.
resource "example" "example" {
  count      = var.deploy_example ? 0 : 1
  name       = ...
  ip_address = ...
}

variable "enable_logs" {
  type        = bool
  description = "Indicates whether to enable something."
  default     = false
}

resource "example" "example" {
  name       = ...
  ip_address = ...

  # Note: dynamic blocks cannot use count!
  # Using for_each with an empty list and list(1) as a readable alternative. 
  dynamic "logs" {
    for_each = var.enable_logs ? [] : [1]
    content {
      name     = "logging"
    }
  }
}
5
  • 5
    This is better than the official documentation
    – Ali Nazari
    Feb 10 at 12:18
  • 1
    In the conditional section, shouldn't the count line be var.deploy_something ? 0 : 1 or am I misunderstanding?
    – Andy Madge
    Feb 20 at 13:43
  • @AndyMadge you are correct, I changed it accordingly! Feb 21 at 12:56
  • 1
    One of the best stackoverflow answers for Terraform. I've bookmarked this and find myself coming back every other week. Thanks :)
    – philmph
    May 31 at 11:47
  • @philmph thanks, great to hear that :) This is why I am on stackoverflow in the first place. May 31 at 12:39
106

Seem's like I found what to do. If you pass not the maps of maps but the list of maps you can use such code

resource "google_compute_instance" "node" {
    for_each = {for vm in var.vms:  vm.hostname => vm}

    name         = "${each.value.hostname}"
    machine_type = "custom-${each.value.cpu}-${each.value.ram*1024}"
    zone         = "${var.gcp_zone}"

    boot_disk {
        initialize_params {
        image = "${var.image_name}"
        size = "${each.value.hdd}"
        }
    }

    network_interface {
        network = "${var.network}"
    }

    metadata = {
        env_id = "${var.env_id}"
        service_types = "${join(",",each.value.service_types)}"
  }
}

It will create actual number of instance and when you remove for example middle one of three(if you create three:)), terraform will remove what we asked.

3
  • 1
    This works well. The list(object)) is converted to a map that can be used as the for_each value. Just select a key (e.g hostname) that is unique
    – svinther
    May 1, 2020 at 10:18
  • 5
    Care to share what : vm.hostname => vm does in the for_each line: for_each = {for vm in var.vms: vm.hostname => vm}? Dec 13, 2021 at 17:56
  • 8
    @user3508953 - This is a way of setting the "key" and "value" for the resource, where vm.hostname is the key and vm is the value. So in this example, we uniquely identify each resource in the loop by hostname. If you had multiple VMs in our list with the same hostname but different CPUs, then we might rewrite as "${vm.hostname}:${vm.cpu}" => vm. This allows terraform to track the resources by a key you define... IIRC Terraform tracks by index otherwise (which can be problematic if you reorder things) Dec 27, 2021 at 16:22
51

From Terraform 1.3, you can use the for_each and objects with modules like the following:

modules/google_compute_instance/variables.tf

variable "hosts" {
    type = map(object({
        cpu           = optional(number, 1)
        ram           = optional(number, 4)
        hdd           = optional(number, 15)
        log_drive     = optional(number, 300)
        template      = optional(string, "Template-New")
        service_types = list(string)
      }))
    }

modules/google_compute_instance/main.tf

resource "google_compute_instance" "gcp_instance" {
  for_each = {
    for key, value in var.hosts :
    key => value
  }

  hostname      = each.key
  cpu           = each.value.cpu
  ram           = each.value.ram
  hdd           = each.value.hdd
  log_drive     = each.value.log_drive
  template      = each.value.template
  service_types = each.value.service_types
}

servers.tf

module "gcp_instances" {
    source = "./modules/google_compute_instance"

    hosts = {
        "test1-srfe" = {
            hdd           = 20,
            log_drive     = 500,
            service_types = ["sql", "db01", "db02"]
        },
        "test1-second" = {
            cpu           = 2,
            ram           = 8,
            template      = "APPs-Template",
            service_types = ["configs"]
        },
    }
}

Of course, you can add as many variables as needed and use them in the module.

5
  • 1
    It's nice but variable is dynamically provided each time. Oct 31, 2019 at 9:02
  • Common variables can be added separately as variables and reused as var.variable. The unique ones have to be declared separately anyway. Nov 1, 2019 at 18:10
  • 3
    You're a hero. Finally I've found a proper example, from start to end.
    – pijemcolu
    May 12, 2021 at 9:31
  • Code was updated to use optional variables with defaults from TF 1.3, and to use the host objects' keys as hostnames instead of duplicating the code. Dec 29, 2022 at 12:55
  • 1
    Wonderful, this worked for me. for_each = { for key, value in local.instances : key => value } If your data type is tuple, this is the only way do it I think. Please correct me If I am wrong. Apr 7 at 14:45
9

You can do the following:

for_each = toset(keys({for i, r in var.vms:  i => r}))
cpu = var.vms[each.value]["cpu"]

Assuming you had the following:

variable "vms" {
    type = list(object({
        hostname        = string
        cpu             = number
        ram             = number
        hdd             = number
        log_drive       = number
        template        = string 
        service_types   = list(string)
    }))
    default = [
        {
            cpu: 1
            ...
        }
    ]
}
7

Using the for_each block is pretty new and there's not too much documentation. Some of the best info comes from their announcement blog post: https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each/

Also make sure to check out the Dynamic Blocks section of their documentation: https://www.terraform.io/docs/configuration/expressions.html#dynamic-blocks

From what your example looks like you need to have a set of values for each instance that is created so you'll have a map of maps:

Below is an example I created using Terraform 0.12.12:

variable "hostnames" {
    default = {
        "one" = {
            "name" = "one",
            "machine" = "n1-standard-1",
            "os" = "projects/coreos-cloud/global/images/coreos-stable-2247-5-0-v20191016",
            "zone" = "us-central1-a"
        },
        "two" = {
            "name" = "two",
            "machine" = "n1-standard-2",
            "os" = "projects/centos-cloud/global/images/centos-8-v20191018",
            "zone" = "us-central1-b"
        }
    }
}

resource "google_compute_instance" "default" {
    for_each = var.hostnames
    name         = each.value.name
    machine_type = each.value.machine
    zone         = each.value.zone

    boot_disk {
        initialize_params {
            image = each.value.os
        }
    }

    scratch_disk {
    }

    network_interface {
        network = "default"
    }
}

Terraform plan output:

Terraform will perform the following actions:

  # google_compute_instance.default["one"] will be created
  + resource "google_compute_instance" "default" {
      + can_ip_forward       = false
      + cpu_platform         = (known after apply)
      + deletion_protection  = false
      + guest_accelerator    = (known after apply)
      + id                   = (known after apply)
      + instance_id          = (known after apply)
      + label_fingerprint    = (known after apply)
      + machine_type         = "n1-standard-1"
      + metadata_fingerprint = (known after apply)
      + name                 = "one"
      + project              = (known after apply)
      + self_link            = (known after apply)
      + tags_fingerprint     = (known after apply)
      + zone                 = "us-central1-a"

      + boot_disk {
          + auto_delete                = true
          + device_name                = (known after apply)
          + disk_encryption_key_sha256 = (known after apply)
          + kms_key_self_link          = (known after apply)
          + mode                       = "READ_WRITE"
          + source                     = (known after apply)

          + initialize_params {
              + image  = "projects/coreos-cloud/global/images/coreos-stable-2247-5-0-v20191016"
              + labels = (known after apply)
              + size   = (known after apply)
              + type   = (known after apply)
            }
        }

      + network_interface {
          + address            = (known after apply)
          + name               = (known after apply)
          + network            = "default"
          + network_ip         = (known after apply)
          + subnetwork         = (known after apply)
          + subnetwork_project = (known after apply)
        }

      + scheduling {
          + automatic_restart   = (known after apply)
          + on_host_maintenance = (known after apply)
          + preemptible         = (known after apply)

          + node_affinities {
              + key      = (known after apply)
              + operator = (known after apply)
              + values   = (known after apply)
            }
        }

      + scratch_disk {
          + interface = "SCSI"
        }
    }

  # google_compute_instance.default["two"] will be created
  + resource "google_compute_instance" "default" {
      + can_ip_forward       = false
      + cpu_platform         = (known after apply)
      + deletion_protection  = false
      + guest_accelerator    = (known after apply)
      + id                   = (known after apply)
      + instance_id          = (known after apply)
      + label_fingerprint    = (known after apply)
      + machine_type         = "n1-standard-2"
      + metadata_fingerprint = (known after apply)
      + name                 = "two"
      + project              = (known after apply)
      + self_link            = (known after apply)
      + tags_fingerprint     = (known after apply)
      + zone                 = "us-central1-b"

      + boot_disk {
          + auto_delete                = true
          + device_name                = (known after apply)
          + disk_encryption_key_sha256 = (known after apply)
          + kms_key_self_link          = (known after apply)
          + mode                       = "READ_WRITE"
          + source                     = (known after apply)

          + initialize_params {
              + image  = "projects/centos-cloud/global/images/centos-8-v20191018"
              + labels = (known after apply)
              + size   = (known after apply)
              + type   = (known after apply)
            }
        }

      + network_interface {
          + address            = (known after apply)
          + name               = (known after apply)
          + network            = "default"
          + network_ip         = (known after apply)
          + subnetwork         = (known after apply)
          + subnetwork_project = (known after apply)
        }

      + scheduling {
          + automatic_restart   = (known after apply)
          + on_host_maintenance = (known after apply)
          + preemptible         = (known after apply)

          + node_affinities {
              + key      = (known after apply)
              + operator = (known after apply)
              + values   = (known after apply)
            }
        }

      + scratch_disk {
          + interface = "SCSI"
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.
1
  • So I need to convert list to map, as I undersand it change the index of list to key and it's gonna be map. Oct 29, 2019 at 10:26
2

Yes this is possible, you need to use the for expression in Terraform to achieve this though, the for loop converts the list of objects into a value in which Terraform can loop over using for_each, without the for expression, Terraform cannot loop over the list of objects because there is no key value for Terraform to reference.

Below is a is a simple example:

# variables.tf
variable "nsg_rules" {
  description = "list of maps consisting of nsg rules"
  type = list(object({
    access                       = string
    destination_address_prefixes = list(string)
    destination_port_ranges      = list(string)
    direction                    = string
    name                         = string
    priority                     = number
    protocol                     = string
    source_address_prefixes      = list(string)
    source_port_range            = string
  }))
  default = [
    {
      access                       = "Deny"
      destination_address_prefixes = ["10.10.1.0/24", "10.10.2.0/24"]
      destination_port_ranges      = ["80"]
      direction                    = "Inbound"
      name                         = "DenyHTTPInbound"
      priority                     = 100
      protocol                     = "*"
      source_address_prefixes      = ["10.0.0.0/24"]
      source_port_range            = "*"
    },
    {
      access                       = "Deny"
      destination_address_prefixes = ["10.10.10.0/24", "10.10.11.0/24"]
      destination_port_ranges      = ["22"]
      direction                    = "Inbound"
      name                         = "DenySSHInbound"
      priority                     = 200
      protocol                     = "*"
      source_address_prefixes      = ["10.0.0.0/24"]
      source_port_range            = "*"
    }
  ]
}

Use the for expression wrapped in curl brackets to convert the variable value, each maps key will be given the value of each maps name input, for example the first map would be given a key of "DenyHTTPInbound"

resource "azurerm_network_security_rule" "nsg_rules" {
  for_each = { for rule in var.nsg_rules : rule.name => rule }
 
  access                       = each.value.access
  destination_address_prefixes = each.value.destination_address_prefixes
  destination_port_ranges      = each.value.destination_port_ranges
  direction                    = each.value.direction
  name                         = each.value.name
  network_security_group_name  = azurerm_network_security_group.nsg.name
  priority                     = each.value.priority
  protocol                     = each.value.protocol
  resource_group_name          = azurerm_resource_group.rg.name
  source_address_prefixes      = each.value.source_address_prefixes
  source_port_range            = each.value.source_port_range
}

ref: https://jimferrari.com/2023/02/13/loop-through-list-of-maps-objects-with-terraform/

0

I took reference from the for_each example above and used below. This did not work for me, link below has details. Terraform for_each on custom registry

module "az"{
source="./modules/az"
vpc_id = module.vpc.vpc_id
for_each = toset(keys({for i,v in var.az_sub: i => v}))
availability_zone = var.az_sub[each.value]["az"]
public_cidr_block = var.az_sub[each.value]["public_cidr_block"]
private_cidr_block  =var.az_sub[each.value]["private_cidr_block"]
}

Error:module.az is object with 2 attributes If I replace for_each with actual values, the module is working perfectly.

0

This is a pretty confusing structure in terraform, but given:

variable services {
  type        = list(map(string))
  description = "services"
  default     = [
    {
      name          = "abc"
      target_port   = 9097
      health_port   = 3780
      health_code   = 200
      protocol      = "HTTP"
    },
    {
      name          = "def"
      target_port   = 8580
      health_port   = 3580
      health_code   = 200
      protocol      = "HTTP"
    },
    {
      name          = "ghi"
      target_port   = 80
      health_port   = 3680
      health_code   = 200
      protocol      = "HTTP"
    }
  ]
}

You iterate through resource as so:

resource "aws_lb_listener" "listeners" {
  for_each   = {
    for service in var.services: service.name => service
  }

  load_balancer_arn = aws_lb.internal.arn
  port              = each.value.target_port
  protocol          = each.value.protocol
  tags              = var.tags

You reference ANOTHER resource which uses a list of objects as so:

  default_action {
    type              = "forward"
    target_group_arn  = aws_lb_target_group.target_groups[each.value.name].id
  }

resource "aws_lb_target_group" "target_groups" {
  for_each   = {
    for service in var.services: service.name => service
  }

Notice since aws_lb_target_group is also using an array of maps, you must specify the map property when referencing from another resource, as shown above! That could trip people up.

And if you want to output the list of objects, you do as so:

output "alb_listener_arns" {
  value   = values(aws_lb_listener.listeners)[*].arn
}

output "target_group_ids" {
  value   = values(aws_lb_target_group.target_groups)[*].id
}
0

I've kept together some snippets for remembering how to do do various list comprehension and looping in TF. Some examples, e.g. if given a users = { "foo": { a = true, b = []}} in a tfvar file.

Make from map into a list:

output "reminderHowToListComprehensionEach" {
   value = [for e in var.users : e]
}

Keep as a map:

output "reminderHowToListComprehensionKeyValue" {
    value = {for k,v in var.users : k => v}
}

Another way

output "reminderHowToListComprehensionKey" {
    value = [for k,v in var.users : k]
}

And more useful probably, filtering the map:

output "reminderHowToListComprehensionValueWithConditional" {
    value = [for k,v in var.users : v if !v["xyz_bool_attribute"]]
}

See also:

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.