Problem with obtain ipv4 address (ipv4_addresses is blank list) when clonning a VM

[anshibanov]

TLDR: agent is enabled and work, and in proxmox-webui i see VM's ip

I can't figure out how to get the IP address of the newly created virtual machine (the VM obtains it via DHCP).
The problem is ipv4_addresses always blank

Below is my Terraform configuration:

resource "proxmox_virtual_environment_vm" "test-ansible-docker2" {
   name      = "test-ansible-docker2"
   node_name = "hv"

   cpu {
     cores = 2
     type  = "host"
   }

   memory {
     dedicated = 1024
   }

   disk {
       interface = "scsi0"
       size      = 15
   }

   clone {
       vm_id = 9002  # debian
       #vm_id = 9001 # ubuntu
   }

   agent {
       enabled = true
   }

   initialization {
     ip_config {
       ipv4 {
         address = "dhcp"
       }
     }
   }

   provisioner "local-exec" {
     when    = create
     command = "sleep 15s"
   }

   reboot = true

   lifecycle {
     ignore_changes = [
       ipv6_addresses,
       network_interface_names,
       network_device,
       vga
     ]
   }
}

output "ipv4_addresses" {
  value = proxmox_virtual_environment_vm.test-ansible-docker2.ipv4_addresses
}

Here is the output:

Initializing the backend...
Successfully configured the backend "http"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Finding latest version of bpg/proxmox...
- Installing bpg/proxmox v0.69.1...
- Installed bpg/proxmox v0.69.1 (self-signed, key ID F0582AD6AE97C188)

$ terraform apply -input=false "planfile"
proxmox_virtual_environment_vm.test-ansible-docker2: Creating...
proxmox_virtual_environment_vm.test-ansible-docker2: Still creating... [10s elapsed]
proxmox_virtual_environment_vm.test-ansible-docker2: Still creating... [20s elapsed]
proxmox_virtual_environment_vm.test-ansible-docker2: Still creating... [30s elapsed]
proxmox_virtual_environment_vm.test-ansible-docker2: Provisioning with 'local-exec'...
proxmox_virtual_environment_vm.test-ansible-docker2 (local-exec): Executing: ["/bin/sh" "-c" "sleep 15s"]
proxmox_virtual_environment_vm.test-ansible-docker2: Still creating... [40s elapsed]
proxmox_virtual_environment_vm.test-ansible-docker2: Creation complete after 48s [id=104]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:
ipv4_addresses = tolist([])

However, in the Proxmox web interface, the IP address is displayed.

[bpg]

Hey @anshibanov! 👋🏼

Since you're making a clone, there could be some additional complexities. The cloning logic is quite messy and needs major attention.

I noticed that you specifically ignored network_interface_names and network_device —- was there a particular reason for that?

I suspect that ignoring the network device might be preventing the provider from correctly retrieving the network configuration when reading back the VM state.

[anshibanov]

Hi @bpg ! And thank you so much for your response. It seems the ignore section was some kind of old artifact in my configuration. I can easily remove it for debugging purposes. Here is what the updated Terraform configuration looks like:

resource "proxmox_virtual_environment_vm" "test-ansible-docker2" {
   name = "test-ansible-docker2"
   node_name = "hv"
   
   cpu {
     cores = 2
     type = "host"
   }

   memory {
    dedicated = 1024
   }

   disk {
       interface = "scsi0"
       size      = 15
   }

   clone {
       vm_id = 9002  # debian
       #vm_id = 9001 # ubuntu
   }

   agent {
       enabled = true
   }
   
  initialization {
    ip_config {
      ipv4 {
        address = "dhcp"
      }
    }
  }

  provisioner "local-exec" {
    when    = create
    command = "sleep 15s"
  }

  reboot = true
}

output "ipv4_addresses" {
  value = proxmox_virtual_environment_vm.test-ansible-docker2.ipv4_addresses
}

The result:

Initializing the backend...
Successfully configured the backend "http"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Finding latest version of bpg/proxmox...
- Installing bpg/proxmox v0.69.1...
- Installed bpg/proxmox v0.69.1 (self-signed, key ID F0582AD6AE97C188)
Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ terraform apply -input=false "planfile"
proxmox_virtual_environment_vm.test-ansible-docker2: Creating...
proxmox_virtual_environment_vm.test-ansible-docker2: Still creating... [10s elapsed]
proxmox_virtual_environment_vm.test-ansible-docker2: Still creating... [20s elapsed]
proxmox_virtual_environment_vm.test-ansible-docker2: Still creating... [30s elapsed]
proxmox_virtual_environment_vm.test-ansible-docker2: Provisioning with 'local-exec'...
proxmox_virtual_environment_vm.test-ansible-docker2 (local-exec): Executing: ["/bin/sh" "-c" "sleep 15s"]
proxmox_virtual_environment_vm.test-ansible-docker2: Still creating... [40s elapsed]
proxmox_virtual_environment_vm.test-ansible-docker2: Still creating... [50s elapsed]
proxmox_virtual_environment_vm.test-ansible-docker2: Creation complete after 50s [id=104]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
ipv4_addresses = tolist([])

Unfortunately, the ipv4_addresses output is still an empty list. :(

[bpg]

Hm.., interesting. Let me take a closer look then 🧐

[bpg]

I couldn't reproduce this on my PVE 8.3.2 setup 🤷🏼

I used the template from this guide (we should consider adding it to the codebase), and the IP appears correctly in the output:

❯ tofu apply -auto-approve
data.local_file.ssh_public_key: Reading...
data.local_file.ssh_public_key: Read complete after 0s [id=84cdc3a1d1eaa8e821cd6dd4287f04ae996b8309]

OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

OpenTofu will perform the following actions:

  # proxmox_virtual_environment_download_file.ubuntu_cloud_image will be created
  + resource "proxmox_virtual_environment_download_file" "ubuntu_cloud_image" {
      + content_type        = "iso"
      + datastore_id        = "local"
      + file_name           = (known after apply)
      + id                  = (known after apply)
      + node_name           = "pve"
      + overwrite           = true
      + overwrite_unmanaged = false
      + size                = (known after apply)
      + upload_timeout      = 600
      + url                 = "https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img"
      + verify              = true
    }

  # proxmox_virtual_environment_file.user_data_cloud_config will be created
  + resource "proxmox_virtual_environment_file" "user_data_cloud_config" {
      + content_type           = "snippets"
      + datastore_id           = "local"
      + file_modification_date = (known after apply)
      + file_name              = (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      + id                     = (known after apply)
      + node_name              = "pve"
      + overwrite              = true
      + timeout_upload         = 1800

      + source_raw {
          + data      = <<-EOT
                #cloud-config
                hostname: test-ubuntu
                users:
                  - default
                  - name: ubuntu
                    groups:
                      - sudo
                    shell: /bin/bash
                    ssh_authorized_keys:
                      - ssh-rsa SKIPPED
                    sudo: ALL=(ALL) NOPASSWD:ALL
                runcmd:
                    - apt update
                    - apt install -y qemu-guest-agent net-tools
                    - timedatectl set-timezone America/Toronto
                    - systemctl enable qemu-guest-agent
                    - systemctl start qemu-guest-agent
                    - echo "done" > /tmp/cloud-config.done
            EOT
          + file_name = "user-data-cloud-config.yaml"
          + resize    = 0
        }
    }

  # proxmox_virtual_environment_vm.ubuntu_clone will be created
  + resource "proxmox_virtual_environment_vm" "ubuntu_clone" {
      + acpi                    = true
      + bios                    = "seabios"
      + id                      = (known after apply)
      + ipv4_addresses          = (known after apply)
      + ipv6_addresses          = (known after apply)
      + keyboard_layout         = "en-us"
      + mac_addresses           = (known after apply)
      + migrate                 = false
      + name                    = "ubuntu-clone"
      + network_interface_names = (known after apply)
      + node_name               = "pve"
      + on_boot                 = true
      + protection              = false
      + reboot                  = false
      + scsi_hardware           = "virtio-scsi-pci"
      + started                 = true
      + stop_on_destroy         = false
      + tablet_device           = true
      + template                = false
      + timeout_clone           = 1800
      + timeout_create          = 1800
      + timeout_migrate         = 1800
      + timeout_move_disk       = 1800
      + timeout_reboot          = 1800
      + timeout_shutdown_vm     = 1800
      + timeout_start_vm        = 1800
      + timeout_stop_vm         = 300
      + vm_id                   = (known after apply)

      + agent {
          + enabled = true
          + timeout = "15m"
          + trim    = false
          + type    = "virtio"
        }

      + clone {
          + full    = true
          + retries = 1
          + vm_id   = (known after apply)
        }

      + initialization {
          + datastore_id = "local-lvm"
          + upgrade      = (known after apply)

          + dns {
              + servers = [
                  + "1.1.1.1",
                ]
            }

          + ip_config {
              + ipv4 {
                  + address = "dhcp"
                }
            }
        }

      + memory {
          + dedicated      = 768
          + floating       = 0
          + keep_hugepages = false
          + shared         = 0
        }
    }

  # proxmox_virtual_environment_vm.ubuntu_template will be created
  + resource "proxmox_virtual_environment_vm" "ubuntu_template" {
      + acpi                    = true
      + bios                    = "ovmf"
      + description             = "Managed by Terraform"
      + id                      = (known after apply)
      + ipv4_addresses          = (known after apply)
      + ipv6_addresses          = (known after apply)
      + keyboard_layout         = "en-us"
      + mac_addresses           = (known after apply)
      + machine                 = "q35"
      + migrate                 = false
      + name                    = "ubuntu-template"
      + network_interface_names = (known after apply)
      + node_name               = "pve"
      + on_boot                 = true
      + protection              = false
      + reboot                  = false
      + scsi_hardware           = "virtio-scsi-pci"
      + stop_on_destroy         = false
      + tablet_device           = true
      + template                = true
      + timeout_clone           = 1800
      + timeout_create          = 1800
      + timeout_migrate         = 1800
      + timeout_move_disk       = 1800
      + timeout_reboot          = 1800
      + timeout_shutdown_vm     = 1800
      + timeout_start_vm        = 1800
      + timeout_stop_vm         = 300
      + vm_id                   = (known after apply)

      + agent {
          + enabled = true
          + timeout = "15m"
          + trim    = false
          + type    = "virtio"
        }

      + cpu {
          + cores      = 2
          + hotplugged = 0
          + limit      = 0
          + numa       = false
          + sockets    = 1
          + type       = "qemu64"
          + units      = 1024
        }

      + disk {
          + aio               = "io_uring"
          + backup            = true
          + cache             = "none"
          + datastore_id      = "local-lvm"
          + discard           = "on"
          + file_format       = (known after apply)
          + file_id           = (known after apply)
          + interface         = "virtio0"
          + iothread          = true
          + path_in_datastore = (known after apply)
          + replicate         = true
          + size              = 20
          + ssd               = false
        }

      + efi_disk {
          + datastore_id      = "local"
          + file_format       = "raw"
          + pre_enrolled_keys = false
          + type              = "4m"
        }

      + initialization {
          + datastore_id      = "local-lvm"
          + upgrade           = (known after apply)
          + user_data_file_id = (known after apply)

          + ip_config {
              + ipv4 {
                  + address = "dhcp"
                }
            }
        }

      + memory {
          + dedicated      = 2048
          + floating       = 0
          + keep_hugepages = false
          + shared         = 0
        }

      + network_device {
          + bridge      = "vmbr0"
          + enabled     = true
          + firewall    = false
          + mac_address = (known after apply)
          + model       = "virtio"
          + mtu         = 0
          + queues      = 0
          + rate_limit  = 0
          + vlan_id     = 0
        }
    }

Plan: 4 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + vm_ipv4_address = (known after apply)
proxmox_virtual_environment_file.user_data_cloud_config: Creating...
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Creating...
proxmox_virtual_environment_file.user_data_cloud_config: Creation complete after 0s [id=local:snippets/user-data-cloud-config.yaml]
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Still creating... [10s elapsed]
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Still creating... [20s elapsed]
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Still creating... [30s elapsed]
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Still creating... [40s elapsed]
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Creation complete after 41s [id=local:iso/jammy-server-cloudimg-amd64.img]
proxmox_virtual_environment_vm.ubuntu_template: Creating...
proxmox_virtual_environment_vm.ubuntu_template: Still creating... [10s elapsed]
proxmox_virtual_environment_vm.ubuntu_template: Creation complete after 15s [id=100]
proxmox_virtual_environment_vm.ubuntu_clone: Creating...
proxmox_virtual_environment_vm.ubuntu_clone: Still creating... [10s elapsed]
proxmox_virtual_environment_vm.ubuntu_clone: Still creating... [20s elapsed]
proxmox_virtual_environment_vm.ubuntu_clone: Still creating... [30s elapsed]
proxmox_virtual_environment_vm.ubuntu_clone: Still creating... [40s elapsed]
proxmox_virtual_environment_vm.ubuntu_clone: Creation complete after 49s [id=101]

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

vm_ipv4_address = "172.17.223.214"

Not sure what else it could be. Try without reboot = true?
Try different ubuntu version? There are few reports that Ubuntu 24.04 has some issues with cloud-init or container initialization. Not sure which one you're using.

[bpg]

#1701 seems related

[anshibanov]

@bpg Thx! Without reboot = true all works!