Removing cloud-init from VM or update env file causes VM replacement

[perryflynn]

Describe the bug

After the VM is bootstrapped, I would like to remove the cloud-init drive from it. Another scenario is to update the user_data_file, which also cases a force replacement of the VM.

Both because user_data_file_id is touched.

To Reproduce

Steps to reproduce the behavior:

1. Create a resource proxmox_virtual_environment_file and proxmox_virtual_environment_vm which references the env file in initialization.user_data_file_id
2. Remove initialization {} or update proxmox_virtual_environment_file content
3. See VM force replacement

Please also provide a minimal Terraform configuration that reproduces the issue.

terraform {
  required_version = ">= 1.0"
  required_providers {
    proxmox = {
      source = "bpg/proxmox"
      version = "0.66.3"
    }
  }
}

provider "proxmox" {
  endpoint = "https://benny.mm.example.com:8006"
  username = "root@pam"
  #password = "" # use env PROXMOX_VE_PASSWORD
}

resource "proxmox_virtual_environment_file" "cloud_config" {
  content_type = "snippets"
  datastore_id = var.pve_snippetstore
  node_name    = var.pve_node

  source_raw {
    data = <<-EOF
    #cloud-config
    packages: []

    keyboard:
      layout: de

    locale: en_US.UTF-8

    timezone: Europe/Berlin
    users: []
    groups: []
    EOF

    file_name = "cloud-config.yaml"
  }
}

resource "proxmox_virtual_environment_vm" "debidesk" {
    vm_id = 104
    name = "debidesk.example.com"
    description = "Remote Desktop based on Debian"
    tags = [ "desktop" ]
    node_name = var.pve_node

    on_boot = false
    boot_order = [ "scsi0", "ide2" ]

    initialization {
      interface = "ide0"
      datastore_id = var.pve_blockstore

      ip_config {
        ipv4 {
          address = "dhcp"
        }
      }

      user_data_file_id = proxmox_virtual_environment_file.cloud_config.id
    }

    agent {
        enabled = true
        timeout = "10s"
    }

    startup {
      order = 200
    }

    operating_system {
      type = "l26"
    }

    cpu {
      type = "x86-64-v3"
      sockets = 1
      cores = 4
    }

    memory {
      dedicated = 2048
    }

    vga {
      type = "qxl"
      memory = 32
    }

    network_device {
      model = "virtio"
      bridge = "vmbr42"
      firewall = true
      enabled = true
    }

    scsi_hardware = "virtio-scsi-single"

    disk {
      interface = "scsi0"
      aio = "io_uring"
      backup = true
      cache = "none"
      discard = "on"
      ssd = true
      iothread = true
      size = 64
      datastore_id = var.pve_blockstore
      file_format = "raw"
    }
}

and the output of terraform|tofu apply.

proxmox_virtual_environment_file.cloud_config: Refreshing state... [id=pve-manual-beta:snippets/cloud-config.yaml]
proxmox_virtual_environment_vm.debidesk: Refreshing state... [id=104]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # proxmox_virtual_environment_file.cloud_config must be replaced
-/+ resource "proxmox_virtual_environment_file" "cloud_config" {
      + file_modification_date = (known after apply)
      ~ file_name              = "cloud-config.yaml" -> (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      ~ id                     = "pve-manual-beta:snippets/cloud-config.yaml" -> (known after apply)
        # (5 unchanged attributes hidden)

      ~ source_raw {
          ~ data      = <<-EOT # forces replacement
                #cloud-config
                packages: []
                
              + # foo
            EOT
            # (2 unchanged attributes hidden)
        }
    }

  # proxmox_virtual_environment_vm.debidesk must be replaced
-/+ resource "proxmox_virtual_environment_vm" "debidesk" {
      ~ id                      = "104" -> (known after apply)
      ~ ipv4_addresses          = [
          - [
              - "127.0.0.1",
            ],
          - [
              - "192.168.42.65",
            ],
        ] -> (known after apply)
      ~ ipv6_addresses          = [
          - [
              - "::1",
            ],
          - [
              - "fe80::be24:11ff:fe14:e3c8",
            ],
        ] -> (known after apply)
      ~ mac_addresses           = [
          - "00:00:00:00:00:00",
          - "BC:24:11:14:E3:C8",
        ] -> (known after apply)
        name                    = "debidesk"
      ~ network_interface_names = [
          - "lo",
          - "ens18",
        ] -> (known after apply)
        tags                    = [
            "desktop",
        ]
        # (26 unchanged attributes hidden)

      - cdrom {
          - enabled   = true -> null
          - file_id   = "pve-manual-beta:iso/perrys-bootstrapper-2024.11.13-x86_64.iso" -> null
          - interface = "ide2" -> null
        }

      ~ cpu {
          - flags        = [] -> null
            # (9 unchanged attributes hidden)
        }

      ~ disk {
          ~ path_in_datastore = "vm-104-disk-0" -> (known after apply)
            # (13 unchanged attributes hidden)
        }
      ~ disk {
          ~ path_in_datastore = "vm-104-disk-1" -> (known after apply)
            # (13 unchanged attributes hidden)
        }

      ~ initialization {
          ~ upgrade              = false -> (known after apply)
          ~ user_data_file_id    = "pve-manual-beta:snippets/cloud-config.yaml" -> (known after apply) # forces replacement
            # (6 unchanged attributes hidden)

            # (1 unchanged block hidden)
        }

      ~ network_device {
          - disconnected = false -> null
          ~ mac_address  = "BC:24:11:14:E3:C8" -> (known after apply)
            # (9 unchanged attributes hidden)
        }

        # (5 unchanged blocks hidden)
    }

Plan: 2 to add, 0 to change, 2 to destroy.

Expected behavior

cloud-init device is removed without replacing VM or cloud-init config is updated without replacing VM.

Additional context
Add any other context about the problem here.

• Single or clustered Proxmox: clustered, 3 nodes
• Proxmox version: 8.2.7
• Provider version (ideally it should be the latest version): 0.66.3
• Terraform/OpenTofu version: terraform v1.9.8
• OS (where you run Terraform/OpenTofu from): Ubuntu Jammy

[CRASH-Tech]

+1

[elias314]

It's standard among Terraform providers to recreate a VM or instance when user data changes. To work around this, use ignore_changes in a lifecycle block to tell Terraform not to update your infrastructure when it sees that the cloud init has changed.

As for removing the cloud init drive completely, I'm less clear on the right approach here. On the one hand it's a user data change which should recreate the VM, but on the other hand, it's a drive and lots of drive operations can happen without recreating the VM. But given that the initialization block is more than just a drive, I'm of the opinion that removing that block completely signals a change to the initial state of that VM and recreating the VM is appropriate and consistent with the behavior of other Terraform providers.

If wrapping the whole initialization block in an ignore_changes block isn't sufficient and you really want to remove the cloud init drive, you could try using Terraform's removed block to remove it from state, and then remove the drive from Proxmox via some other means.