terraform destroy // content deadline exceeded (Client.Timeout exceeded while awaiting hearders)

[dgst77]

I am able to create multiple tenants using Terraform f50s .However when I try to destroy I get the error below

error: Delete :https://x.x.x.x:8888/restconf/data/f5-tenants:tenants/tenant=test" context deadline exceeded(Clients.Timeout exceeded while awaiting headers)

Terraform Version

1.9.5

Affected Resource(s)

f50s_tenant

[dgst77]

debug logs // during terraform destroy
#######################################################
f5os_tenant.tenants["qicb-is-ltm"]: Still destroying... [id=qicb-is-ltm, 50s elapsed]
2024-10-01T13:49:02.182-0600 [DEBUG] provider.terraform-provider-f5os_v1.3.7: Called provider defined Resource Delete: @module=sdk.framework tf_provider_addr=registry.terraform.io/f5networks/f5os tf_req_id=a60fbe4d-8aa9-c995-d667-93aadbcc092a tf_resource_type=f5os_tenant tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_deleteresource.go:88 timestamp=2024-10-01T13:49:02.181-0600
2024-10-01T13:49:02.182-0600 [ERROR] provider.terraform-provider-f5os_v1.3.7: Response contains error diagnostic: diagnostic_summary="Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qicb-tier1-ltm\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" tf_proto_version=6.3 tf_rpc=ApplyResourceChange @module=sdk.proto diagnostic_detail="" diagnostic_severity=ERROR tf_provider_addr=registry.terraform.io/f5networks/f5os tf_req_id=a60fbe4d-8aa9-c995-d667-93aadbcc092a tf_resource_type=f5os_tenant @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:55 timestamp=2024-10-01T13:49:02.182-0600
f5os_tenant.tenants["qicb-tier1-ltm"]: Still destroying... [id=qicb-tier1-ltm, 1m0s elapsed]
2024-10-01T13:49:02.194-0600 [ERROR] vertex "f5os_tenant.tenants["qicb-tier1-ltm"] (destroy)" error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qicb-tier1-ltm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2024-10-01T13:49:11.717-0600 [DEBUG] provider.terraform-provider-f5os_v1.3.7: Called provider defined Resource Delete: tf_req_id=cfc21d61-cf09-21bc-2eaf-99b89f2fca8b @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_deleteresource.go:88 @module=sdk.framework tf_provider_addr=registry.terraform.io/f5networks/f5os tf_resource_type=f5os_tenant tf_rpc=ApplyResourceChange timestamp=2024-10-01T13:49:11.717-0600
2024-10-01T13:49:11.717-0600 [ERROR] provider.terraform-provider-f5os_v1.3.7: Response contains error diagnostic: @module=sdk.proto tf_req_id=cfc21d61-cf09-21bc-2eaf-99b89f2fca8b @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:55 diagnostic_detail="" diagnostic_severity=ERROR diagnostic_summary="Delete "https://10.145.80.10:8888/restconf/data/f5-tenants:tenants/tenant=qicb-is-ltm\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" tf_proto_version=6.3 tf_provider_addr=registry.terraform.io/f5networks/f5os tf_resource_type=f5os_tenant tf_rpc=ApplyResourceChange timestamp=2024-10-01T13:49:11.717-0600
2024-10-01T13:49:11.729-0600 [DEBUG] State storage *statemgr.Filesystem declined to persist a state snapshot
2024-10-01T13:49:11.729-0600 [ERROR] vertex "f5os_tenant.tenants["qicb-is-ltm"] (destroy)" error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qicb-is-ltm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
╷
│ Error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qicb-is-ltm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
│
│ Error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qica-tier3-ltm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
│
│ Error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qica-tier3-gtm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
│
│ Error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qicb-is-gtm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
│
│ Error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qicb-tier2-ltm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

│ Error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qicb-tier1-ltm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
│
│ Error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qica-is-gtm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
│
│ Error: Delete "https://localhost:8888/restconf/data/f5-tenants:tenants/tenant=qica-tier2-ltm": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
│
│
╵
2024-10-01T13:49:11.741-0600 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-10-01T13:49:11.743-0600 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/f5networks/f5os/1.3.7/darwin_arm64/terraform-provider-f5os_v1.3.7 pid=9691
2024-10-01T13:49:11.743-0600 [DEBUG] provider: plugin exited

[dgst77]

test.tf

terraform {
required_version = ">= 1.0"

required_providers {
f5os = {
source = "f5networks/f5os"
version = "1.3.7"
}
vault = {
source = "hashicorp/vault"
version = "4.3.0"
}
}
}

provider "f5os" {
username = var.username
password = var.password
host = "https://x.x.x.x"
teem_disable = true
}

Definition of List of TFVARS file for each tenants

variable "tenant_configs" {
type = list(string)
default = [
"xxxa-tier1-ltm.tfvars",
"xxxa-tier2-ltm.tfvars",
"xxxa-tier3-ltm.tfvars"

Read and parse the tenant config file

locals {
tenant_data = {
for config_file in var.tenant_configs :
trimspace(replace(config_file, "/^(.*)\.tfvars$/", "$1")) => yamldecode(file(config_file))
}
}

Define all VLANS ON RSERIES HOST

variable "vlans" {
type = list(object({
name = string
vlan_id = number
}))
default = [
{
name = "123"
vlan_id = 123
},
{
name = "124"
vlan_id = 124
}
]

create vlans on rseries host

resource "f5os_vlan" "vlan" {
for_each = {
for vlan in var.vlans : vlan.name => vlan
}
name = each.value.name
vlan_id = each.value.vlan_id

}

resource "f5os_lag" "f5-lagg" {
name = "f5-aci"
interval = "FAST"
mode = "ACTIVE"
members = ["1.0", "2.0"]
trunk_vlans = [
532,
542,
556
]
depends_on = [ f5os_vlan.vlan ]

Create F5OS Tenant

resource "f5os_tenant" "tenants" {
for_each = local.tenant_data

name = each.key
image_name = try(each.value.image_name)
mgmt_ip = try(each.value.mgmt_ip)
mgmt_gateway = try(each.value.mgmt_gateway)
mgmt_prefix = try(each.value.mgmt_prefix)
type = try(each.value.type)
cpu_cores = try(each.value.cpu_cores)
vlans = try(each.value.Vlans, [])
running_state = try(each.value.running_state)
virtual_disk_size = try(each.value.virtual_disk_size)

depends_on = [ f5os_vlan.vlan, f5os_lag.f5-aci ]

}

[dgst77]

forgot to mention this - terraform destroy , deletes all the tenants but does not delete the vlans and lag

[pgouband]

Hi @dgst77,

Which platform (rSeries or Velos) and F5OS version are tested?

[dgst77]

Retries  Sent from Yahoo Mail for iPhone On Thursday, October 3, 2024, 3:25 AM, Patrick ***@***.***> wrote: Hi @dgst77, Which platform (rSeries or Velos) and F5OS version are tested? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[dgst77]

Rseries  Sent from Yahoo Mail for iPhone On Thursday, October 3, 2024, 5:56 AM, Kofi Fefey ***@***.***> wrote: Retries  Sent from Yahoo Mail for iPhone On Thursday, October 3, 2024, 3:25 AM, Patrick ***@***.***> wrote: Hi @dgst77, Which platform (rSeries or Velos) and F5OS version are tested? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[dgst77]

One other detail …. Terraform destroy delete all the tenants then run into the error state above regarding client timeout and do not delete the vlans and tag / trunk

[dgst77]

Rseries

[pgouband]

Hi @dgst77,

Can you share F5OS version?

[dgst77]

F5OS version is 1.7.0-871 and Rseries platform is 12800

[pgouband]

Hi,

Thanks for reporting. Added to the backlog and internal tracking ID for this request is: INFRAANO-1688.

[prateekramani]

Hi @dgst77 ,

I was able to reproduce the "context deadline exeeded" issue , for which I have added a retry mechanism. This issue is mainly due to a poor HTTP connection.
But I was not able to reproduce the second issue. I am able to delete the Tenants, Vlans and Lag, all in one go.
I used rSeries 1.8.0-16036 , and Terraform version: 1.5.7

Thanks

[dgst77]

awesome . Thank you very much