Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ISSUE] Issue with databricks_mount resource on Azure Government #4272

Open
connorbrown-db opened this issue Nov 26, 2024 · 2 comments · May be fixed by #4274
Open

[ISSUE] Issue with databricks_mount resource on Azure Government #4272

connorbrown-db opened this issue Nov 26, 2024 · 2 comments · May be fixed by #4274

Comments

@connorbrown-db
Copy link

Configuration

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~>4.0"
    }
    databricks = {
      source  = "databricks/databricks"
      version = "~>1.0"
    }
  }
}

provider "azurerm" {
  features {}
  environment = "usgovernment"
}

provider "databricks" {
  # ...
}

data "azurerm_client_config" "current" {}

resource "azurerm_storage_account" "example" {
  # ...
}

resource "azurerm_storage_container" "example" {
  # ...
}

resource "databricks_secret_scope" "example" {
  # ...
}

resource "databricks_mount" "this" {
  name = "${azurerm_storage_account.example.name}/${azurerm_storage_container.example.name}/"
  uri  = "abfss://${azurerm_storage_container.example.name}@${azurerm_storage_account.example.name}.dfs.core.usgovcloudapi.net/"
  extra_configs = {
    "fs.azure.account.auth.type"                          = "OAuth",
    "fs.azure.account.oauth.provider.type"                = "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
    "fs.azure.account.oauth2.client.id"                   = "{{secrets/${databricks_secret_scope.example.id}/clientid}}",
    "fs.azure.account.oauth2.client.secret"               = "{{secrets/${databricks_secret_scope.example.id}/password}}",
    "fs.azure.account.oauth2.client.endpoint"             = "https://login.microsoftonline.com/${data.azurerm_client_config.current.tenant_id}/oauth2/token",
    "fs.azure.createRemoteFileSystemDuringInitialization" = "false",
  }
}

Expected Behavior

The mount should be created as expected.

Actual Behavior

When using the abfs block to create a mount in Azure Government, the mount fails with the following error: Error: cannot create mount: Mount failed due to invalid mount source [abfss..//<container_name>@<sa_name>.dfs.core.windows.net].

This error is thrown because the URI is incorrect for storage accounts in Azure Government. Instead, the URI should end with core.usgovcloudapi.net. See here for the source of the problem.

Steps to Reproduce

  1. Create a storage account/container in Azure Government
  2. Use the abfs block in a databricks_mount resource to mount the container to a Databricks workspace
  3. Terraform apply

Terraform and provider versions

Terraform v1.9.6
on darwin_arm64
+ provider registry.terraform.io/databricks/databricks v1.59.0
+ provider registry.terraform.io/hashicorp/azurerm v4.11.0

Is it a regression?

No

Important Factoids

  • This should only be a problem on Azure Government, potentially other Azure Clouds that do not use the standard dfs.core.windows.net zone for storage accounts.

Would you like to implement a fix?

Not immediately.

Workaround

You can workaround this issue by using a combination of the uri and extra_configs parameters like the example below.

resource "databricks_mount" "example" {
  name       = "${azurerm_storage_account.example.name}/${azurerm_storage_container.example.name}/"
  uri        = "abfss://${azurerm_storage_container.example.name}@${azurerm_storage_account.example.name}.dfs.core.usgovcloudapi.net/"
  extra_configs = {
    "fs.azure.account.auth.type"                          = "OAuth",
    "fs.azure.account.oauth.provider.type"                = "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
    "fs.azure.account.oauth2.client.id"                   = "{{secrets/${databricks_secret_scope.example.id}/clientid}}",
    "fs.azure.account.oauth2.client.secret"               = "{{secrets/${databricks_secret_scope.example.id}/clientsecret}}",
    "fs.azure.account.oauth2.client.endpoint"             = "https://login.microsoftonline.com/${data.azuread_client_config.current.tenant_id}/oauth2/token",
    "fs.azure.createRemoteFileSystemDuringInitialization" = "false",
  }
}

This will make the provider use the correct URI.
@alexott
Copy link
Contributor

alexott commented Nov 27, 2024

Do you use only ADLS gen2 or also blob storage? Do you have a doc that says how different prefixes should look for different cloud versions?

@connorbrown-db
Copy link
Author

I am only using ADLS Gen2. Below is JSON of the Azure Clouds and their corresponding storage API base URIs.

{
  "name": "AzureCloud",
  "cloudEndpoint": "https://management.azure.com/",
  "storageEndpoint": "core.windows.net"
}
{
  "name": "AzureChinaCloud",
  "cloudEndpoint": "https://management.chinacloudapi.cn",
  "storageEndpoint": "core.chinacloudapi.cn"
}
{
  "name": "AzureUSGovernment",
  "cloudEndpoint": "https://management.usgovcloudapi.net/",
  "storageEndpoint": "core.usgovcloudapi.net"
}
{
  "name": "AzureGermanCloud",
  "cloudEndpoint": "https://management.microsoftazure.de",
  "storageEndpoint": "core.cloudapi.de"
}

You can reproduce this JSON with the Azure CLI: az cloud list | jq '.[] | {name, cloudEndpoint: .endpoints.resourceManager, storageEndpoint: .suffixes.storageEndpoint}'.

alexott added a commit that referenced this issue Nov 28, 2024
@alexott
[Fix] Use correct domain for Azure Gov and China
84eb799 
Resolves #4272
@alexott alexott linked a pull request Nov 28, 2024 that will close this issue
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Fix] Use correct domain for Azure Gov and China databricks/terraform-provider-databricks
2 participants
@alexott @connorbrown-db