s3_bucket module always fails in eu-north-1

[vonschultz]

Summary

With the introduction of Transfer Acceleration support the s3_bucket module always fail in some regions, including eu-north-1. Transfer Acceleration is only supported in regions listed here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/transfer-acceleration.html. For example, I can use s3_bucket in eu-west-1 without any errors.

This is closely related to #2180, but I've verified that the fix for that issue, #2202, does not solve my problem, possibly because I'm getting MethodNotAllowed instead of UnsupportedArgument.

Issue Type

Bug Report

Component Name

s3_bucket

Ansible Version

$ ansible --version
ansible [core 2.17.3]
  config file = /home/von/.ansible.cfg
  configured module search path = ['/home/von/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages/ansible
  ansible collection location = /home/von/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/von/src/embedl/ci/provisioning/.tox/ansible/bin/ansible
  python version = 3.10.14 (main, Apr  6 2024, 18:45:05) [GCC 9.4.0] (/home/von/src/embedl/ci/provisioning/.tox/ansible/bin/python)
  jinja version = 3.1.4
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

# /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               8.1.0  
ansible.netcommon                        6.1.3  
ansible.posix                            1.5.4  
ansible.utils                            4.1.0  
ansible.windows                          2.4.0  
arista.eos                               9.0.0  
awx.awx                                  24.6.1 
azure.azcollection                       2.6.0  
check_point.mgmt                         5.2.3  
chocolatey.chocolatey                    1.5.1  
cisco.aci                                2.10.1 
cisco.asa                                5.0.1  
cisco.dnac                               6.17.1 
cisco.intersight                         2.0.10 
cisco.ios                                8.0.0  
cisco.iosxr                              9.0.0  
cisco.ise                                2.9.3  
cisco.meraki                             2.18.1 
cisco.mso                                2.9.0  
cisco.nxos                               8.1.0  
cisco.ucs                                1.10.0 
cloud.common                             3.0.0  
cloudscale_ch.cloud                      2.4.0  
community.aws                            8.0.0  
community.ciscosmb                       1.0.9  
community.crypto                         2.21.1 
community.digitalocean                   1.26.0 
community.dns                            3.0.3  
community.docker                         3.12.1 
community.general                        9.3.0  
community.grafana                        1.9.1  
community.hashi_vault                    6.2.0  
community.hrobot                         2.0.1  
community.library_inventory_filtering_v1 1.0.1  
community.libvirt                        1.3.0  
community.mongodb                        1.7.6  
community.mysql                          3.9.0  
community.network                        5.0.3  
community.okd                            3.0.1  
community.postgresql                     3.4.1  
community.proxysql                       1.6.0  
community.rabbitmq                       1.3.0  
community.routeros                       2.18.0 
community.sap_libs                       1.4.2  
community.sops                           1.8.2  
community.vmware                         4.5.0  
community.windows                        2.2.0  
community.zabbix                         2.5.1  
containers.podman                        1.15.4 
cyberark.conjur                          1.3.0  
cyberark.pas                             1.0.27 
dellemc.enterprise_sonic                 2.4.0  
dellemc.openmanage                       9.5.0  
dellemc.powerflex                        2.5.0  
dellemc.unity                            2.0.0  
f5networks.f5_modules                    1.30.1 
fortinet.fortimanager                    2.6.0  
fortinet.fortios                         2.3.7  
frr.frr                                  2.0.2  
google.cloud                             1.3.0  
grafana.grafana                          5.4.0  
hetzner.hcloud                           3.1.1  
ibm.qradar                               3.0.0  
ibm.spectrum_virtualize                  2.0.0  
ibm.storage_virtualize                   2.4.1  
ieisystem.inmanage                       2.0.0  
infinidat.infinibox                      1.4.5  
infoblox.nios_modules                    1.6.1  
inspur.ispim                             2.2.3  
inspur.sm                                2.3.0  
junipernetworks.junos                    8.0.0  
kaytus.ksmanage                          1.2.2  
kubernetes.core                          3.2.0  
kubevirt.core                            1.5.0  
lowlydba.sqlserver                       2.3.3  
lvrfrc87.git_acp                         2.2.0  
microsoft.ad                             1.6.0  
netapp.cloudmanager                      21.22.1
netapp.ontap                             22.12.0
netapp.storagegrid                       21.12.0
netapp_eseries.santricity                1.4.0  
netbox.netbox                            3.19.1 
ngine_io.cloudstack                      2.3.0  
ngine_io.exoscale                        1.1.0  
openstack.cloud                          2.2.0  
openvswitch.openvswitch                  2.1.1  
ovirt.ovirt                              3.2.0  
purestorage.flasharray                   1.30.2 
purestorage.flashblade                   1.18.0 
sensu.sensu_go                           1.14.0 
splunk.es                                3.0.0  
t_systems_mms.icinga_director            2.0.1  
telekom_mms.icinga_director              2.1.2  
theforeman.foreman                       4.1.0  
vmware.vmware                            1.4.0  
vmware.vmware_rest                       3.0.1  
vultr.cloud                              1.13.0 
vyos.vyos                                4.1.0  
wti.remote                               1.0.5  

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.35.8
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.35.8
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
CONFIG_FILE() = /home/von/.ansible.cfg
DEFAULT_HOST_LIST(env: ANSIBLE_INVENTORY) = ['/home/von/src/embedl/ci/provisioning/inventories/production']
DEFAULT_LOAD_CALLBACK_PLUGINS(/home/von/.ansible.cfg) = True
DEFAULT_STDOUT_CALLBACK(/home/von/.ansible.cfg) = yaml
DEFAULT_VAULT_IDENTITY_LIST(env: ANSIBLE_VAULT_IDENTITY_LIST) = ['prod@/home/von/.config/ansible-prod-vault.txt']
EDITOR(env: EDITOR) = emacsclient -t

OS / Environment

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.6 LTS
Release:	20.04
Codename:	focal

Steps to Reproduce

- amazon.aws.s3_bucket:
    name: ansibleissuedebugbucket
    region: eu-north-1
    state: present

Expected Results

Bucket is created, or no changes are made if the bucket already exists.

Actual Results

fatal: [s3.amazonaws.com]: FAILED! => changed=false 
  boto3_version: 1.35.8
  botocore_version: 1.35.8
  error:
    code: MethodNotAllowed
    message: The specified method is not allowed against this resource.
    method: GET
    resource_type: accelerate
  msg: 'Failed to fetch bucket transfer acceleration state: An error occurred (MethodNotAllowed) when calling the GetBucketAccelerateConfiguration operation: The specified method is not allowed against this resource.'
  response_metadata:
    host_id: qjOs6WQzYiENa7r0LypeOzfdSNCiA2c0WyB2+RBjfoTnim74BGo9wORVEYMwemfUIzLbxOx4Vw8=
    http_headers:
      content-type: application/xml
      date: Thu, 29 Aug 2024 14:42:45 GMT
      server: AmazonS3
      transfer-encoding: chunked
      x-amz-id-2: qjOs6WQzYiENa7r0LypeOzfdSNCiA2c0WyB2+RBjfoTnim74BGo9wORVEYMwemfUIzLbxOx4Vw8=
      x-amz-request-id: SR0RTXK1WMF38AB2
    http_status_code: 405
    request_id: SR0RTXK1WMF38AB2
    retry_attempts: 0

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[tremble]

Thanks for the report, could you test and see if swapping "UnsupportedArgument" for "MethodNotAllowed" (using the fix from #2202) fixes your issue?

[vonschultz]

Yes, @tremble, I can confirm that changing UnsupportedArgument to MethodNotAllowed in aws/plugins/modules/s3_bucket.py (from #2202) does fix the issue.

[tremble]

@vonschultz There's a slightly different fix in #2267, if you're able to test it and comment over there it would be helpful.

Hopefully we'll be releasing 8.2.0 in the next week or so...