Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle MD5 from etag when upper-case are used. #1286

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Handle MD5 from etag when upper-case are used. #1286

wants to merge 1 commit into from

Conversation

leffss
Copy link

@leffss leffss commented Nov 1, 2022

Some S3-compatible providers, like AlibabaCloud, return etag in upper-case (e.g. A256A58D6429BF79ABBC0DB3CEF2FC71)

@leffss
Handle MD5 from etag when upper-case are used.
d38caa2 
Some S3-compatible providers, like AlibabaCloud, return etag in upper-case (e.g. A256A58D6429BF79ABBC0DB3CEF2FC71)
@fviard
Copy link
Contributor

fviard commented Nov 5, 2022

Thank you for your PR.
It is not the first time that we get a suggestion like this, but I'm not sure that it is a good solution, because this would impact all the other s3 compatible services even if they don't need that.

To solve this problem, what I'm thinking about is to allow some "tweaking" per service.
To help me with that, would it be possible possible for you to run some commands on your Alibabacloud with the "-d" (ie debug) flag on, and to show me an example of response from the server?
I'm especially interested in the "server" string in the header of the response.
Thanks

@leffss
Copy link
Author

leffss commented Nov 6, 2022

Thank you for your PR. It is not the first time that we get a suggestion like this, but I'm not sure that it is a good solution, because this would impact all the other s3 compatible services even if they don't need that.

To solve this problem, what I'm thinking about is to allow some "tweaking" per service. To help me with that, would it be possible possible for you to run some commands on your Alibabacloud with the "-d" (ie debug) flag on, and to show me an example of response from the server? I'm especially interested in the "server" string in the header of the response. Thanks

.s3cfg setting:

[default]
access_key = access_key
secret_key = secret_key
bucket_location = oss-cn-hongkong
host_base = oss-cn-hongkong.aliyuncs.com
host_bucket = %(bucket)s.oss-cn-hongkong.aliyuncs.com
default_mime_type = binary/octet-stream
encoding = UTF-8
enable_multipart = True
multipart_max_chunks = 10000
multipart_chunk_size_mb = 128
recursive = True
recv_chunk = 65536
send_chunk = 65536
server_side_encryption = False
signature_v2 = False
socket_timeout = 300
use_mime_magic = False
verbosity = DEBUG
#verbosity = WARNING

debug log:

$ s3cmd --no-check-md5 put requirements.txt s3://leffss-gitlab-backup-upload/test/requirements.txt
DEBUG: Updating Config.Config cache_file -> 
DEBUG: Updating Config.Config follow_symlinks -> False
DEBUG: Command: put
INFO: Cache file not found or empty, creating/populating it.
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
INFO: Compiling list of local files...
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Unicodising b'requirements.txt' using UTF-8
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Unicodising b'' using UTF-8
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Unicodising b'requirements.txt' using UTF-8
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Applying --exclude/--include
DEBUG: CHECK: 'requirements.txt'
DEBUG: PASS: 'requirements.txt'
INFO: Running stat() and reading/calculating MD5 values on 1 files, this may take some time...
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
INFO: Summary: 1 local files to upload
DEBUG: String 'b'root'' encoded to 'root'
DEBUG: String 'b'root'' encoded to 'root'
DEBUG: attr_header: {'x-amz-meta-s3cmd-attrs': 'atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root'}
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: CreateRequest: resource[uri]=/test/requirements.txt
upload: 'requirements.txt' -> 's3://leffss-gitlab-backup-upload/test/requirements.txt'  [1 of 1]
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: canonical_headers = content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042108Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
PUT
/test/requirements.txt

content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042108Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
----------------------
DEBUG: signature-v4 headers: {'content-length': '13', 'content-type': 'text/plain', 'x-amz-meta-s3cmd-attrs': 'atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root', 'x-amz-storage-class': 'STANDARD', 'x-amz-date': '20221106T042108Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=LTAI5tSnhs1a7aCvGwrxbfGR/20221106/oss-cn-hongkong/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=8c254701dbd48d21548876ca8b5f05407f3ffb9b8dfef1dda31a1c005c49b06e', 'x-amz-content-sha256': '91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465'}
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: ConnMan.get(): creating new connection: https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: Using ca_certs_file None
DEBUG: Using ssl_client_cert_file None
DEBUG: Using ssl_client_key_file None
DEBUG: httplib.HTTPSConnection() has both context and check_hostname
DEBUG: non-proxied HTTPSConnection(leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com, None)
DEBUG: format_uri(): /test/requirements.txt
 13 of 13   100% in    0s    50.96 B/sDEBUG: ConnMan.put(): connection put back to pool (https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#1)
DEBUG: Response:
{'data': b'',
 'headers': {'connection': 'keep-alive',
             'content-length': '0',
             'date': 'Sun, 06 Nov 2022 04:21:08 GMT',
             'etag': '"54C15BBE5108CB3CCDEF41274654127F"',
             'server': 'AliyunOSS',
             'x-amz-request-id': '636736341F85633435B2C303',
             'x-oss-hash-crc64ecma': '11264086281538792249',
             'x-oss-server-time': '50'},
 'reason': 'OK',
 'size': 13,
 'status': 200}
 13 of 13   100% in    0s    32.03 B/s  done
DEBUG: MD5 sums: computed=54c15bbe5108cb3ccdef41274654127f, received=54C15BBE5108CB3CCDEF41274654127F
WARNING: MD5 Sums don't match!
WARNING: Retrying upload of requirements.txt
upload: 'requirements.txt' -> 's3://leffss-gitlab-backup-upload/test/requirements.txt'  [1 of 1]
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: canonical_headers = content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042108Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
PUT
/test/requirements.txt

content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042108Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
----------------------
DEBUG: signature-v4 headers: {'content-length': '13', 'content-type': 'text/plain', 'x-amz-meta-s3cmd-attrs': 'atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root', 'x-amz-storage-class': 'STANDARD', 'x-amz-date': '20221106T042108Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=LTAI5tSnhs1a7aCvGwrxbfGR/20221106/oss-cn-hongkong/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=8c254701dbd48d21548876ca8b5f05407f3ffb9b8dfef1dda31a1c005c49b06e', 'x-amz-content-sha256': '91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465'}
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: ConnMan.get(): re-using connection: https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#1
DEBUG: format_uri(): /test/requirements.txt
 13 of 13   100% in    0s    14.51 KB/sDEBUG: ConnMan.put(): connection put back to pool (https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#2)
DEBUG: Response:
{'data': b'',
 'headers': {'connection': 'keep-alive',
             'content-length': '0',
             'date': 'Sun, 06 Nov 2022 04:21:08 GMT',
             'etag': '"54C15BBE5108CB3CCDEF41274654127F"',
             'server': 'AliyunOSS',
             'x-amz-request-id': '636736341F8563343580C403',
             'x-oss-hash-crc64ecma': '11264086281538792249',
             'x-oss-server-time': '4'},
 'reason': 'OK',
 'size': 13,
 'status': 200}
 13 of 13   100% in    0s    89.13 B/s  done
DEBUG: MD5 sums: computed=54c15bbe5108cb3ccdef41274654127f, received=54C15BBE5108CB3CCDEF41274654127F
WARNING: MD5 Sums don't match!
WARNING: Retrying upload of requirements.txt
upload: 'requirements.txt' -> 's3://leffss-gitlab-backup-upload/test/requirements.txt'  [1 of 1]
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: canonical_headers = content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042109Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
PUT
/test/requirements.txt

content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042109Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
----------------------
DEBUG: signature-v4 headers: {'content-length': '13', 'content-type': 'text/plain', 'x-amz-meta-s3cmd-attrs': 'atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root', 'x-amz-storage-class': 'STANDARD', 'x-amz-date': '20221106T042109Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=LTAI5tSnhs1a7aCvGwrxbfGR/20221106/oss-cn-hongkong/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=465ed95b1adc1191cdc19fccbc93817886ed9cd779964347ae90d9b2800b6c42', 'x-amz-content-sha256': '91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465'}
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: ConnMan.get(): re-using connection: https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#2
DEBUG: format_uri(): /test/requirements.txt
 13 of 13   100% in    0s    10.30 KB/sDEBUG: ConnMan.put(): connection put back to pool (https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#3)
DEBUG: Response:
{'data': b'',
 'headers': {'connection': 'keep-alive',
             'content-length': '0',
             'date': 'Sun, 06 Nov 2022 04:21:08 GMT',
             'etag': '"54C15BBE5108CB3CCDEF41274654127F"',
             'server': 'AliyunOSS',
             'x-amz-request-id': '636736341F8563343536C503',
             'x-oss-hash-crc64ecma': '11264086281538792249',
             'x-oss-server-time': '6'},
 'reason': 'OK',
 'size': 13,
 'status': 200}
 13 of 13   100% in    0s    89.14 B/s  done
DEBUG: MD5 sums: computed=54c15bbe5108cb3ccdef41274654127f, received=54C15BBE5108CB3CCDEF41274654127F
WARNING: MD5 Sums don't match!
WARNING: Retrying upload of requirements.txt
upload: 'requirements.txt' -> 's3://leffss-gitlab-backup-upload/test/requirements.txt'  [1 of 1]
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: canonical_headers = content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042109Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
PUT
/test/requirements.txt

content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042109Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
----------------------
DEBUG: signature-v4 headers: {'content-length': '13', 'content-type': 'text/plain', 'x-amz-meta-s3cmd-attrs': 'atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root', 'x-amz-storage-class': 'STANDARD', 'x-amz-date': '20221106T042109Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=LTAI5tSnhs1a7aCvGwrxbfGR/20221106/oss-cn-hongkong/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=465ed95b1adc1191cdc19fccbc93817886ed9cd779964347ae90d9b2800b6c42', 'x-amz-content-sha256': '91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465'}
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: ConnMan.get(): re-using connection: https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#3
DEBUG: format_uri(): /test/requirements.txt
 13 of 13   100% in    0s    15.10 KB/sDEBUG: ConnMan.put(): connection put back to pool (https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#4)
DEBUG: Response:
{'data': b'',
 'headers': {'connection': 'keep-alive',
             'content-length': '0',
             'date': 'Sun, 06 Nov 2022 04:21:09 GMT',
             'etag': '"54C15BBE5108CB3CCDEF41274654127F"',
             'server': 'AliyunOSS',
             'x-amz-request-id': '636736341F85633435E3C503',
             'x-oss-hash-crc64ecma': '11264086281538792249',
             'x-oss-server-time': '4'},
 'reason': 'OK',
 'size': 13,
 'status': 200}
 13 of 13   100% in    0s    90.66 B/s  done
DEBUG: MD5 sums: computed=54c15bbe5108cb3ccdef41274654127f, received=54C15BBE5108CB3CCDEF41274654127F
WARNING: MD5 Sums don't match!
WARNING: Retrying upload of requirements.txt
upload: 'requirements.txt' -> 's3://leffss-gitlab-backup-upload/test/requirements.txt'  [1 of 1]
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: canonical_headers = content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042109Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
PUT
/test/requirements.txt

content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042109Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
----------------------
DEBUG: signature-v4 headers: {'content-length': '13', 'content-type': 'text/plain', 'x-amz-meta-s3cmd-attrs': 'atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root', 'x-amz-storage-class': 'STANDARD', 'x-amz-date': '20221106T042109Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=LTAI5tSnhs1a7aCvGwrxbfGR/20221106/oss-cn-hongkong/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=465ed95b1adc1191cdc19fccbc93817886ed9cd779964347ae90d9b2800b6c42', 'x-amz-content-sha256': '91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465'}
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: ConnMan.get(): re-using connection: https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#4
DEBUG: format_uri(): /test/requirements.txt
 13 of 13   100% in    0s    13.34 KB/sDEBUG: ConnMan.put(): connection put back to pool (https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#5)
DEBUG: Response:
{'data': b'',
 'headers': {'connection': 'keep-alive',
             'content-length': '0',
             'date': 'Sun, 06 Nov 2022 04:21:09 GMT',
             'etag': '"54C15BBE5108CB3CCDEF41274654127F"',
             'server': 'AliyunOSS',
             'x-amz-request-id': '636736351F85633435ADC603',
             'x-oss-hash-crc64ecma': '11264086281538792249',
             'x-oss-server-time': '4'},
 'reason': 'OK',
 'size': 13,
 'status': 200}
 13 of 13   100% in    0s    90.39 B/s  done
DEBUG: MD5 sums: computed=54c15bbe5108cb3ccdef41274654127f, received=54C15BBE5108CB3CCDEF41274654127F
WARNING: MD5 Sums don't match!
WARNING: Retrying upload of requirements.txt
upload: 'requirements.txt' -> 's3://leffss-gitlab-backup-upload/test/requirements.txt'  [1 of 1]
DEBUG: DeUnicodising 'requirements.txt' using UTF-8
DEBUG: Using signature v4
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: canonical_headers = content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042109Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

DEBUG: Canonical Request:
PUT
/test/requirements.txt

content-length:13
content-type:text/plain
host:leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
x-amz-content-sha256:91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
x-amz-date:20221106T042109Z
x-amz-meta-s3cmd-attrs:atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root
x-amz-storage-class:STANDARD

content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class
91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465
----------------------
DEBUG: signature-v4 headers: {'content-length': '13', 'content-type': 'text/plain', 'x-amz-meta-s3cmd-attrs': 'atime:1667636059/ctime:1667299379/gid:0/gname:root/mode:33188/mtime:1667299379/uid:0/uname:root', 'x-amz-storage-class': 'STANDARD', 'x-amz-date': '20221106T042109Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=LTAI5tSnhs1a7aCvGwrxbfGR/20221106/oss-cn-hongkong/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3cmd-attrs;x-amz-storage-class,Signature=465ed95b1adc1191cdc19fccbc93817886ed9cd779964347ae90d9b2800b6c42', 'x-amz-content-sha256': '91bd5e257c4a6ff6d6df14ac01ba7b3d14d20bdb8feb563d131b4ac4f9815465'}
DEBUG: get_hostname(leffss-gitlab-backup-upload): leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com
DEBUG: ConnMan.get(): re-using connection: https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#5
DEBUG: format_uri(): /test/requirements.txt
 13 of 13   100% in    0s    11.94 KB/sDEBUG: ConnMan.put(): connection put back to pool (https://leffss-gitlab-backup-upload.oss-cn-hongkong.aliyuncs.com#6)
DEBUG: Response:
{'data': b'',
 'headers': {'connection': 'keep-alive',
             'content-length': '0',
             'date': 'Sun, 06 Nov 2022 04:21:09 GMT',
             'etag': '"54C15BBE5108CB3CCDEF41274654127F"',
             'server': 'AliyunOSS',
             'x-amz-request-id': '636736351F856334357FC703',
             'x-oss-hash-crc64ecma': '11264086281538792249',
             'x-oss-server-time': '4'},
 'reason': 'OK',
 'size': 13,
 'status': 200}
 13 of 13   100% in    0s    90.43 B/s  done
DEBUG: MD5 sums: computed=54c15bbe5108cb3ccdef41274654127f, received=54C15BBE5108CB3CCDEF41274654127F
WARNING: MD5 Sums don't match!
WARNING: Too many failures. Giving up on 'requirements.txt'
ERROR: Upload of 'requirements.txt' failed too many times (Last reason: Too many failures. Giving up on 'requirements.txt')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
investigation-needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@leffss @fviard