Skip to content

Commit

Permalink
Avoid logging signable body by default whose data can be very large (#…
Browse files Browse the repository at this point in the history
…3917)

## Motivation and Context
While investigating a connect timeout issue for uploading object(s) in
[`aws-s3-transfer-manager-rs`](https://github.com/awslabs/aws-s3-transfer-manager-rs),
we saw that the size of trace log was about 70 GB and that the last 1 GB
only had 30 lines, with each line having couple MB's body to be logged
(due to [this
location](https://github.com/awslabs/aws-sdk-rust/blob/953cd6c7af04f02938a0dcf36f793ebe7a06cc57/sdk/aws-sigv4/src/http_request/sign.rs#L224)).

## Description
This PR disables logging the actual body data in `SignableBody` by
default. Customers can set the `LOG_SIGNABLE_BODY` environment variable
to log the body data if they want to, as described in the comment within
the `Debug` implementation.

## Testing
- Added a small unit test
- Tests in CI

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
  • Loading branch information
ysaito1001 authored Nov 20, 2024
1 parent df77d5f commit aac9bec
Show file tree
Hide file tree
Showing 3 changed files with 47 additions and 3 deletions.
2 changes: 1 addition & 1 deletion aws/rust-runtime/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion aws/rust-runtime/aws-sigv4/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "aws-sigv4"
version = "1.2.5"
version = "1.2.6"
authors = ["AWS Rust SDK Team <[email protected]>", "David Barsky <[email protected]>"]
description = "SigV4 signer for HTTP requests and Event Stream messages."
edition = "2021"
Expand Down
46 changes: 45 additions & 1 deletion aws/rust-runtime/aws-sigv4/src/http_request/sign.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ use std::borrow::Cow;
use std::fmt::{Debug, Formatter};
use std::str;

const LOG_SIGNABLE_BODY: &str = "LOG_SIGNABLE_BODY";

/// Represents all of the information necessary to sign an HTTP request.
#[derive(Debug)]
#[non_exhaustive]
Expand Down Expand Up @@ -72,7 +74,7 @@ impl<'a> SignableRequest<'a> {
}

/// A signable HTTP request body
#[derive(Debug, Clone, Eq, PartialEq)]
#[derive(Clone, Eq, PartialEq)]
#[non_exhaustive]
pub enum SignableBody<'a> {
/// A body composed of a slice of bytes
Expand All @@ -93,6 +95,30 @@ pub enum SignableBody<'a> {
StreamingUnsignedPayloadTrailer,
}

/// Formats the value using the given formatter. To print the body data, set the environment variable `LOG_SIGNABLE_BODY=true`.
impl<'a> Debug for SignableBody<'a> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
let should_log_signable_body = std::env::var(LOG_SIGNABLE_BODY)
.map(|v| v.eq_ignore_ascii_case("true"))
.unwrap_or_default();
match self {
Self::Bytes(arg0) => {
if should_log_signable_body {
f.debug_tuple("Bytes").field(arg0).finish()
} else {
let redacted = format!("** REDACTED **. To print {body_size} bytes of raw data, set environment variable `LOG_SIGNABLE_BODY=true`", body_size = arg0.len());
f.debug_tuple("Bytes").field(&redacted).finish()
}
}
Self::UnsignedPayload => write!(f, "UnsignedPayload"),
Self::Precomputed(arg0) => f.debug_tuple("Precomputed").field(arg0).finish(),
Self::StreamingUnsignedPayloadTrailer => {
write!(f, "StreamingUnsignedPayloadTrailer")
}
}
}
}

impl SignableBody<'_> {
/// Create a new empty signable body
pub fn empty() -> SignableBody<'static> {
Expand Down Expand Up @@ -1121,4 +1147,22 @@ mod tests {
request.uri().path_and_query().unwrap().to_string()
);
}

#[test]
fn test_debug_signable_body() {
let sut = SignableBody::Bytes(b"hello signable body");
assert_eq!(
"Bytes(\"** REDACTED **. To print 19 bytes of raw data, set environment variable `LOG_SIGNABLE_BODY=true`\")",
format!("{sut:?}")
);

let sut = SignableBody::UnsignedPayload;
assert_eq!("UnsignedPayload", format!("{sut:?}"));

let sut = SignableBody::Precomputed("precomputed".to_owned());
assert_eq!("Precomputed(\"precomputed\")", format!("{sut:?}"));

let sut = SignableBody::StreamingUnsignedPayloadTrailer;
assert_eq!("StreamingUnsignedPayloadTrailer", format!("{sut:?}"));
}
}

0 comments on commit aac9bec

Please sign in to comment.